WO2012077533A1

WO2012077533A1 - Image decoding device, motion vector decoding method, image encoding device, and motion vector encoding method

Info

Publication number: WO2012077533A1
Application number: PCT/JP2011/077510
Authority: WO
Inventors: 佐藤　数史
Original assignee: ソニー株式会社
Priority date: 2010-12-06
Filing date: 2011-11-29
Publication date: 2012-06-14
Also published as: US20130259134A1; CN103238329A; JP2012124591A

Abstract

In the present invention, a reversible decoding unit (52) acquires, from compressed image information, horizontal predicted block information indicating a block wherein motion vector information has been selected as horizontal predicted motion vector information from a block to be decoded and an adjacent block that has been decoded, and vertical predicted block information indicating a block wherein motion vector information has been selected as vertical predicted motion vector information. A predicted motion vector information setting unit (73) sets motion vector information of the block indicated by the horizontal predicted block information as horizontal predicted motion vector information, and sets motion vector information of the block indicated by the vertical predicted block information as vertical predicted motion vector information. Using the set horizontal predicted motion vector information and vertical predicted motion vector information, the motion vector information generating unit of a motion compensation unit (72) generates motion vector information of the block to be decoded. As a result, encoding efficiency is increased.

Description

Image decoding apparatus and motion vector decoding method, image encoding apparatus and motion vector encoding method

This technology relates to an image decoding device and a motion vector decoding method, and an image encoding device and a motion vector encoding method. Specifically, the efficiency in moving picture encoding is improved.

In recent years, image information is handled as digital, and at that time, a device that transmits and stores information with high efficiency, for example, a device that complies with a system such as MPEG that compresses by orthogonal transform such as discrete cosine transform and motion compensation, It is becoming popular in general households.

In particular, MPEG2 (ISO / IEC13818-2) is defined as a general-purpose image encoding method, and is currently widely used in a wide range of applications for professional use and consumer use. By using the MPEG2 compression method, for example, a standard resolution interlaced scanned image having 720 × 480 pixels can be assigned a code amount (bit rate) of 4 to 8 Mbps, thereby realizing a high compression ratio and good image quality. It is. Further, in the case of a high-resolution interlaced scanned image having 1920 × 1088 pixels, it is possible to realize a high compression ratio and good image quality by assigning a code amount of 18 to 22 Mbps.

Compared to conventional encoding methods such as MPEG2 and MPEG4, a large amount of computation is required for encoding and decoding, but standardization that realizes higher encoding efficiency is Joint Model of Enhanced-Compression Video Coding. H. It became an international standard under the names of H.264 and MPEG-4 Part 10 (hereinafter referred to as “H.264 / AVC (Advanced Video Coding)”).

H. In H.264 / AVC, as shown in FIG. 1 (A), one macro block composed of 16 × 16 pixels is replaced with any block of 16 × 16, 16 × 8, 8 × 16, or 8 × 8. It is possible to have independent motion vector information divided into sizes. Further, the 8 × 8 pixel sub-macroblock is divided into motion compensation block sizes of 8 × 8, 8 × 4, 4 × 8, and 4 × 4 as shown in FIG. , Each can have independent motion vector information. In MPEG-2, the unit of motion prediction / compensation processing is 16 × 16 pixels in the frame motion compensation mode, and 16 units for each of the first field and the second field in the field motion compensation mode. Motion prediction / compensation processing is performed in units of × 8 pixels.

H. In H.264 / AVC, such motion prediction / compensation processing is performed, so that a large amount of motion vector information is generated. If this is encoded as it is, encoding efficiency is reduced.

As a technique for solving this problem, H. In H.264 / AVC, the amount of motion vector information is reduced using the following median prediction.

In FIG. 2, a block E is a target block to be encoded, and blocks A to D are already encoded and are adjacent to the target block E.

Now, let X = A, B, C, D, E and the motion vector information for the block X be represented by mvX.

Using the motion vector information regarding the blocks A, B, and C, predicted motion vector information pmvE for the target block E is generated as shown in Equation (1) by median prediction.
pmvE = med (mvA, mvB, mvC) (1)

If the information about the adjacent block C cannot be obtained because it is the edge of the image frame, the information about the adjacent block D is substituted.

Data mvdE encoded in the image compression information as motion vector information for the target block E is generated as shown in Expression (2) using pmvE.
mvdE = mvE−pmvE (2)
Note that the actual processing is performed independently for each of the horizontal and vertical components of the motion vector information.

H. In H.264 / AVC, a multi-reference frame method is defined. Using FIG. The multiple reference frame system defined in H.264 / AVC will be described.

In MPEG2 or the like, in the case of a P picture, motion prediction / compensation processing is performed with reference to only one reference frame stored in the frame memory. However, H. In H.264 / AVC, as shown in FIG. 3, it is possible to store a plurality of reference frames in a memory and refer to a different memory for each block.

By the way, the amount of information in motion vector information in a B picture is enormous. In H.264 / AVC, a mode called a direct mode is provided. In the direct mode, the motion vector information is not stored in the image compression information, and the decoding apparatus extracts the motion vector information of the block from the motion vector information of the surrounding or anchor block (Co-Located Block). The anchor block is a block having the same xy coordinates as the target block in the reference image.

There are two types of direct mode: spatial direct mode (Spatial Direct Mode) and temporal direct mode (Temporal Direct Mode), which can be switched for each slice.

In the spatial direct mode, as shown in Expression (3), the motion vector information pmvE generated by the median prediction is set as motion vector information mvE applied to the block.
mvE = pmvE (3)

Next, a temporal direct mode will be described with reference to FIG. In FIG. 4, a block at the same space address as the current block in the L0 reference picture is an anchor block, and motion vector information in the anchor block is a motion “mvcol”. Also, the distance on the time axis between the current picture and the L0 reference picture is “TDB”, and the distance on the time axis between the L0 reference picture and the L1 reference picture is “TDD”. In this case, L0 motion vector information mvL0 and L1 motion vector information mvL1 in the picture are calculated as in equations (4) and (5).
mvL0 = (TDB / TDD) mvcol (4)
mvL1 = ((TDD−TDB) / TDD) mvcol (5)
In the image compression information, since there is no information indicating the distance on the time axis, the calculation is performed using POC (Picture Order Count) in equations (4) and (5).

In the AVC image compression information, the direct mode can be defined in units of 16 × 16 pixel macroblocks or in units of 8 × 8 pixel sub-macroblocks.

Incidentally, Non-Patent Document 1 has been proposed to improve the encoding of motion vector information using median prediction as shown in FIG. In Non-Patent Document 1, it is possible to adaptively use either temporal prediction motion vector information or spatio-temporal prediction motion vector information in addition to spatial prediction motion vector information obtained by median prediction.

That is, in FIG. 5, the motion vector information mvcol is the motion vector information for the anchor block for the target block. Also, the motion vector information mvtk (k = 0 to 8) is used as the motion vector information of the surrounding blocks.

The temporal prediction motion vector information mvtm is generated from five pieces of motion vector information using, for example, Expression (6). The temporal prediction motion vector information mvtm may be generated from nine motion vectors using equation (7).
mvtm5 = med (mvcol, mvt0,... mvt3) (6)
mvtm9 = med (mvcol, mvt0,... mvt7) (7)

Also, the spatio-temporal predicted motion vector information mvspt is generated from the five motion vector information using Expression (8).
mvspt = med (mvcol, mvcol, mvA, mvB, mvC) (8)

In an image processing apparatus that encodes image information, a cost function value is calculated for each block when using each predicted motion vector information, and optimal predicted motion vector information is selected. In the image compression information, for example, a flag that can identify which prediction motion vector information is used for each block is transmitted.

Also, for large picture frames such as UHD (Ultra High Definition: 4000 pixels × 2000 pixels), MPEG2 and H.264 are used. The macroblock size of 16 pixels × 16 images defined by H.264 / AVC may not be optimal. For example, in a large image frame, it may be possible to increase the encoding efficiency by increasing the macroblock size. Therefore, as shown in Non-Patent Document 2, coding unit CU (Coding Unit) is defined in HEVC (High Efficiency Video Coding) which is a next-generation encoding method. In Non-Patent Document 2, the maximum size (LCU = Largest Coding Unit) and the minimum size (SCU = Smallest Coding Unit) of the coding unit CU are defined in the SPS (Sequence Parameter Set) of the image compression information to be output. . Furthermore, in each LCU, it is possible to divide into smaller sized coding units CU by setting split-flag = 1 within a range not smaller than the size of the SCU.

FIG. 6 illustrates the hierarchical structure of the coding unit CU. FIG. 6 shows a case where the maximum size is 128 pixels × 128 pixels and the depth (Depth) of the hierarchy is “5”. For example, when the layer depth is “0”, a block of 2N × 2N (N = 64 pixels) is set as the coding unit CU0. Also, if split flag = 1, the coding unit CU0 is divided into four independent N × N blocks, and the N × N block is a block in the next lower layer. That is, the depth of the hierarchy is “1”, and a 2N × 2N (N = 32 pixels) block is the coding unit CU1. Similarly, when split flag = 1, it is divided into four independent blocks. Further, in the case of the depth “4” which is the deepest hierarchy, a block of 2N × 2N (N = 4 pixels) is set as the coding unit CU4, and 8 × 8 pixels is the minimum size of the coding unit CU. In HEVC, a prediction unit (PU: Prediction Unit) that is a basic unit for prediction by dividing a coding unit is also defined.

By the way, Non-Patent Document 1 has a problem that since it cannot have prediction information independently for the horizontal direction and the vertical direction of the motion vector component, sufficient improvement in coding efficiency cannot be realized. Yes. For example, when there are three types of candidates in the horizontal direction and three types in the vertical direction, there are nine (3 × 3) combinations of candidates in the horizontal direction and the vertical direction. A method for performing the conversion processing is also conceivable. However, if there are many combinations, there are problems that the number of types of flags increases and the amount of codes indicating the flags increases.

Therefore, it is an object of this technique to provide an image decoding device and a motion vector decoding method, an image encoding device and a motion vector encoding method that can improve the encoding efficiency.

According to a first aspect of the present technology, horizontal prediction block information indicating a block in which motion vector information is selected as horizontal prediction motion vector information from a decoded block adjacent to the target block, and motion vector information is vertical prediction motion vector information. A lossless decoding unit that obtains vertical prediction block information indicating a block selected as image compression information, and sets motion vector information of the block indicated by the horizontal prediction block information as horizontal prediction motion vector information, and A prediction motion vector information setting unit that sets motion vector information of a block indicated by prediction block information as the vertical prediction motion vector information, and the horizontal prediction motion vector information and vertical prediction set by the prediction motion vector information setting unit The motion vector of the target block using motion vector information In the image decoding apparatus and a motion vector information generation unit for generating information.

This technology divides input image data into a plurality of pixel blocks, detects motion vector information for each block, and performs motion compensation predictive coding to perform image decoding processing for decoding image compression information generated In the apparatus, horizontal prediction block information indicating a block in which motion vector information is selected as horizontal prediction motion vector information from a decoded block adjacent to the target block, and a block in which motion vector information is selected as vertical prediction motion vector information. The predicted vertical block information shown is acquired from the compressed image information. The motion vector information of the block indicated by the horizontal prediction block information is set as horizontal prediction motion vector information, and the motion vector information of the block indicated by the vertical prediction block information is set as vertical prediction motion vector information. Motion vector information of the target block is generated using the set horizontal predicted motion vector information and vertical predicted motion vector information.

Also, horizontal and vertical motion vector information indicating motion vector information selected from the decoded block adjacent to the horizontal and vertical components of the motion vector information of the target block or the horizontal and vertical motion vector information of the target block. Identification information indicating which one of the two is used is acquired from the image compression information. Based on this identification information, horizontal prediction motion vector information and vertical prediction motion vector information, or horizontal and vertical prediction motion vector information are set, and motion vector information of the target block is generated.

According to a second aspect of this technique, horizontal prediction block information indicating a block in which motion vector information is selected as horizontal prediction motion vector information from a decoded block adjacent to the target block, and motion vector information is vertical prediction motion vector information. Obtaining from the image compression information vertical prediction block information indicating the selected block, and setting motion vector information of the block indicated by the horizontal prediction block information as horizontal prediction motion vector information, and the vertical prediction block information And setting the motion vector information of the target block using the set horizontal predicted motion vector information and vertical predicted motion vector information. A motion vector information decoding method comprising:

According to a third aspect of the present technology, for each of a horizontal component and a vertical component of the motion vector information of the target block, motion vector information is selected from an encoded block adjacent to the target block to generate horizontal predicted motion vector information. And a predicted motion vector information setting unit that generates horizontal predicted block information and vertical predicted block information indicating a block in which the motion vector information is selected.

This technique divides input image data into a plurality of pixel blocks, detects motion vector information for each block, and performs motion compensation predictive coding. For each component, motion vector information is selected from an encoded block adjacent to the target block, and horizontal prediction motion vector information and vertical prediction motion vector information are set. For example, for the horizontal component of motion vector information obtained by performing motion search in the optimal prediction mode that minimizes the cost function value, the motion vector information of an adjacent encoded block that has the highest encoding efficiency is obtained. It is selected and set as horizontal prediction motion vector information. Also, with respect to the vertical component of the motion vector information obtained by performing the motion search in the optimal prediction mode, the motion vector information of the adjacent encoded block having the highest encoding efficiency is selected and the vertical prediction motion vector Set as information. Using this horizontal predicted motion vector information and vertical predicted motion vector information, the motion vector information of the target block is compressed. Also, horizontal prediction block information and vertical prediction block information indicating a block for which motion vector information is selected are generated, and the horizontal prediction block information and the vertical prediction block information are included in the image compression information.

Also, with respect to the horizontal and vertical components of the motion vector information of the target block, the setting is made such that the motion vector information selected from the encoded block adjacent to the target block is the horizontal and vertical predicted motion vector information, or the horizontal predicted motion vector information The setting of the vertical prediction motion vector information can be switched for each picture or each slice. For example, horizontal prediction motion vector information and vertical prediction motion vector information are set for a P picture, and horizontal vertical prediction motion vector information is set for a B picture. Further, identification information indicating whether horizontal predicted motion vector information, vertical predicted motion vector information, or horizontal / vertical predicted motion vector information is used is provided in the image compression information.

Further, for example, codes are assigned to the horizontal prediction block information and the vertical prediction block information, respectively, and the codes assigned to the horizontal prediction block information and the vertical prediction block information are included in the image compression information. Furthermore, when performing the encoding process of the motion vector information detected based on the image data produced | generated by the imaging device, code allocation is performed based on the motion detection result of an imaging device.

According to a fourth aspect of the present technology, for each of a horizontal component and a vertical component of the motion vector information of the target block, motion vector information is selected from an encoded block adjacent to the target block to generate horizontal predicted motion vector information. The motion vector information encoding method includes a step of setting the vertical prediction motion vector information and generating horizontal prediction block information and vertical prediction block information indicating a block for which the motion vector information is selected.

According to this technique, for each of the horizontal component and the vertical component of the motion vector information of the target block, the motion vector information is selected from the encoded block adjacent to the target block, and the horizontal prediction motion vector information and the vertical prediction motion are selected. Vector information is set, respectively, and the motion vector information of the target block is compressed using the set horizontal predicted motion vector information and vertical predicted motion vector information. Also, horizontal prediction block information and vertical prediction block information indicating a block for which motion vector information is selected are generated. Further, the motion vector information is decoded based on the horizontal prediction block information and the vertical prediction block information. For this reason, for example, horizontal prediction motion vector information and vertical prediction motion vector information with horizontal prediction block information and vertical prediction block information that have a data amount smaller than a flag corresponding to a combination of horizontal prediction motion vector information and vertical prediction motion vector information candidates Can be set, and the encoding efficiency can be improved.

H. It is a figure which shows the block in H.264 / AVC. It is a figure for demonstrating median prediction. It is a figure for demonstrating Multi-Reference | standard Frame system. It is a figure for demonstrating time direct mode. It is a figure for demonstrating temporal prediction motion vector information and spatiotemporal prediction motion vector information. It is the figure which illustrated the hierarchical structure of coding unit CU. It is a figure which shows the structure of an image coding apparatus. It is a figure which shows the structure of a motion prediction / compensation part and a prediction motion vector information setting part. It is a figure for demonstrating the motion prediction and compensation process of 1/4 pixel precision. It is a flowchart which shows operation | movement of an image coding apparatus. It is a flowchart which shows a prediction process. It is a flowchart which shows an intra prediction process. It is a flowchart which shows the inter prediction process. It is a flowchart which shows a prediction motion vector information setting process. It is a figure which shows the structure of an image decoding apparatus. It is a figure which shows the structure of a motion compensation part and a prediction motion vector information setting part. It is a flowchart which shows operation | movement of an image decoding apparatus. It is a flowchart which shows a prediction image generation process. It is a flowchart which shows the inter estimated image production | generation process. It is a flowchart which shows a motion vector information reconstruction process. It is a figure which shows the other structure of the prediction motion vector information setting part used with an image coding apparatus. It is a figure which shows the other structure of the prediction motion vector information setting part used with an image decoding apparatus. It is the figure which illustrated schematic structure of the computer apparatus. It is the figure which illustrated schematic structure of the television apparatus. It is the figure which illustrated schematic structure of the mobile phone. It is the figure which illustrated schematic structure of the recording / reproducing apparatus. It is the figure which illustrated schematic structure of the imaging device.

Hereinafter, embodiments for carrying out the technology will be described. The description will be given in the following order.
1. 1. Configuration of image encoding device 2. Operation of image encoding device 3. Configuration of image decoding apparatus 4. Operation of image decoding apparatus 5. Other configuration of prediction motion vector information setting unit In the case of software processing When applied to electronic equipment

<1. Configuration of Image Encoding Device>
FIG. 7 shows the configuration of the image encoding device. The image encoding device 10 includes an analog / digital conversion unit (A / D conversion unit) 11, a screen rearrangement buffer 12, a subtraction unit 13, an orthogonal transformation unit 14, a quantization unit 15, a lossless encoding unit 16, and a storage buffer 17. The rate control unit 18 is provided. Furthermore, the image encoding device 10 includes an inverse quantization unit 21, an inverse orthogonal transform unit 22, an addition unit 23, a deblocking filter 24, a frame memory 25, an intra prediction unit 31, a motion prediction / compensation unit 32, and predicted motion vector information. A setting unit 33 and a predicted image / optimum mode selection unit 35 are provided.

The A / D converter 11 converts an analog image signal into digital image data and outputs the digital image data to the screen rearrangement buffer 12.

The screen rearrangement buffer 12 rearranges the frames of the image data output from the A / D conversion unit 11. The screen rearrangement buffer 12 rearranges the frames in accordance with a GOP (Group of Pictures) structure related to the encoding process, and subtracts the image data after the rearrangement, the intra prediction unit 31, and the motion prediction / compensation unit. 32.

The subtraction unit 13 is supplied with the image data output from the screen rearrangement buffer 12 and the predicted image data selected by the predicted image / optimum mode selection unit 35 described later. The subtraction unit 13 calculates prediction error data that is a difference between the image data output from the screen rearrangement buffer 12 and the prediction image data supplied from the prediction image / optimum mode selection unit 35, and sends the prediction error data to the orthogonal transformation unit 14. Output.

The orthogonal transform unit 14 performs orthogonal transform processing such as discrete cosine transform (DCT) and Karoonen-Loeve transform on the prediction error data output from the subtraction unit 13. The orthogonal transform unit 14 outputs transform coefficient data obtained by performing the orthogonal transform process to the quantization unit 15.

The quantization unit 15 is supplied with transform coefficient data output from the orthogonal transform unit 14 and a rate control signal from a rate control unit 18 described later. The quantization unit 15 quantizes the transform coefficient data and outputs the quantized data to the lossless encoding unit 16 and the inverse quantization unit 21. Further, the quantization unit 15 changes the bit rate of the quantized data by switching the quantization parameter (quantization scale) based on the rate control signal from the rate control unit 18.

The lossless encoding unit 16 is supplied with quantized data output from the quantization unit 15, prediction mode information from the intra prediction unit 31 described later, prediction mode information from the motion prediction / compensation unit 32, and the like. Also, information indicating whether the optimal mode is intra prediction or inter prediction is supplied from the predicted image / optimum mode selection unit 35. Note that the prediction mode information includes prediction mode, block size information of a prediction unit, and the like according to intra prediction or inter prediction. The lossless encoding unit 16 performs lossless encoding processing on the quantized data by, for example, variable length encoding or arithmetic encoding, generates image compression information, and outputs it to the accumulation buffer 17. Moreover, the lossless encoding part 16 performs the lossless encoding of the prediction mode information supplied from the intra prediction part 31, when the optimal mode is intra prediction. In addition, when the optimum mode is inter prediction, the lossless encoding unit 16 performs lossless encoding of prediction mode information, prediction block information, difference motion vector information, and the like supplied from the motion prediction / compensation unit 32. Further, the lossless encoding unit 16 includes information subjected to lossless encoding in the image compression information. For example, the lossless encoding unit 16 adds the header information of the encoded stream that is the image compression information.

The accumulation buffer 17 accumulates the compressed image information from the lossless encoding unit 16. The accumulation buffer 17 outputs the accumulated image compression information at a transmission rate corresponding to the transmission path.

The rate control unit 18 monitors the free capacity of the storage buffer 17, generates a rate control signal according to the free capacity, and outputs it to the quantization unit 15. The rate control unit 18 acquires information indicating the free capacity from the accumulation buffer 17, for example. The rate control unit 18 reduces the bit rate of the quantized data by the rate control signal when the free space is low. Further, when the free capacity of the storage buffer 17 is sufficiently large, the rate control unit 18 increases the bit rate of the quantized data by the rate control signal.

The inverse quantization unit 21 performs an inverse quantization process on the quantized data supplied from the quantization unit 15. The inverse quantization unit 21 outputs transform coefficient data obtained by performing the inverse quantization process to the inverse orthogonal transform unit 22.

The inverse orthogonal transform unit 22 performs an inverse orthogonal transform process on the transform coefficient data supplied from the inverse quantization unit 21, and outputs the obtained data to the addition unit 23.

The adding unit 23 adds the data supplied from the inverse orthogonal transform unit 22 and the predicted image data supplied from the predicted image / optimum mode selection unit 35 to generate decoded image data, and the deblocking filter 24 and the frame memory To 25. The decoded image data is used as image data for the reference image.

The deblocking filter 24 performs a filter process for reducing block distortion that occurs during image coding. The deblocking filter 24 performs a filter process for removing block distortion from the decoded image data supplied from the adding unit 23, and outputs the decoded image data after the filter process to the frame memory 25.

The frame memory 25 holds the decoded image data before the filtering process supplied from the adding unit 23 and the decoded image data after the filtering process supplied from the deblocking filter 24. The decoded image data held in the frame memory 25 is supplied as reference image data to the intra prediction unit 31 or the motion prediction / compensation unit 32 via the selector 26.

When the intra prediction unit 31 performs the intra prediction, the selector 26 supplies the decoded image data before the deblocking filter process held in the frame memory 25 to the intra prediction unit 31 as reference image data. Further, when the motion prediction / compensation unit 32 performs inter prediction, the selector 26 supplies the decoded image data after the deblocking filter processing held in the frame memory 25 to the motion prediction / compensation unit 32 as reference image data. .

The intra prediction unit 31 uses the input image data supplied from the screen rearrangement buffer 12 and the reference image data supplied from the frame memory 25 to predict the target block in all candidate intra prediction modes and Determine the intra prediction mode. For example, the intra prediction unit 31 calculates the cost function value in each intra prediction mode, and sets the intra prediction mode in which the coding efficiency is the best based on the calculated cost function value as the optimal intra prediction mode. The intra prediction unit 31 outputs the predicted image data generated in the optimal intra prediction mode and the cost function value in the optimal intra prediction mode to the predicted image / optimum mode selection unit 35. Further, the intra prediction unit 31 outputs prediction mode information indicating the optimal intra prediction mode to the lossless encoding unit 16.

The motion prediction / compensation unit 32 uses the input image data supplied from the screen rearrangement buffer 12 and the reference image data supplied from the frame memory 25 to predict the target block in all candidate inter prediction modes. Determine the optimal inter prediction mode. For example, the motion prediction / compensation unit 32 calculates the cost function value in each inter prediction mode, and sets the inter prediction mode in which the coding efficiency is the best based on the calculated cost function value as the optimal inter prediction mode. In addition, the motion prediction / compensation unit 32 calculates a cost function value using the prediction block information and the difference motion vector information generated by the prediction motion vector information setting unit 33. Further, the motion prediction / compensation unit 32 outputs the predicted image data generated in the optimal inter prediction mode and the cost function value in the optimal inter prediction mode to the predicted image / optimum mode selection unit 35. In addition, the motion prediction / compensation unit 32 outputs prediction mode information, prediction block information, difference motion vector information, and the like regarding the optimal inter prediction mode to the lossless encoding unit 16.

The predicted motion vector information setting unit 33 sets the horizontal motion vector information of the encoded adjacent block as a candidate for predicted horizontal motion vector information for the target block. Also, the predicted motion vector information setting unit 33 generates differential motion vector information indicating the difference between the candidate horizontal predicted motion vector information and the horizontal motion vector information of the target block for each candidate. Further, the predicted motion vector information setting unit 33 sets horizontal motion vector information having the highest coding efficiency of the difference motion vector information among the candidates as the predicted horizontal motion vector information. The prediction motion vector information setting unit 33 generates horizontal prediction block information indicating which adjacent block motion vector information the set horizontal prediction motion vector information is. For example, a flag (hereinafter referred to as “horizontal prediction block flag”) is generated as horizontal prediction block information.

The predicted motion vector information setting unit 33 sets the vertical motion vector information of the adjacent block that has been encoded for the target block as the candidate for the predicted vertical motion vector information. Also, the predicted motion vector information setting unit 33 generates differential motion vector information indicating the difference between the candidate vertical predicted motion vector information and the vertical motion vector information of the target block for each candidate. Further, the predicted motion vector information setting unit 33 sets the vertical motion vector information having the highest coding efficiency of the difference motion vector information among the candidates as the predicted vertical motion vector information. The predicted motion vector information setting unit 33 generates vertical predicted block information indicating which adjacent block motion vector information the set vertical predicted motion vector information is. For example, a flag (hereinafter referred to as “vertical prediction block flag”) is generated as the vertical prediction block information.

Further, the predicted motion vector information setting unit 33 uses the motion vector information of the block indicated by the predicted block flag for the horizontal component and the vertical component, respectively, as the predicted motion vector information. Also, the predicted motion vector information setting unit 33 calculates difference motion vector information, which is the difference between the motion vector information of the target block and the predicted motion vector information, for each of the horizontal component and the vertical component, and sends it to the motion prediction / compensation unit 32. Output.

FIG. 8 shows the configuration of the motion prediction / compensation unit 32 and the predicted motion vector information setting unit 33. The motion prediction / compensation unit 32 includes a motion search unit 321, a cost function value calculation unit 322, a mode determination unit 323, a motion compensation processing unit 324, and a motion vector information buffer 325.

The motion search unit 321 is supplied with the rearranged input image data supplied from the screen rearrangement buffer 12 and the reference image data read from the frame memory 25. The motion search unit 321 performs motion search in all candidate inter prediction modes and detects a motion vector. The motion search unit 321 outputs motion vector information indicating the detected motion vector to the cost function value calculation unit 322 together with the input image data and the reference image data when the motion vector is detected.

The cost function value calculation unit 322 is supplied with motion vector information, input image data, reference image data, and predicted motion vector information setting unit 33 from the motion search unit 321 and predicted block information and difference motion vector information. The cost function value calculation unit 322 calculates cost function values in all candidate inter prediction modes using the motion vector information, the input image data, the reference image data, the prediction block flag, and the difference motion vector information.

Calculating the cost function value is, for example, H. As defined by JM (Joint Model), which is reference software in the H.264 / AVC system, this is performed based on either the High Complexity mode or the Low Complexity mode.

That is, in the High Complexity mode, all the prediction modes that are candidates are subjected to the lossless encoding process, and the cost function value represented by the following equation (9) is calculated for each prediction mode.
Cost (Mode∈Ω) = D + λ · R (9)

Ω indicates the entire set of prediction modes that are candidates for encoding the image of the block. D indicates the differential energy (distortion) between the decoded image and the input image when encoding is performed in the prediction mode. R is a generated code amount including orthogonal transform coefficients, prediction mode information, prediction block information, differential motion vector information, and the like, and λ is a Lagrange multiplier given as a function of the quantization parameter QP.

That is, in order to perform encoding in the High Complexity mode, the parameters D and R are calculated, and therefore, it is necessary to perform temporary encoding once in all candidate prediction modes, which requires a higher calculation amount. .

On the other hand, in the Low Complexity mode, prediction images are generated and header bits including prediction block information, differential motion vector information, prediction mode information, and the like are generated in all candidate prediction modes, and the following equation (10 ) Is calculated.
Cost (Mode∈Ω) = D + QP2Quant (QP) · Header_Bit (10)

Ω indicates the entire set of prediction modes that are candidates for encoding the image of the block. D indicates the differential energy (distortion) between the decoded image and the input image when encoding is performed in the prediction mode. Header_Bit is a header bit for the prediction mode, and QP2Quant is a function given as a function of the quantization parameter QP.

That is, in the Low Complexity mode, it is necessary to perform prediction processing for each prediction mode, but since the decoded image is not necessary, it is possible to realize with a calculation amount lower than that in the High Complexity mode.

The cost function value calculation unit 322 outputs the calculated cost function value to the mode determination unit 323.

The mode determination unit 323 determines the mode having the minimum cost function value as the optimal inter prediction mode. In addition, the mode determination unit 323 includes the motion compensation processing unit 324 including the optimal inter prediction mode information indicating the determined optimal inter prediction mode together with the motion vector information, the prediction block flag, the difference motion vector information, and the like related to the optimal inter prediction mode. Output to. Note that the prediction mode information includes block size information and the like.

The motion compensation processing unit 324 performs motion compensation on the reference image data read from the frame memory 25 based on the optimal inter prediction mode information and the motion vector information, and generates predicted image data to generate a predicted image / optimum mode. The data is output to the selection unit 35. In addition, the motion compensation processing unit 324 outputs prediction mode information for optimal inter prediction, difference motion vector information in the mode, and the like to the lossless encoding unit 16.

The motion vector information buffer 325 holds motion vector information related to the optimal inter prediction mode. Also, the motion vector information buffer 325 outputs the motion vector information of the adjacent block that has been encoded with respect to the target block to be encoded, to the predicted motion vector information setting unit 33.

In the motion prediction / compensation unit 32, for example, H.264 A motion prediction / compensation process with 1/4 pixel accuracy defined in H.264 / AVC is performed. FIG. 9 is a diagram for explaining a motion prediction / compensation process with ¼ pixel accuracy. In FIG. 9, the position “A” is the position of the integer precision pixel stored in the frame memory 25, the positions “b”, “c”, and “d” are the positions of half pixel precision, the positions “e 1”, “ “e2” and “e3” are positions with 1/4 pixel accuracy.

In the following, Clip1 () is defined as shown in Expression (11).

In Expression (11), when the input image has 8-bit precision, the value of max_pix is 255.

The pixel values at the positions “b” and “d” are generated as in Expressions (12) and (13) using a 6-tap FIR filter.
F = A ₋₂ −5 · A ₋₁ + 20 · A ₀ + 20 · A ₁ −5 · A ₂ + A ₃ (12)
b, d = Clip1 ((F + 16) >> 5) (13)

The pixel value at the position “c” is generated as shown in either Expression (14) or Expression (15) and Expression (16) using a 6-tap FIR filter.
F = b ₋₂ −5 · b ₋₁ + 20 · b ₀ + 20 · b ₁ −5 · b ₂ + b ₃ (14)
F = d ₋₂ −5 · d ₋₁ + 20 · d ₀ + 20 · d ₁ −5 · d ₂ + d ₃ (15)
c = Clip1 ((F + 512) >> 10) (16)
Note that the Clip1 process is performed only once at the end after performing both the horizontal and vertical product-sum processes.

The pixel values at the positions “e1” to “e3” are generated as shown in equations (17) to (19) by linear interpolation.
e1 = (A + b + 1) >> 1 (17)
e2 = (b + d + 1) >> 1 (18)
e3 = (b + c + 1) >> 1 (19)
In this way, the motion prediction / compensation unit 32 performs a motion prediction / compensation process with 1/4 pixel accuracy.

The prediction motion vector information setting unit 33 includes a horizontal prediction motion vector information generation unit 331, a vertical prediction motion vector information generation unit 332, and an identification information generation unit 334.

The horizontal prediction motion vector information generation unit 331 sets horizontal prediction motion vector information that provides the highest encoding efficiency in the encoding process for the horizontal component of the motion vector information of the target block. The predicted horizontal motion vector information generation unit 331 uses the predicted horizontal motion vector information of the adjacent block supplied from the motion prediction / compensation unit 32 as a candidate for predicted horizontal motion vector information. Further, the horizontal predicted motion vector information generation unit 331 generates horizontal differential motion vector information indicating a difference between the horizontal motion vector information of each candidate and the horizontal motion vector information of the target block supplied from the motion prediction / compensation unit 32. To do. Further, the horizontal predicted motion vector information generation unit 331 sets the candidate horizontal motion vector information that minimizes the code amount of the horizontal difference motion vector information as horizontal predicted motion vector information. The horizontal predicted motion vector information generation unit 331 outputs the horizontal difference motion vector information when horizontal predicted motion vector information and horizontal predicted motion vector information are used to the identification information generation unit 334 as a horizontal predicted motion vector information generation result.

The vertical prediction motion vector information generation unit 332 sets vertical prediction motion vector information that provides the highest encoding efficiency in the encoding process for the vertical component of the motion vector information of the target block. The vertical motion vector predictor information generation unit 332 sets the vertical motion vector information of the encoded adjacent block supplied from the motion prediction / compensation unit 32 as a candidate for the vertical motion vector predictor information. Further, the vertical motion vector predictor information generation unit 332 generates vertical difference motion vector information indicating the difference between the vertical motion vector information of each candidate and the vertical motion vector information of the target block supplied from the motion prediction / compensation unit 32. To do. Further, the horizontal prediction motion vector information generation unit 331 sets the candidate vertical motion vector information that minimizes the code amount of the vertical difference motion vector information as the vertical prediction motion vector information. The predicted vertical motion vector information generation unit 332 outputs the vertical difference motion vector information when the predicted vertical motion vector information and the predicted vertical motion vector information are used to the identification information generation unit 334 as the predicted vertical motion vector information generation result.

The identification information generation unit 334 generates horizontal prediction block information, for example, a horizontal prediction block flag indicating a block in which the motion vector information is selected as the horizontal prediction motion vector information, based on the horizontal prediction motion vector information generation result. The identification information generation unit 334 outputs the generated horizontal prediction block flag to the cost function value calculation unit 322 of the motion prediction / compensation unit 32 together with the horizontal difference motion vector information. Also, the identification information generation unit 334 generates vertical prediction block information, for example, a vertical prediction block flag, indicating a block in which the motion vector information is selected as the vertical prediction motion vector information based on the vertical prediction motion vector information generation result. The identification information generation unit 334 outputs the generated vertical prediction block flag to the cost function value calculation unit 322 of the motion prediction / compensation unit 32 together with the vertical difference motion vector information.

Note that the predicted motion vector information setting unit 33 calculates the difference motion vector information indicating the difference between the horizontal (vertical) motion vector information of the target block and the motion vector information of each candidate, together with information indicating the candidate block, as a cost function value calculation unit. You may supply to 322. In this case, the candidate horizontal (vertical) motion vector information that minimizes the cost function value calculated by the cost function value calculation unit 322 is set as the horizontal (vertical) predicted motion vector information. Further, identification information indicating a candidate block having the smallest cost function value is used in inter prediction.

Returning to FIG. 7, the predicted image / optimum mode selection unit 35 compares the cost function value supplied from the intra prediction unit 31 with the cost function value supplied from the motion prediction / compensation unit 32, and the cost function value is small. Is selected as the optimum mode with the best coding efficiency. Further, the predicted image / optimum mode selection unit 35 outputs the predicted image data generated in the optimal mode to the subtraction unit 13 and the addition unit 23. Further, the predicted image / optimum mode selection unit 35 outputs information indicating whether the optimal mode is the intra prediction mode or the inter prediction mode to the lossless encoding unit 16. Note that the predicted image / optimum mode selection unit 35 switches between intra prediction and inter prediction in units of slices.

<2. Operation of Image Encoding Device>
FIG. 10 is a flowchart showing the operation of the image coding apparatus. In step ST11, the A / D converter 11 performs A / D conversion on the input image signal.

In step ST12, the screen rearrangement buffer 12 performs image rearrangement. The screen rearrangement buffer 12 stores the image data supplied from the A / D conversion unit 11, and rearranges from the display order of each picture to the encoding order.

In step ST13, the subtraction unit 13 generates prediction error data. The subtraction unit 13 calculates a difference between the image data of the images rearranged in step ST12 and the predicted image data selected by the predicted image / optimum mode selection unit 35, and generates prediction error data. The prediction error data has a smaller data amount than the original image data. Therefore, the data amount can be compressed as compared with the case where the image is encoded as it is.

In step ST14, the orthogonal transform unit 14 performs an orthogonal transform process. The orthogonal transformation unit 14 performs orthogonal transformation on the prediction error data supplied from the subtraction unit 13. Specifically, orthogonal transformation such as discrete cosine transformation and Karhunen-Loeve transformation is performed on the prediction error data, and transformation coefficient data is output.

In step ST15, the quantization unit 15 performs a quantization process. The quantization unit 15 quantizes the transform coefficient data. At the time of quantization, rate control is performed as described in the process of step ST25 described later.

In step ST16, the inverse quantization unit 21 performs an inverse quantization process. The inverse quantization unit 21 inversely quantizes the transform coefficient data quantized by the quantization unit 15 with characteristics corresponding to the characteristics of the quantization unit 15.

In step ST17, the inverse orthogonal transform unit 22 performs an inverse orthogonal transform process. The inverse orthogonal transform unit 22 performs inverse orthogonal transform on the transform coefficient data inversely quantized by the inverse quantization unit 21 with characteristics corresponding to the characteristics of the orthogonal transform unit 14.

In step ST18, the adding unit 23 generates reference image data. The adding unit 23 adds the predicted image data supplied from the predicted image / optimum mode selecting unit 35 and the data after inverse orthogonal transformation of the position corresponding to the predicted image, and obtains reference image data (decoded image data). Generate.

In step ST19, the deblocking filter 24 performs filter processing. The deblocking filter 24 filters the decoded image data output from the adding unit 23 to remove block distortion.

In step ST20, the frame memory 25 stores reference image data. The frame memory 25 stores reference image data (decoded image data) after filtering.

In step ST21, the intra prediction unit 31 and the motion prediction / compensation unit 32 each perform a prediction process. That is, the intra prediction unit 31 performs intra prediction processing in the intra prediction mode, and the motion prediction / compensation unit 32 performs motion prediction / compensation processing in the inter prediction mode. The details of the prediction process will be described later with reference to FIG. 11. By this process, the prediction process is performed in all candidate prediction modes, and the cost function values in all candidate prediction modes are respectively determined. Calculated. Then, based on the calculated cost function value, the optimal intra prediction mode and the optimal inter prediction mode are selected, and the predicted image generated in the selected prediction mode and its cost function value and prediction mode information are predicted image / optimum It is supplied to the mode selection unit 35.

In step ST22, the predicted image / optimum mode selection unit 35 selects predicted image data. The predicted image / optimum mode selection unit 35 determines the optimum mode with the best coding efficiency based on the cost function values output from the intra prediction unit 31 and the motion prediction / compensation unit 32. Further, the predicted image / optimum mode selection unit 35 selects the predicted image data of the determined optimal mode and outputs it to the subtraction unit 13 and the addition unit 23. As described above, the predicted image data is used for the calculations in steps ST13 and ST18.

In step ST23, the lossless encoding unit 16 performs a lossless encoding process. The lossless encoding unit 16 performs lossless encoding on the quantized data output from the quantization unit 15. That is, lossless encoding such as variable length encoding or arithmetic encoding is performed on the quantized data, and the data is compressed. Further, the lossless encoding unit 16 performs lossless encoding such as prediction mode information corresponding to the prediction image data selected in step ST22, and adds the prediction mode to the image compression information generated by lossless encoding of the quantized data. Lossless encoded data such as information is included.

In step ST24, the accumulation buffer 17 performs accumulation processing. The accumulation buffer 17 accumulates the compressed image information output from the lossless encoding unit 16. The compressed image information stored in the storage buffer 17 is appropriately read out and transmitted to the decoding side via the transmission path.

In step ST25, the rate control unit 18 performs rate control. When accumulating image compression information in the accumulation buffer 17, the rate control unit 18 controls the rate of the quantization operation of the quantization unit 15 so that overflow or underflow does not occur in the accumulation buffer 17.

Next, the prediction process in step ST21 in FIG. 10 will be described with reference to the flowchart in FIG.

In step ST31, the intra prediction unit 31 performs an intra prediction process. The intra prediction unit 31 performs intra prediction on the image of the target block in all candidate intra prediction modes. Note that the decoded image data before the blocking filter processing is performed by the deblocking filter 24 is used as the image data of the decoded image referred to in the intra prediction. By this intra prediction process, intra prediction is performed in all candidate intra prediction modes, and cost function values are calculated for all candidate intra prediction modes. Then, based on the calculated cost function value, one intra prediction mode with the best coding efficiency is selected from all the intra prediction modes.

In step ST32, the motion prediction / compensation unit 32 performs an inter prediction process. The motion prediction / compensation unit 32 uses the decoded image data after the deblocking filter processing stored in the frame memory 25 to perform inter prediction processing in a candidate inter prediction mode. By this inter prediction processing, prediction processing is performed in all candidate inter prediction modes, and cost function values are calculated for all candidate inter prediction modes. Then, based on the calculated cost function value, one inter prediction mode with the best coding efficiency is selected from all the inter prediction modes.

Next, the intra prediction process in step ST31 in FIG. 11 will be described with reference to the flowchart in FIG.

In step ST41, the intra prediction unit 31 performs intra prediction in each prediction mode. The intra prediction unit 31 generates predicted image data for each intra prediction mode using the decoded image data before the blocking filter processing.

In step ST42, the intra prediction unit 31 calculates a cost function value in each prediction mode. The cost function value is calculated as described above, for example, H.264. As defined by JM (Joint Model), which is reference software in the H.264 / AVC system, this is performed based on either the High Complexity mode or the Low Complexity mode. In other words, in the High Complexity mode, as a process of step ST42, all the candidate prediction modes are subjected to the lossless encoding process, and the cost function value represented by the equation (9) is set for each prediction mode. To calculate. In the Low Complexity mode, as a process of step ST42, for all the prediction modes that are candidates, generation of a prediction image and header bits such as motion vector information and prediction mode information are generated, and expressed by Expression (10). The calculated cost function value is calculated for each prediction mode.

In step ST43, the intra prediction unit 31 determines the optimal intra prediction mode. Based on the cost function value calculated in step ST42, the intra prediction unit 31 selects one intra prediction mode having a minimum cost function value from them, and determines the optimal intra prediction mode.

Next, the inter prediction process in step ST32 in FIG. 11 will be described with reference to the flowchart in FIG.

In step ST51, the motion prediction / compensation unit 32 performs a motion prediction process. The motion prediction / compensation unit 32 performs motion prediction for each prediction mode to detect a motion vector, and proceeds to step ST52.

In step ST52, the motion vector predictor information setting unit 33 performs a motion vector predictor information setting process. The predicted motion vector information setting unit 33 generates a predicted block flag and difference motion vector information for the target block.

FIG. 14 is a flowchart showing the predicted motion vector information setting process. In step ST <b> 61, the predicted motion vector information setting unit 33 selects a candidate for predicted horizontal motion vector information. The motion vector predictor information setting unit 33 selects the horizontal motion vector information of the encoded block adjacent to the target block as a candidate for the motion vector predictive motion vector information, and proceeds to step ST62.

In step ST62, the predicted motion vector information setting unit 33 performs horizontal predicted motion vector information setting processing. The predicted motion vector information setting unit 33 detects the i-th horizontal motion vector information that minimizes the amount of encoding of the horizontal difference motion vector information based on, for example, Expression (20).
arg _i min (R (mvx−pmvx (i))) (20)

Note that “mvx” indicates the horizontal motion vector information of the target block, and “pmvx (i)” indicates the i th candidate of the predicted horizontal motion vector information. “R (mvx−pmvx (i))” is a code amount when the horizontal difference motion vector information indicating the difference between the i th candidate of the horizontal prediction motion vector and the horizontal motion vector information of the target block is encoded. Is shown.

The predicted motion vector information setting unit 33 generates a horizontal predicted block flag indicating an adjacent block of the horizontal motion vector information that minimizes the coding amount detected based on the equation (20). Further, the predicted motion vector information setting unit 33 generates horizontal difference motion vector information when the horizontal motion vector information is used, and the process proceeds to step ST63.

In step ST63, the motion vector predictor information setting unit 33 selects a candidate for motion vector predictor information. The motion vector predictor information setting unit 33 selects the vertical motion vector information of the encoded block adjacent to the target block as a candidate for the motion vector predictor information, and proceeds to step ST64.

In step ST64, the motion vector predictor information setting unit 33 performs a vertical motion vector predictor information setting process. The motion vector predictor information setting unit 33 detects the j-th vertical motion vector information that minimizes the amount of encoding of the vertical difference report based on, for example, Expression (21).
arg _j min (R (mvy-pmvy (j))) (21)

Note that “mvy” indicates the vertical motion vector information of the target block, and “pmvy (j)” indicates the j-th candidate for the predicted vertical motion vector information. “R (mvy−pmvy (j))” is a code amount when the vertical difference motion vector information indicating the difference between the j th candidate of the vertical prediction motion vector and the vertical motion vector information of the target block is encoded. Is shown.

The predicted motion vector information setting unit 33 generates a vertical predicted block flag indicating an adjacent block of the vertical motion vector information in which the coding amount detected based on Expression (21) is minimized. Also, the predicted motion vector information setting unit 33 generates vertical difference motion vector information when the vertical motion vector information is used, ends the predicted motion vector information setting process, and returns to step ST53 in FIG.

In step ST53, the motion prediction / compensation unit 32 calculates a cost function value in each prediction mode. The motion prediction / compensation unit 32 calculates the cost function value using the above-described equation (9) or equation (10). Also, the motion prediction / compensation unit 32 calculates the generated code amount using the difference motion vector information. Note that the cost function value for the inter prediction mode is calculated using the H.264 standard. Evaluation of cost function values in skipped macroblocks and direct mode defined in the H.264 / AVC format is also included.

In step ST54, the motion prediction / compensation unit 32 determines the optimal inter prediction mode. Based on the cost function value calculated in step ST54, the motion prediction / compensation unit 32 selects one prediction mode having the minimum cost function value from them, and determines the optimum inter prediction mode.

In this way, the image encoding device 10 individually sets the horizontal prediction motion vector and the vertical prediction motion vector for the target block. Further, the image encoding device 10 performs variable-length encoding on the horizontal differential motion vector information that is the difference between the horizontal motion vector information of the target block and the predicted horizontal motion vector information. In addition, the image encoding device 10 performs variable-length encoding on the vertical difference motion vector information that is the difference between the vertical motion vector of the target block and the predicted vertical motion vector information. The predicted horizontal motion vector information and the predicted vertical motion vector information are indicated by a prediction block flag as to which of the adjacent encoded blocks.

Therefore, the data amount of the prediction block flag can be reduced as compared with the case of using the horizontal / vertical prediction motion vector information shown in Expression (22). In addition, as shown in Expression (22), the horizontal / vertical prediction motion vector information includes the motion vector of the adjacent block that minimizes the code amount obtained by adding the code amount of the horizontal difference motion vector information and the code amount of the vertical difference motion vector information. Information.
arg _k min (R (mvx−pmvx (k))
+ R (mvy−pmvy (k))) (22)

For example, when there are three types of candidates for the horizontal motion vector information and three types of candidates for the vertical motion vector information, the present technology may prepare six types (three types + three types) of flags. However, if a block is determined based on the code amount obtained by adding the code amount of the horizontal difference motion vector information and the code amount of the vertical difference motion vector information, nine types (three types × three types) of flags must be prepared. That is, in the present technology, the number of flags to be prepared is reduced, so that the efficiency in encoding motion vector information can be improved.

<3. Configuration of Image Decoding Device>
Next, the image decoding apparatus will be described. Image compression information generated by encoding an input image is supplied to an image decoding apparatus via a predetermined transmission path, recording medium, or the like and decoded.

FIG. 15 shows the configuration of the image decoding apparatus. The image decoding device 50 includes a storage buffer 51, a lossless decoding unit 52, an inverse quantization unit 53, an inverse orthogonal transform unit 54, an addition unit 55, a deblocking filter 56, a screen rearrangement buffer 57, a digital / analog conversion unit ( D / A converter 58). Furthermore, the image decoding device 50 includes a frame memory 61,

selectors

62 and 75, an intra prediction unit 71, a motion compensation unit 72, and a predicted motion vector information setting unit 73.

The accumulation buffer 51 accumulates the transmitted image compression information. The lossless decoding unit 52 decodes the image compression information supplied from the accumulation buffer 51 by a method corresponding to the encoding method of the lossless encoding unit 16 in FIG.

The lossless decoding unit 52 outputs prediction mode information obtained by decoding the image compression information to the intra prediction unit 71 and the motion compensation unit 72. Further, the lossless decoding unit 52 outputs prediction block information (prediction block flag) and differential motion vector information obtained by decoding the image compression information to the motion compensation unit 72.

The inverse quantization unit 53 inversely quantizes the quantized data decoded by the lossless decoding unit 52 by a method corresponding to the quantization method of the quantization unit 15 of FIG. The inverse orthogonal transform unit 54 performs inverse orthogonal transform on the output of the inverse quantization unit 53 by a method corresponding to the orthogonal transform method of the orthogonal transform unit 14 of FIG.

The addition unit 55 adds the data after inverse orthogonal transformation and the predicted image data supplied from the selector 75 to generate decoded image data, and outputs the decoded image data to the deblocking filter 56 and the frame memory 61.

The deblocking filter 56 performs deblocking filter processing on the decoded image data supplied from the addition unit 55, removes block distortion, supplies the frame memory 61 to the frame memory 61, and outputs the frame memory 61 to the screen rearrangement buffer 57. To do.

The screen rearrangement buffer 57 rearranges images. That is, the order of frames rearranged for the encoding order in the screen rearrangement buffer 12 in FIG. 7 is rearranged in the original display order and output to the D / A conversion unit 58.

The D / A conversion unit 58 performs D / A conversion on the image data supplied from the screen rearrangement buffer 57 and outputs it to a display (not shown) to display an image.

The frame memory 61 stores the decoded image data before the filtering process by the deblocking filter 24 and the decoded image data after the filtering process by the deblocking filter 24 are performed.

Based on the prediction mode information supplied from the lossless decoding unit 52, the selector 62 supplies the decoded image data before filtering stored in the frame memory 61 to the intra prediction unit 71 in the case of decoding an intra prediction image. To do. In addition, in the case of decoding an inter prediction image, the selector 62 supplies the decoded image data after the filter processing stored in the frame memory 61 to the motion compensation unit 72.

The intra prediction unit 71 generates predicted image data based on the prediction mode information supplied from the lossless decoding unit 52 and the decoded image data supplied from the frame memory 61 via the selector 62, and the generated predicted image data Is output to the selector 75.

The motion compensation unit 72 adds the difference motion vector information supplied from the lossless decoding unit 52 and the predicted motion vector information supplied from the predicted motion vector information setting unit 73 to obtain motion vector information of a block to be decoded. Generate. In addition, the motion compensation unit 72 performs motion compensation using the decoded image data supplied from the frame memory 61 based on the generated motion vector information and the prediction mode information supplied from the lossless decoding unit 52, so that predicted image data Is output to the selector 75.

The prediction motion vector information setting unit 73 sets prediction motion vector information based on the prediction block information supplied from the lossless decoding unit 52. The predicted motion vector information setting unit 73 sets the horizontal motion vector information of the block indicated by the horizontal predicted block flag information in the decoded adjacent block as the predicted horizontal motion vector information for the target block. Also, the vertical motion vector information of the block indicated by the vertical prediction block flag in the decoded adjacent block is used as the vertical prediction motion vector information. The predicted motion vector information setting unit 73 outputs the set horizontal predicted motion vector information and vertical motion vector information to the motion compensation unit 72.

FIG. 16 shows the configuration of the motion compensation unit 72 and the predicted motion vector information setting unit 73.

The motion compensation unit 72 includes a block size information buffer 721, a differential motion vector information buffer 722, a motion vector information generation unit 723, a motion compensation processing unit 724, and a motion vector information buffer 725.

The block size information buffer 721 stores block size information included in the prediction mode information supplied from the lossless decoding unit 52. Also, the block size information buffer 721 outputs the stored block size information to the motion compensation processing unit 724 and the predicted motion vector information setting unit 73.

The difference motion vector information buffer 722 stores the difference motion vector information supplied from the lossless decoding unit 52. Further, the differential motion vector information buffer 722 outputs the stored differential motion vector information to the motion vector information generation unit 723.

The motion vector information generation unit 723 adds the horizontal difference motion vector information supplied from the difference motion vector information buffer 722 and the predicted horizontal motion vector information set by the predicted motion vector information setting unit 73. In addition, the motion vector information generation unit 723 adds the vertical difference motion vector information supplied from the difference motion vector information buffer 722 and the predicted vertical motion vector information set by the predicted motion vector information setting unit 73. The motion vector information generation unit 723 outputs the motion vector information obtained by adding the difference motion vector information and the predicted motion vector information to the motion compensation processing unit 724 and the motion vector information buffer 725.

The motion compensation processing unit 724 reads the image data of the reference image from the frame memory 61 based on the prediction mode information supplied from the lossless decoding unit 52. The motion compensation processing unit 724 performs motion compensation based on the image data of the reference image, the block information supplied from the block size information buffer 721, and the motion vector information supplied from the motion vector information generation unit 723. The motion compensation processing unit 724 outputs the predicted image data generated by the motion compensation to the selector 75.

The motion vector information buffer 725 stores the motion vector information supplied from the motion vector information generation unit 723. In addition, the motion vector information buffer 725 outputs the stored motion vector information to the predicted motion vector information setting unit 73.

The predicted motion vector information setting unit 73 includes a flag buffer 730, a horizontal predicted motion vector information generation unit 731 and a vertical predicted motion vector information generation unit 732.

The flag buffer 730 stores the prediction block flag supplied from the lossless decoding unit 52. Also, the flag buffer 730 outputs the stored prediction block flag to the horizontal prediction motion vector information generation unit 731 and the vertical prediction motion vector information generation unit 732.

The horizontal predicted motion vector information generation unit 731 selects the horizontal motion vector information indicated by the horizontal prediction block flag from the horizontal motion vector information of the adjacent block stored in the motion vector information buffer 725 of the motion compensation unit 72, and performs horizontal processing. Set to predicted motion vector information. The predicted horizontal motion vector information generation unit 731 outputs the set predicted horizontal motion vector information to the motion vector information generation unit 723 of the motion compensation unit 72.

The vertical prediction motion vector information generation unit 732 selects the vertical motion vector information indicated by the vertical prediction block flag from the vertical motion vector information of the adjacent block stored in the motion vector information buffer 725 of the motion compensation unit 72 and performs vertical operation. Set to predicted motion vector information. The predicted vertical motion vector information generation unit 732 outputs the set predicted vertical motion vector information to the motion vector information generation unit 723 of the motion compensation unit 72.

15, the selector 75 selects the intra prediction unit 71 for intra prediction and the motion compensation unit 72 for inter prediction based on the prediction mode information supplied from the lossless decoding unit 52. The selector 75 outputs the predicted image data generated by the selected intra prediction unit 71 or motion compensation unit 72 to the addition unit 55.

<4. Operation of Image Decoding Device>
Next, the image decoding processing operation performed by the image decoding device 50 will be described with reference to the flowchart of FIG.

In step ST81, the accumulation buffer 51 accumulates the transmitted image compression information. In step ST82, the lossless decoding unit 52 performs lossless decoding processing. The lossless decoding unit 52 decodes the compressed image information supplied from the accumulation buffer 51. That is, quantized data of each picture encoded by the lossless encoding unit 16 in FIG. 7 is obtained. Further, when the lossless decoding unit 52 performs lossless decoding of the prediction mode information included in the image compression information and the obtained prediction mode information is information related to the intra prediction mode, the prediction mode information is converted into the intra prediction unit 71. Output to. Moreover, the lossless decoding part 52 outputs prediction mode information to the motion compensation part 72, when prediction mode information is the information regarding inter prediction mode.

In step ST83, the inverse quantization unit 53 performs an inverse quantization process. The inverse quantization unit 53 inversely quantizes the quantized data decoded by the lossless decoding unit 52 with characteristics corresponding to the characteristics of the quantization unit 15 in FIG.

In step ST84, the inverse orthogonal transform unit 54 performs an inverse orthogonal transform process. The inverse orthogonal transform unit 54 performs inverse orthogonal transform on the transform coefficient data inversely quantized by the inverse quantization unit 53 with characteristics corresponding to the characteristics of the orthogonal transform unit 14 of FIG.

In step ST85, the addition unit 55 generates decoded image data. The adder 55 adds the data obtained by performing the inverse orthogonal transform process and the predicted image data selected in step ST89 described later to generate decoded image data. As a result, the original image is decoded.

In step ST86, the deblocking filter 56 performs filter processing. The deblocking filter 56 performs a deblocking filter process on the decoded image data output from the adding unit 55 to remove block distortion included in the decoded image.

In step ST87, the frame memory 61 performs a process of storing decoded image data.

In step ST88, the intra prediction unit 71 and the motion compensation unit 72 perform a predicted image generation process. The intra prediction unit 71 and the motion compensation unit 72 perform a prediction image generation process corresponding to the prediction mode information supplied from the lossless decoding unit 52, respectively.

That is, when prediction mode information for intra prediction is supplied from the lossless decoding unit 52, the intra prediction unit 71 generates predicted image data based on the prediction mode information. When inter prediction mode information is supplied from the lossless decoding unit 52, the motion compensation unit 72 performs motion compensation based on the prediction mode information to generate predicted image data.

In step ST89, the selector 75 selects predicted image data. The selector 75 selects the prediction image supplied from the intra prediction unit 71 and the prediction image data supplied from the motion compensation unit 72, supplies the selected prediction image data to the addition unit 55, and, as described above, In step ST85, it is added to the output of the inverse orthogonal transform unit 54.

In step ST90, the screen rearrangement buffer 57 performs image rearrangement. That is, the screen rearrangement buffer 57 rearranges the order of frames rearranged for encoding by the screen rearrangement buffer 12 of the image encoding device 10 of FIG. 7 to the original display order.

In step ST91, the D / A converter 58 D / A converts the image data from the screen rearrangement buffer 57. This image is output to a display (not shown), and the image is displayed.

Next, the predicted image generation processing in step ST88 in FIG. 17 will be described with reference to the flowchart in FIG.

In step ST101, the lossless decoding unit 52 determines whether or not the target block is intra-coded. When the prediction mode information obtained by performing lossless decoding is prediction mode information for intra prediction, the lossless decoding unit 52 supplies the prediction mode information to the intra prediction unit 71 and proceeds to step ST102. Also, when the prediction mode information is inter prediction mode information, the lossless decoding unit 52 supplies the prediction mode information to the motion compensation unit 72 and proceeds to step ST103.

In step ST102, the intra prediction unit 71 performs intra prediction image generation processing. The intra prediction unit 71 performs intra prediction using the decoded image data before deblocking filter processing and the prediction mode information stored in the frame memory 61, and generates predicted image data.

In step ST103, the motion compensation unit 72 performs inter prediction image generation processing. The motion compensation unit 72 performs motion compensation of the reference image read from the frame memory 61 based on the prediction mode information and the difference motion vector information from the lossless decoding unit 52, and generates predicted image data.

FIG. 19 is a flowchart showing the inter predicted image generation processing in step ST103. In step ST111, the motion compensation unit 72 acquires prediction mode information. The motion compensation unit 72 acquires the prediction mode information from the lossless decoding unit 52, and proceeds to step ST112.

In step ST112, the motion compensation unit 72 and the predicted motion vector information setting unit 73 perform a motion vector information reconstruction process. FIG. 20 is a flowchart showing the motion vector information reconstruction process.

In step ST121, the motion compensation unit 72 and the predicted motion vector information setting unit 73 acquire the predicted block flag and the difference motion vector information. The motion compensation unit 72 acquires the difference motion vector information from the lossless decoding unit 52. Further, the motion vector predictor information setting unit 73 acquires the prediction block flag from the lossless decoding unit 52, and proceeds to step ST122.

In step ST122, the predicted motion vector information setting unit 73 performs horizontal predicted motion vector information setting processing. The horizontal predicted motion vector information generation unit 731 selects horizontal motion vector information of the block indicated by the horizontal prediction block flag from the horizontal motion vector information of the adjacent block stored in the motion vector information buffer 725 of the motion compensation unit 72. To do. The predicted horizontal motion vector information generation unit 731 sets the selected predicted horizontal motion vector information as the predicted horizontal motion vector information.

In step ST123, the motion compensation unit 72 reconstructs horizontal motion vector information. The motion compensation unit 72 reconstructs the horizontal motion vector information by adding the horizontal difference motion vector information and the horizontal predicted motion vector information, and proceeds to step ST124.

In step ST124, the predicted motion vector information setting unit 73 performs vertical predicted motion vector information setting processing. The predicted vertical motion vector information generation unit 732 selects the vertical motion vector information of the block indicated by the vertical prediction block flag from the vertical motion vector information of the adjacent block stored in the motion vector information buffer 725 of the motion compensation unit 72. To do. The predicted vertical motion vector information generation unit 732 sets the selected vertical motion vector information as the predicted vertical motion vector information.

In step ST125, the motion compensation unit 72 reconstructs the vertical motion vector information. The motion compensation unit 72 reconstructs the vertical motion vector information by adding the vertical difference motion vector information and the predicted vertical motion vector information, and proceeds to step ST113 in FIG.

In step ST113, the motion compensation unit 72 generates predicted image data. Based on the prediction mode information acquired in step ST111 and the motion vector information reconstructed in step ST112, the motion compensation unit 72 reads reference image data from the frame memory 61 to perform motion compensation, generates predicted image data, and selects a selector. Output to 75.

As described above, the image decoding apparatus 50 uses the horizontal motion vector information of the adjacent block indicated by the horizontal prediction block flag as the horizontal prediction motion vector information, and the vertical motion vector information of the adjacent block indicated by the vertical prediction block flag as the vertical. Set to predicted motion vector information. Therefore, in order to improve the encoding efficiency in the image encoding device 10, the motion vector information can be correctly reconstructed even if the horizontal prediction motion vector information and the vertical prediction motion vector information are individually set.

<5. Other Configurations of Image Encoding Device and Image Decoding Device>
In the above-described image encoding device and image decoding device, the case has been described in which horizontal prediction motion vector information and vertical prediction motion vector information are individually set and the motion vector information is encoded and decoded. However, not only can the horizontal predicted motion vector information and the vertical predicted motion vector information be individually set, but also the horizontal and vertical motion vector information can be set, so that it is possible to achieve optimum coding efficiency. In this case, the motion vector predictor information setting unit 33a used in the image encoding device 10 has the configuration shown in FIG. Further, the motion vector predictor information setting unit 73a used in the image decoding apparatus 50 has the configuration shown in FIG.

In FIG. 21, the horizontal / vertical prediction motion vector information generation unit 333 sets the motion vector information of the encoded adjacent block supplied from the motion prediction / compensation unit 32 as a candidate for prediction motion vector information. Further, the horizontal / vertical prediction motion vector information generation unit 333 generates difference motion vector information indicating a difference between the motion vector information of each candidate and the motion vector information of the target block supplied from the motion prediction / compensation unit 32. Further, the horizontal / vertical prediction motion vector information generation unit 333 sets the motion vector information with the minimum coding amount detected based on the above equation (23) as the horizontal / vertical prediction motion vector information. The horizontal / vertical prediction motion vector information generation unit 333 outputs the difference motion vector information when the horizontal / vertical prediction motion vector information and the horizontal / vertical prediction motion vector information are used to the identification information generation unit 334a as the horizontal / vertical prediction motion vector information generation result. To do.

The identification information generation unit 334a individually selects one of horizontal prediction motion vector information and vertical prediction motion vector information, or horizontal / vertical prediction motion vector information, and selects the selected prediction motion vector information together with difference motion vector information as a cost function. The value is output to the value calculation unit 322. For example, when horizontal prediction motion vector information and vertical prediction motion vector information are selected as the prediction motion vector information, as described above, the identification information generation unit 334a calculates the cost function value of the horizontal prediction block flag and the horizontal difference motion vector information. To the unit 322. Also, the identification information generation unit 334a outputs the vertical prediction block flag and the vertical difference motion vector information to the cost function value calculation unit 322. Further, when the horizontal / vertical prediction motion vector information is selected as the prediction motion vector information, the identification information generation unit 334a generates horizontal / vertical prediction block information indicating the block whose motion vector information is selected as the horizontal / vertical prediction motion vector information. . For example, the identification information generation unit 334a generates a horizontal / vertical prediction block flag as the horizontal / vertical prediction block information. The identification information generation unit 334a outputs the generated horizontal / vertical prediction block flag to the cost function value calculation unit 322 together with the difference motion vector information.

Also, the identification information generation unit 334a generates identification information indicating which of the predicted horizontal motion vector information, the predicted vertical motion vector information, or the predicted horizontal and vertical motion vector information is selected. This identification information is supplied to the lossless encoding unit 16 via the motion prediction / compensation unit 32 and included in the picture parameter set or slice header of the image compression information.

When selecting the prediction motion vector information, the identification information generation unit 334a may switch between the horizontal prediction motion vector information and the vertical prediction motion vector information or the horizontal and vertical prediction motion vector information in units of pictures or slices. . Further, when selecting any one of the horizontal prediction motion vector information, the vertical prediction motion vector information, and the horizontal / vertical prediction motion vector information in units of pictures, the identification information generation unit 334a performs selection according to, for example, the picture type of the target block. You may do it. That is, in the P picture, even if there is some overhead for flag information, it is important to improve the efficiency of motion vector coding. Therefore, in the case of a P picture, the horizontal prediction block flag and horizontal difference motion vector information, and the vertical prediction block flag and vertical difference motion vector information are output to the cost function value calculation unit 322. In addition, in a B picture, having a horizontal prediction block flag and a vertical prediction block flag for List0 prediction and List1 prediction, respectively, can realize optimal encoding efficiency especially in the case of a low bit rate. Not exclusively. Therefore, in the case of a B picture, it is possible to achieve optimal encoding efficiency by outputting the horizontal / vertical prediction block flag and the difference motion vector information to the cost function value calculation unit 322 as in the conventional art.

22, the flag buffer 730a switches the supply destination of the prediction block flag based on the identification information included in the image compression information. For example, when the horizontal prediction motion vector information and the vertical prediction motion vector information are selected, the flag buffer 730 a outputs the prediction block flag to the horizontal prediction motion vector information generation unit 731 and the vertical prediction motion vector information generation unit 732. When the horizontal / vertical prediction motion vector information is selected, the flag buffer 730a outputs the prediction block flag to the horizontal / vertical prediction motion vector information generation unit 733. Further, for example, when the prediction motion vector information is switched according to the picture type, the flag buffer 730a switches the supply destination of the prediction block flag according to the picture type. For example, it is assumed that motion vector information is encoded using horizontal prediction motion vector information and vertical prediction motion vector information in the case of a P picture, and horizontal and vertical prediction motion vector information in the case of a B picture. In this case, the flag buffer 730a sets the prediction block flag in the case of a P picture and the horizontal prediction motion vector information generation unit 731 and the vertical prediction motion vector information generation unit 732, and in the case of a B picture, sets the prediction block flag as horizontal and vertical prediction motion vector information. This is supplied to the generation unit 733.

Further, the lossless encoding unit 16 may perform different code assignments in the horizontal direction and the vertical direction. For example, it is assumed that spatial prediction motion vector information and temporal prediction motion vector information can be used as the prediction motion vector information. In this case, in consideration of an imaging operation when a moving image to be encoded is generated, a code with a small amount of data is assigned to prediction motion vector information with high prediction accuracy. For example, when a captured image is recorded by an imaging device described later, when the panning operation of the imaging device is performed and the imaging direction moves in the horizontal direction, the motion vector information in the vertical direction is almost “0”. At this time, the prediction accuracy of the temporal prediction motion vector information is higher than the spatial prediction motion vector information in the vertical direction, and the prediction accuracy of the spatial prediction motion vector information is higher than the temporal prediction motion vector information in the horizontal direction. There are many cases. Therefore, regarding the horizontal prediction block information, code number “0” is assigned to the spatial prediction motion vector information block, and code number “1” is assigned to the temporal direction motion vector information block. As for the vertical prediction block information, a code number “1” is assigned to the spatial prediction vector information block, and a code number “0” is assigned to the temporal direction motion vector information block. In this way, by assigning different codes for the horizontal prediction block information and the vertical prediction block information, it is possible to use a large number of codes with a small amount of data, so that higher encoding efficiency can be realized.

<6. For software processing>
The series of processes described in the specification can be executed by hardware, software, or a combined configuration of both. When processing by software is executed, a program in which a processing sequence is recorded is installed and executed in a memory in a computer incorporated in dedicated hardware. Alternatively, the program can be installed and executed on a general-purpose computer capable of executing various processes.

FIG. 23 is a diagram illustrating a configuration of a computer device that executes the above-described series of processing by a program. The CPU 801 of the computer device 80 executes various processes according to programs recorded in the ROM 802 or the recording unit 808.

The RAM 803 appropriately stores programs executed by the CPU 801 and various data. These CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804.

An input / output interface 805 is also connected to the CPU 801 via the bus 804. An input unit 806 such as a touch panel, a keyboard, a mouse, and a microphone, and an output unit 807 including a display are connected to the input / output interface 805. The CPU 801 executes various processes in response to commands input from the input unit 806. Then, the CPU 801 outputs the processing result to the output unit 807.

The recording unit 808 connected to the input / output interface 805 includes, for example, a hard disk, and records programs executed by the CPU 801 and various data. A communication unit 809 communicates with an external device via a wired or wireless communication medium such as a network such as the Internet or a local area network or digital broadcasting. Further, the computer device 80 may acquire a program via the communication unit 809 and record it in the ROM 802 or the recording unit 808.

When the removable medium 85 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is mounted, the drive 810 drives them to acquire a recorded program or data. The acquired program and data are transferred to the ROM 802, RAM 803, or recording unit 808 as necessary.

The CPU 801 reads and executes a program for performing the above-described series of processing, and performs encoding processing and image processing on the image signal recorded in the recording unit 808 and the removable medium 85 and the image signal supplied via the communication unit 809. Decodes the compressed information.

<7. When applied to electronic devices>
In the above, H.264 is used as the encoding method / decoding method. Although the H.264 / AVC format is used, the present technology can also be applied to an image encoding device / image decoding device using an encoding method / decoding method for performing other motion prediction / compensation processing.

Furthermore, the present technology, for example, converts image information (bitstream) compressed by orthogonal transform such as discrete cosine transform and motion compensation via network media such as satellite broadcasting, cable TV (television), the Internet, and mobile phones. Applicable when receiving. Further, the present invention can be applied to an image encoding device and an image decoding device that are used when processing on a storage medium such as an optical, magnetic disk, and flash memory.

The image encoding device 10 and the image decoding device 50 described above can be applied to any electronic device. Examples thereof will be described below.

FIG. 24 illustrates a schematic configuration of a television apparatus to which the present technology is applied. The television apparatus 90 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, a display unit 906, an audio signal processing unit 907, a speaker 908, and an external interface unit 909. Furthermore, the television apparatus 90 includes a control unit 910, a user interface unit 911, and the like.

The tuner 902 selects a desired channel from the broadcast wave signal received by the antenna 901, performs demodulation, and outputs the obtained stream to the demultiplexer 903.

The demultiplexer 903 extracts video and audio packets of the program to be viewed from the stream, and outputs the extracted packet data to the decoder 904. The demultiplexer 903 outputs a packet of data such as EPG (Electronic Program Guide) to the control unit 910. If scrambling is being performed, descrambling is performed by a demultiplexer or the like.

The decoder 904 performs packet decoding processing, and outputs video data generated by the decoding processing to the video signal processing unit 905 and audio data to the audio signal processing unit 907.

The video signal processing unit 905 performs noise removal, video processing according to user settings, and the like on the video data. The video signal processing unit 905 generates video data of a program to be displayed on the display unit 906, image data by processing based on an application supplied via a network, and the like. The video signal processing unit 905 generates video data for displaying a menu screen for selecting an item and the like, and superimposes the video data on the video data of the program. The video signal processing unit 905 generates a drive signal based on the video data generated in this way, and drives the display unit 906.

The display unit 906 drives a display device (for example, a liquid crystal display element or the like) based on a drive signal from the video signal processing unit 905 to display a program video or the like.

The audio signal processing unit 907 performs predetermined processing such as noise removal on the audio data, performs D / A conversion processing and amplification processing on the processed audio data, and outputs the audio data by supplying the audio data to the speaker 908. .

The external interface unit 909 is an interface for connecting to an external device or a network, and transmits and receives data such as video data and audio data.

A user interface unit 911 is connected to the control unit 910. The user interface unit 911 includes an operation switch, a remote control signal receiving unit, and the like, and supplies an operation signal corresponding to a user operation to the control unit 910.

The control unit 910 is configured using a CPU (Central Processing Unit), a memory, and the like. The memory stores programs executed by the CPU, various data necessary for the CPU to perform processing, EPG data, data acquired via a network, and the like. The program stored in the memory is read and executed by the CPU at a predetermined timing such as when the television device 90 is activated. The CPU controls each unit so that the television device 90 operates according to the user operation by executing the program.

The television device 90 is provided with a bus 912 for connecting the tuner 902, the demultiplexer 903, the video signal processing unit 905, the audio signal processing unit 907, the external interface unit 909, and the control unit 910.

In the thus configured television apparatus, the decoder 904 is provided with the function of the image decoding apparatus (image decoding method) of the present application. Therefore, the television apparatus can correctly restore the motion vector information of the target block to be decoded based on the generated predicted motion vector information and the received difference motion vector information. Therefore, even if the broadcast station side sets the horizontal prediction motion vector information and the vertical prediction motion vector information individually and performs a process of increasing the encoding efficiency, the television apparatus can correctly decode.

FIG. 25 illustrates a schematic configuration of a mobile phone to which the present technology is applied. The cellular phone 92 includes a communication unit 922, an audio codec 923, a camera unit 926, an image processing unit 927, a demultiplexing unit 928, a recording / reproducing unit 929, a display unit 930, and a control unit 931. These are connected to each other via a bus 933.

In addition, an antenna 921 is connected to the communication unit 922, and a speaker 924 and a microphone 925 are connected to the audio codec 923. Further, an operation unit 932 is connected to the control unit 931.

The mobile phone 92 performs various operations such as transmission / reception of voice signals, transmission / reception of e-mail and image data, image shooting, and data recording in various modes such as a voice call mode and a data communication mode.

In the voice call mode, the voice signal generated by the microphone 925 is converted into voice data and compressed by the voice codec 923 and supplied to the communication unit 922. The communication unit 922 performs audio data modulation processing, frequency conversion processing, and the like to generate a transmission signal. The communication unit 922 supplies a transmission signal to the antenna 921 and transmits it to a base station (not shown). In addition, the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and supplies the obtained audio data to the audio codec 923. The audio codec 923 performs audio data expansion or conversion into an analog audio signal, and outputs it to the speaker 924.

In the data communication mode, when mail transmission is performed, the control unit 931 receives character data input by operating the operation unit 932 and displays the input characters on the display unit 930. In addition, the control unit 931 generates mail data based on a user instruction or the like in the operation unit 932 and supplies the mail data to the communication unit 922. The communication unit 922 performs mail data modulation processing, frequency conversion processing, and the like, and transmits the obtained transmission signal from the antenna 921. In addition, the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and restores mail data. This mail data is supplied to the display unit 930 to display the mail contents.

Note that the mobile phone 92 can also store the received mail data in a storage medium by the recording / playback unit 929. The storage medium is any rewritable storage medium. For example, the storage medium is a removable medium such as a semiconductor memory such as a RAM or a built-in flash memory, a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card.

When transmitting image data in the data communication mode, the image data generated by the camera unit 926 is supplied to the image processing unit 927. The image processing unit 927 performs image data encoding processing and generates image compression information.

The demultiplexing unit 928 multiplexes the image compression information generated by the image processing unit 927 and the audio data supplied from the audio codec 923 by a predetermined method, and supplies the multiplexed data to the communication unit 922. The communication unit 922 performs modulation processing and frequency conversion processing of multiplexed data, and transmits the obtained transmission signal from the antenna 921. In addition, the communication unit 922 performs amplification, frequency conversion processing, demodulation processing, and the like of the reception signal received by the antenna 921, and restores multiplexed data. This multiplexed data is supplied to the demultiplexing unit 928. The demultiplexing unit 928 performs demultiplexing of the multiplexed data, and supplies image compression information to the image processing unit 927 and audio data to the audio codec 923.

The image processing unit 927 performs a decoding process on the image compression information to generate image data. The image data is supplied to the display unit 930 and the received image is displayed. The audio codec 923 converts the audio data into an analog audio signal, supplies the analog audio signal to the speaker 924, and outputs the received audio.

In the cellular phone device configured as described above, the image processing unit 927 is provided with the functions of the image encoding device (image encoding method) and the image decoding device (image decoding method) of the present application. Therefore, when transmitting an image, for the target block, horizontal prediction motion vector information for the horizontal component of the motion vector information and vertical prediction motion vector information for the vertical component are individually set to improve encoding efficiency. be able to. Also, it is possible to correctly decode the compressed image information generated by the image encoding process.

FIG. 26 illustrates a schematic configuration of a recording / reproducing apparatus to which the present technology is applied. The recording / reproducing apparatus 94 records, for example, audio data and video data of a received broadcast program on a recording medium, and provides the recorded data to the user at a timing according to a user instruction. The recording / reproducing device 94 can also acquire audio data and video data from another device, for example, and record them on a recording medium. Furthermore, the recording / reproducing device 94 decodes and outputs the audio data and video data recorded on the recording medium, thereby enabling image display and audio output on the monitor device or the like.

The recording / reproducing apparatus 94 includes a tuner 941, an external interface unit 942, an encoder 943, an HDD (Hard Disk Drive) unit 944, a disk drive 945, a selector 946, a decoder 947, an OSD (On-Screen Display) unit 948, a control unit 949, A user interface unit 950 is included.

Tuner 941 selects a desired channel from a broadcast signal received by an antenna (not shown). The tuner 941 outputs image compression information obtained by demodulating the received signal of the desired channel to the selector 946.

The external interface unit 942 includes at least one of an IEEE 1394 interface, a network interface unit, a USB interface, a flash memory interface, and the like. The external interface unit 942 is an interface for connecting to an external device, a network, a memory card, and the like, and receives data such as video data and audio data to be recorded.

The encoder 943 performs encoding by a predetermined method when the video data and audio data supplied from the external interface unit 942 are not encoded, and outputs image compression information to the selector 946.

The HDD unit 944 records content data such as video and audio, various programs, and other data on a built-in hard disk, and reads them from the hard disk during playback.

The disk drive 945 records and reproduces signals with respect to the mounted optical disk. An optical disk such as a DVD disk (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.), Blu-ray disk, or the like.

The selector 946 selects any stream from the tuner 941 or the encoder 943 and supplies it to either the HDD unit 944 or the disk drive 945 when recording video or audio. In addition, the selector 946 supplies the stream output from the HDD unit 944 or the disk drive 945 to the decoder 947 when playing back video or audio.

The decoder 947 performs a stream decoding process. The decoder 947 supplies the video data generated by performing the decoding process to the OSD unit 948. The decoder 947 outputs audio data generated by performing the decoding process.

The OSD unit 948 generates video data for displaying a menu screen for selecting an item and the like, and superimposes it on the video data output from the decoder 947 and outputs the video data.

A user interface unit 950 is connected to the control unit 949. The user interface unit 950 includes an operation switch, a remote control signal receiving unit, and the like, and supplies an operation signal corresponding to a user operation to the control unit 949.

The control unit 949 is configured using a CPU, a memory, and the like. The memory stores programs executed by the CPU and various data necessary for the CPU to perform processing. The program stored in the memory is read and executed by the CPU at a predetermined timing such as when the recording / reproducing apparatus 94 is activated. The CPU executes the program to control each unit so that the recording / reproducing device 94 operates in accordance with the user operation.

In the recording / reproducing apparatus configured as described above, the encoder 943 is provided with the function of the image encoding apparatus (image encoding method) of the present application. The decoder 947 is provided with the function of the image decoding apparatus (image decoding method) of the present application. Therefore, when recording an image on a recording medium, encoding efficiency is set by individually setting the predicted horizontal motion vector information for the horizontal component of the motion vector information and the predicted vertical motion vector information for the vertical component for the target block. Can be improved. Also, it is possible to correctly decode the compressed image information generated by the image encoding process.

FIG. 27 illustrates a schematic configuration of an imaging apparatus to which the present technology is applied. The imaging device 96 images a subject and displays an image of the subject on a display unit, or records it on a recording medium as image data.

The imaging device 96 includes an optical block 961, an imaging unit 962, a camera signal processing unit 963, an image data processing unit 964, a display unit 965, an external interface unit 966, a memory unit 967, a media drive 968, an OSD unit 969, and a control unit 970. Have. In addition, a user interface unit 971 and a motion detection sensor unit 972 are connected to the control unit 970. Furthermore, the image data processing unit 964, the external interface unit 966, the memory unit 967, the media drive 968, the OSD unit 969, the control unit 970, and the like are connected via a bus 973.

The optical block 961 is configured using a focus lens, a diaphragm mechanism, and the like. The optical block 961 forms an optical image of the subject on the imaging surface of the imaging unit 962. The imaging unit 962 is configured using a CCD or CMOS image sensor, generates an electrical signal corresponding to the optical image by photoelectric conversion, and supplies the electrical signal to the camera signal processing unit 963.

The camera signal processing unit 963 performs various camera signal processing such as knee correction, gamma correction, and color correction on the electrical signal supplied from the imaging unit 962. The camera signal processing unit 963 supplies the image data after the camera signal processing to the image data processing unit 964.

The image data processing unit 964 performs an encoding process on the image data supplied from the camera signal processing unit 963. The image data processing unit 964 supplies the compressed image information generated by performing the encoding process to the external interface unit 966 and the media drive 968. Further, the image data processing unit 964 performs a decoding process on the compressed image information supplied from the external interface unit 966 and the media drive 968. The image data processing unit 964 supplies the image data generated by performing the decoding process to the display unit 965. Further, the image data processing unit 964 superimposes the processing for supplying the image data supplied from the camera signal processing unit 963 to the display unit 965 and the display data acquired from the OSD unit 969 on the image data. To supply.

The OSD unit 969 generates display data such as a menu screen and icons made up of symbols, characters, or figures and outputs them to the image data processing unit 964.

The external interface unit 966 includes, for example, a USB input / output terminal, and is connected to a printer when printing an image. In addition, a drive is connected to the external interface unit 966 as necessary, a removable medium such as a magnetic disk or an optical disk is appropriately mounted, and a program read from the medium is installed as necessary. Furthermore, the external interface unit 966 has a network interface connected to a predetermined network such as a LAN or the Internet. For example, the control unit 970 reads the image compression information from the memory unit 967 according to an instruction from the user interface unit 971, and supplies the compressed image information from the external interface unit 966 to another device connected via the network. it can. Also, the control unit 970 may acquire image compression information and image data supplied from another device via the network via the external interface unit 966 and supply the acquired information to the image data processing unit 964. it can.

As the recording medium driven by the media drive 968, any readable / writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory is used. The recording medium may be any type of removable medium, and may be a tape device, a disk, or a memory card. Of course, a non-contact IC card or the like may be used.

Further, the media drive 968 and the recording medium may be integrated, and may be configured by a non-portable storage medium such as a built-in hard disk drive or an SSD (Solid State Drive).

The control unit 970 is configured using a CPU, a memory, and the like. The memory stores programs executed by the CPU, various data necessary for the CPU to perform processing, and the like. The program stored in the memory is read and executed by the CPU at a predetermined timing such as when the imaging device 96 is activated. The CPU executes the program to control each unit so that the imaging device 96 operates according to the user operation.

In the imaging device configured as described above, the image data processing unit 964 is provided with the functions of the image encoding device (image encoding method) and the image decoding device (image decoding method) of the present application. Therefore, when recording a captured image, the horizontal prediction motion vector information for the horizontal component of the motion vector information and the vertical prediction motion vector information for the vertical component are individually set for the target block to improve the encoding efficiency. Can be made. Also, it is possible to correctly decode the compressed image information generated by the image encoding process.

Furthermore, a motion detection sensor unit 972 configured using a gyro or the like is provided in the imaging device 96, and a code with a small amount of data is predicted with high accuracy based on the detection result of motion such as panning or tilting of the imaging device 96. Assigned to predicted motion vector information. In this way, the coding efficiency can be further improved by dynamically allocating codes according to the motion detection result of the imaging apparatus.

Note that the present technology should not be interpreted as being limited to the above-described embodiment. This embodiment discloses the present technology in the form of exemplification, and it is obvious that those skilled in the art can make modifications and substitutions of the embodiment without departing from the gist of the present technology. In other words, the scope of the claims should be considered in order to determine the gist of the present technology.

In the image encoding device and the motion vector encoding method of this technique, and the image decoding device and the motion vector decoding method, the code adjacent to the target block for each of the horizontal component and the vertical component of the motion vector information of the target block. The motion vector information is selected from the converted block, and the horizontal prediction motion vector information and the vertical prediction motion vector information are set, respectively, and the motion of the target block is set using the set horizontal prediction motion vector information and vertical prediction motion vector information. Vector information compression processing is performed. Also, horizontal prediction block information and vertical prediction block information indicating a block for which motion vector information is selected are generated. Further, the motion vector information is decoded based on the horizontal prediction block information and the vertical prediction block information. For this reason, for example, horizontal prediction motion vector information and vertical prediction motion vector information with horizontal prediction block information and vertical prediction block information that have a data amount smaller than a flag corresponding to a combination of horizontal prediction motion vector information and vertical prediction motion vector information candidates Can be set, and the encoding efficiency can be improved. Accordingly, since high encoding efficiency can be realized, image compression information (bitstream) is transmitted / received via network media such as satellite broadcasting, cable TV, the Internet, and cellular phones, or optical disks, magnetic disks, and flash memories. It is suitable for an apparatus for recording and reproducing images using a storage medium such as

DESCRIPTION OF SYMBOLS 10 ... Image coding apparatus, 11 ... A / D conversion part, 12, 57 ... Screen rearrangement buffer, 13 ... Subtraction part, 14 ... Orthogonal transformation part, 15 ... Quantum 16, lossless encoding unit 17, 51, accumulation buffer 18, rate control unit 21, 53, inverse quantization unit 22, 54, inverse orthogonal transform unit , 23, 55 ... adder, 24, 56 ... deblocking filter, 25, 61 ... frame memory, 26, 62, 75 ... selector, 31, 71 ... intra prediction unit, 32 ... Motion prediction / compensation unit, 33, 33a, 73, 73a ... Prediction motion vector information setting unit, 35 ... Predicted image / optimum mode selection unit, 50 ... Image decoding device, 52 ...・ Lossless decoding unit, 58... D / A conversion unit, 72... Motion compensation unit 80: Computer device, 90: Television device, 92: Mobile phone, 94: Recording / reproducing device, 96: Imaging device, 321: Motion search unit, 322: Cost Function value calculation unit, 323 ... mode determination unit, 324 ... motion compensation processing unit, 325 ... motion vector buffer, 331, 731 ... horizontal prediction motion vector information generation unit, 332, 732 ... Vertical prediction motion vector information generation unit, 333, 733... Vertical and vertical prediction motion vector information generation unit, 334, 334a... Identification information generation unit, 721... Block size information buffer, 722. Information buffer, 723 ... motion vector information generation unit, 724 ... motion compensation processing unit, 725 ... motion vector information buffer, 730, 730a ... flag buffer

Claims

Horizontal prediction block information indicating a block in which motion vector information is selected as horizontal prediction motion vector information from a decoded block adjacent to the target block, and vertical prediction indicating a block in which motion vector information is selected as vertical prediction motion vector information A lossless decoding unit that acquires block information from image compression information;
Prediction motion in which motion vector information of a block indicated by the horizontal prediction block information is set as horizontal prediction motion vector information, and motion vector information of a block indicated by the vertical prediction block information is set as the vertical prediction motion vector information A vector information setting section;
An image decoding apparatus comprising: a motion vector information generation unit configured to generate motion vector information of the target block using the horizontal prediction motion vector information and vertical prediction motion vector information set by the prediction motion vector information setting unit.
The lossless decoding unit may select a motion vector selected from the adjacent decoded block with respect to the horizontal prediction motion vector information and the vertical prediction motion vector information, or a horizontal component and a vertical component of the motion vector information of the target block. Identification information indicating which of the horizontal and vertical predicted motion vector information indicating the information is used is acquired from the image compression information,
The predicted motion vector information setting unit sets the horizontal predicted motion vector information and the vertical predicted motion vector information, or the horizontal and vertical predicted motion vector information based on the identification information,
The image decoding according to claim 1, wherein the motion vector information generation unit generates motion vector information of the target block using the horizontal prediction motion vector information and vertical prediction motion vector information, or the horizontal and vertical prediction motion vector information. apparatus.
The lossless decoding unit decodes a code included in image compression information, acquires the horizontal prediction block information and the vertical prediction block information,
The image decoding apparatus according to claim 1, wherein the prediction motion vector information setting unit sets the horizontal prediction motion vector and the vertical prediction motion vector information based on the horizontal prediction block information and the vertical prediction block information.
Horizontal prediction block information indicating a block in which motion vector information is selected as horizontal prediction motion vector information from a decoded block adjacent to the target block, and vertical prediction indicating a block in which motion vector information is selected as vertical prediction motion vector information Obtaining block information from image compression information;
Setting the motion vector information of the block indicated by the horizontal prediction block information as horizontal prediction motion vector information, and setting the motion vector information of the block indicated by the vertical prediction block information as the vertical prediction motion vector information; ,
A motion vector information decoding method comprising: a step of generating motion vector information of the target block using the set horizontal prediction motion vector information and vertical prediction motion vector information.
For each of the horizontal and vertical components of the motion vector information of the target block, select the motion vector information from the encoded block adjacent to the target block, and set the horizontal prediction motion vector information and the vertical prediction motion vector information. An image coding apparatus comprising: a prediction motion vector information setting unit that performs horizontal prediction block information and vertical prediction block information indicating a block in which the motion vector information is selected.
The predicted motion vector information setting unit selects and sets motion vector information having the highest encoding efficiency in the horizontal component encoding process as the horizontal predicted motion vector information, and is the highest in the vertical component encoding process. 6. The image encoding device according to claim 5, wherein motion vector information with high encoding efficiency is selected and set as the vertical predicted motion vector information.
A cost function value calculation unit for calculating a cost function value for each prediction mode;
A mode determination unit for determining the optimal prediction mode;
The image encoding device according to claim 6, wherein the mode determination unit determines a mode in which the calculated cost function value is a minimum as an optimal prediction mode.
The image encoding device according to claim 5, wherein the horizontal prediction block information and the vertical prediction block information are transmitted by being included in image compression information.
The predicted motion vector information setting unit sets, as horizontal and vertical predicted motion vector information, motion vector information selected from an encoded block adjacent to the target block with respect to a horizontal component and a vertical component of motion vector information of the target block. The image encoding device according to claim 5, wherein the setting to be performed or the setting of the predicted horizontal motion vector information and the predicted vertical motion vector information can be switched for each picture or for each slice.
The prediction motion vector information setting unit generates identification information indicating which of the horizontal prediction motion vector information, the vertical prediction motion vector information, or the horizontal and vertical prediction motion vector information is used. Image encoding device.
The image encoding device according to claim 10, wherein the generated identification information is included in a picture parameter set or slice header of image compression information.
The prediction motion vector information setting unit sets the horizontal prediction motion vector information and the vertical prediction motion vector information for a P picture, and sets the horizontal and vertical prediction motion vector information for a B picture. 9. The image encoding device according to 9.
A lossless encoding unit that encodes motion vector information of the target block;
6. The lossless encoding unit performs different code assignments between the horizontal prediction block information and the vertical prediction block information, and includes the codes assigned to the horizontal prediction block information and the vertical prediction block information in image compression information. The image encoding device described.
The lossless encoding unit performs prediction block information indicating a block in which motion vector information is selected as spatial prediction motion vector information, and prediction block information indicating a block in which motion vector information is selected as temporal prediction motion vector information. The image encoding device according to claim 13, wherein different code allocation is performed for the horizontal prediction block information and the vertical prediction block information.
The lossless encoding unit performs the code assignment based on the motion detection result of the imaging device when performing the encoding process of the motion vector information of the target block detected using the image data generated by the imaging device. 15. The image encoding device according to claim 14, wherein the image encoding device is performed.
For each of the horizontal and vertical components of the motion vector information of the target block, select the motion vector information from the encoded block adjacent to the target block, and set the horizontal prediction motion vector information and the vertical prediction motion vector information. A motion vector information encoding method comprising a step of generating horizontal prediction block information and vertical prediction block information indicating a block for which the motion vector information is selected.