JP2015186052A - Stereoscopic video encoding device and stereoscopic video encoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stereoscopic video encoding device and method capable of reducing hindrance to a correct stereoscopic effect even when bit rate for coding is not enough.SOLUTION: A device which encodes stereoscopic video including right-eye images and left-eye images includes: an encoding part which performs in-screen encoding in which a predictive image is generated using pixel values adjoining a block to be encoded and already having been encoded and differences between pixel values of the block to be encoded and the predictive image are encoded, and an in-screen prediction mode selection part which selects one of a plurality of available prediction modes for in-screen encoding. The in-screen prediction mode selection part selects a prediction mode on the basis of prediction mode coefficients set for the respective prediction modes, and weights prediction mode coefficients such that a prediction mode coefficient for a prediction mode in which a predictive image is generated using horizontally adjacent pixels of the blocks to be encoded is less selected than prediction mode coefficients for other prediction modes.

Description

  The present invention relates to an encoding apparatus and method for encoding video, and particularly to an encoding technique for encoding stereoscopic video using a plurality of videos taken by a plurality of imaging devices.

  In recent years, with the spread of displays capable of stereoscopic display, attention has been focused on imaging apparatuses capable of capturing stereoscopic images. The stereoscopic image is usually created using a right-eye image and a left-eye image captured by two cameras having parallax. In stereoscopic video coding, in addition to motion prediction for predicting motion vectors of objects in the time direction used in the MPEG method known as a conventional compression coding / decoding technology, a parallax vector between a plurality of imaging devices is also calculated. The parallax prediction to be predicted can be used (see Patent Document 1).

JP 2000-165909 A

  However, in the above prior art, especially when the encoding bit rate is not sufficient and the encoding deterioration becomes remarkable, the parallax between the right eye image and the left eye image is disturbed due to the encoding deterioration, and the correct stereoscopic effect is obtained. There was a problem that was disturbed.

  Accordingly, the present invention has been made in view of the above problems, and provides a stereoscopic video encoding apparatus and method that can reduce the inhibition of the correct stereoscopic effect even when the encoding bit rate is not sufficient. For the purpose.

  The configuration of the stereoscopic video encoding device according to the present invention is such that a stereoscopic video encoding device that encodes a stereoscopic video including a right-eye image and a left-eye image uses an already encoded pixel value adjacent to an encoding target block. There are a plurality of prediction modes for generating the predicted image and performing the intra-frame encoding for encoding the difference between the pixel value of the encoding target block and the pixel value of the predicted image, and the prediction mode of the intra-screen encoding, An intra-screen prediction mode selection unit that selects one prediction mode from among the plurality of prediction modes, and the intra-screen prediction mode selection unit predicts based on a prediction mode coefficient set for each prediction mode. A prediction mode coefficient for a prediction mode in which a mode is selected and a prediction image is generated using pixels adjacent in the horizontal direction of the encoding target block, and a prediction mode coefficient for another prediction mode is selected. More, wherein the weighting to be less likely to be selected.

  The configuration of the stereoscopic video encoding method according to the present invention is a stereoscopic video encoding method for encoding a stereoscopic video including a right-eye image and a left-eye image, using already encoded pixel values adjacent to the encoding target block. There are a plurality of encoding steps for generating a prediction image and performing intra-screen encoding for encoding the difference between the pixel value of the encoding target block and the pixel value of the prediction image, and a plurality of prediction modes of the intra-screen encoding, An intra-screen prediction mode selection step of selecting one prediction mode from the plurality of prediction modes, and in the intra-screen prediction mode selection step, based on the prediction mode coefficient set for each prediction mode A prediction mode coefficient for a prediction mode that selects a prediction mode and generates a prediction image using pixels adjacent to the encoding target block in the horizontal direction is a prediction mode for another prediction mode. Than the number, and wherein the weighting to be less likely to be selected.

  ADVANTAGE OF THE INVENTION According to this invention, even if it is a case where the bit rate of encoding is not enough, the stereoscopic video encoding apparatus which can reduce inhibition of a correct stereoscopic effect can be provided.

It is a block diagram which shows the structural example of the stereo image coding apparatus which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the production | generation method of the estimated image of an intra prediction method. It is a block diagram of the intra estimation part which concerns on the 1st Embodiment of this invention. It is a block diagram of the intra / inter determination part which concerns on the 2nd Embodiment of this invention. It is a block diagram of the intra estimation part which concerns on the 3rd Embodiment of this invention. It is a figure which shows the relationship between a prediction mode coefficient and an encoding bit rate.

  Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram of a stereoscopic video encoding apparatus according to the present invention. As shown in FIG. 1, the stereoscopic video encoding apparatus according to the present invention includes a main image capturing unit 101, a sub image capturing unit 102, a frame memory 103, a filtered reference frame memory 104, a motion / disparity prediction unit 105, Disparity compensation unit 106, intra prediction unit 107, orthogonal transform unit 108, quantization unit 109, entropy coding unit 110, inverse quantization unit 111, inverse orthogonal transform unit 112, intra / inter determination unit 113, subtractor 114, addition , A pre-filter reference frame memory 116, and a loop filter 117.

  In such a configuration, first, a method for encoding an input image will be described. The main image capturing unit 101 and the sub image capturing unit 102 include an image pickup lens and an image pickup device for photoelectrically converting the formed image, an A / D conversion processing unit for converting an analog signal read from the image pickup device into a digital signal, and the like. In addition, a luminance signal and a color difference signal are generated from the converted digital signal, and data is written to the frame memory 103. In this embodiment, it is assumed that the main image captures a left-eye image, and the sub-image captures a right-eye image. The sub image can use the main image as a reference image, but the main image cannot use the sub image as a reference image.

  The main image and the sub image are stored in the frame memory 103 in the display order, and the encoding target block is sequentially transmitted to the motion / disparity prediction unit 105, the intra prediction unit 107, the subtractor 114, and the origin difference calculation unit 118 in the encoding order. To do.

  The filtered reference frame memory 104 stores the filtered encoded image as a reference image, and sequentially transmits the reference image of the encoding target block to the motion / disparity prediction unit 105 and the motion / disparity compensation unit 106 in the encoding order. .

  The pre-filter reference frame memory 116 stores the encoded image before being filtered and sequentially transmits it to the intra prediction unit 107.

  The motion / disparity prediction unit 105 changes processing depending on whether the encoding target block transmitted from the frame memory 103 is a main image or a sub image.

  In the case of the main image, the pre-encoded and decoded main image transmitted from the post-filter reference frame memory 104 is used as a reference image, detected as a motion vector, and transmitted to the motion / disparity compensation unit 106 together with the reference image data number. To do.

  In the case of a sub-image, the received motion vector or disparity vector is received from the filtered reference frame memory 104 using the main image at the same time in the display order encoded and decoded in advance or the sub-image previously encoded and decoded as a reference image. To detect. The detected motion vector or disparity vector is transmitted to the motion / disparity compensation unit 106 together with the reference image data number.

  In addition, in the case of a sub-image, one of the reference images is selected as a final reference image from the main image at the same time in the display order encoded and decoded in advance and the sub-image encoded and decoded in advance. There is no particular limitation on how to do this. In this embodiment, it is assumed that the smaller difference value between the encoding target block and the reference image at the vector position is selected.

  The motion / disparity compensation unit 106 uses the motion vector or the disparity vector transmitted from the motion / disparity prediction unit 105 to refer to the reference frame image indicated by the filtered reference frame image data number in the filtered reference frame memory 104. Thus, predicted image data of each block is generated and transmitted to the intra / inter determination unit 113 together with the motion vector or the disparity vector.

  On the other hand, the intra prediction unit 107 generates an intra prediction image for each of a plurality of intra prediction modes using the decoded data around the encoding target block transmitted from the pre-filter reference frame memory 116. FIG. 2 schematically shows the type of intra prediction and the method of generating a predicted image in each intra prediction method. A white square indicates a block to be encoded, and here, a block of four horizontal pixels and four vertical pixels is used. A black square indicates an already encoded pixel adjacent to the block to be encoded. A predicted image is generated using the adjacent pixel values in the direction of the arrow. For example, in FIG. 2 (a) Vertical prediction, a predicted image is generated using upper adjacent pixels. In addition, when an arrow is slanting, there exists what produces | generates an estimated image by a calculation using a several adjacent pixel value.

  An appropriate intra prediction mode is selected using the encoding target block transmitted from the frame memory 103 and the generated predicted image, and is transmitted to the intra / inter determination unit 113 together with the predicted image.

  The intra / inter determination unit 113 selects predicted image data having a high correlation with the encoding target block from the predicted image data transmitted from the motion / disparity compensation unit 106 and the intra prediction unit 107, and transmits the selected predicted image data to the subtracter 114. . Further, when intra prediction is selected, information such as an intra prediction mode is transmitted to the entropy coding unit 110, and when inter prediction is selected, information such as a motion vector is transmitted.

  The subtracter 114 subtracts the encoding target block transmitted from the frame memory 103 and the predicted image block transmitted from the intra / inter determination unit 113, and outputs image residual data.

  The orthogonal transform unit 108 performs orthogonal transform processing on the image residual data output from the subtractor 114 and transmits transform coefficients to the quantization unit 109.

  The quantization unit 109 quantizes the transform coefficient from the orthogonal transform unit 108 using a predetermined quantization parameter, and transmits the quantized coefficient to the entropy coding unit 110 and the inverse quantization unit 111.

  The entropy encoding unit 110 receives the transform coefficient quantized by the quantization unit 109, performs entropy encoding such as CAVLC, CABAC, and outputs the encoded data.

  Next, a method for generating reference image data using the transform coefficient quantized by the quantization unit 109 will be described.

  The inverse quantization unit 111 performs inverse quantization on the quantized transform coefficient transmitted from the quantization unit 109.

  The inverse orthogonal transform unit 112 performs inverse orthogonal transform on the transform coefficient inversely quantized by the inverse quantization unit 111, generates decoded residual data, and transmits the decoded residual data to the adder 115.

  The adder 115 adds decoded residual data and predicted image data described later to generate reference image data, and stores the reference image data in the pre-filter reference frame memory 116. Also, it is transmitted to the loop filter 117.

  The loop filter 117 stores the filtered reference image data obtained by filtering the reference image data to remove noise in the filtered reference frame memory 104.

  Next, a method for determining an intra prediction mode in the intra prediction unit 107, which is a feature of the present invention, will be described in detail. FIG. 3 is a block diagram of the intra prediction unit 107. The prediction mode generation unit 304 sequentially selects one of the plurality of prediction modes illustrated in FIG. 2 and transmits the selected prediction mode to the prediction image generation unit 301 and the prediction mode coefficient generation unit 305. The predicted image generation unit 301 reads the already-encoded adjacent pixels necessary for generating a predicted image of the designated prediction mode from the pre-filter reference frame memory 116, generates a predicted image, and generates an evaluation value calculation unit. To 302.

  The prediction mode coefficient generation unit 305 generates a prediction mode coefficient according to the designated prediction mode. For example, 2.0 is generated when the prediction mode is Mode 1 (Horizontal prediction), and 1.0 is generated in other prediction modes. Other generation examples of the prediction mode coefficient include 2.0 when the prediction mode is Mode 1, 1.5 when Mode 4, 4, 6, and 8 and 1.0 when other prediction modes are used. It may occur. The generation of the prediction mode coefficient is not limited to the above example, and the prediction mode coefficient only needs to be larger as the prediction direction has a stronger horizontal component.

  The evaluation value calculation unit 302 reads out the block image to be encoded from the frame memory 103 and calculates a difference value from the prediction image generated by the prediction image generation unit 301. As the difference value, a sum of absolute differences is calculated by adding the absolute value of the difference value for each pixel by the number of pixels of the block. The difference absolute value sum is multiplied by the prediction mode coefficient generated by the prediction mode coefficient generation unit 305 to calculate an evaluation value for the prediction mode generated by the prediction mode generation unit 304 and transmitted to the prediction mode determination unit 303. . The prediction mode determination unit 303 determines the smallest evaluation value for each prediction mode, which is sequentially transmitted, as the prediction mode for the block to be encoded, and transmits it to the intra / inter determination unit 113 together with the predicted image.

  As a result, it becomes difficult to select a prediction mode in which a prediction image of intra prediction is extended in the horizontal direction, and as a result, encoding deterioration that is extended in the horizontal direction is suppressed, and the inhibition of correct stereoscopic effect is reduced. can do.

(Second Embodiment)
Hereinafter, a stereoscopic video encoding apparatus according to the second embodiment of the present invention will be described. Since the configuration of the stereoscopic video encoding apparatus according to the second embodiment of the present invention is the same as that of the first embodiment described above, description thereof is omitted.

  FIG. 4 is a block diagram of the intra / inter determination unit 113. The intra / inter determination unit 113 receives the inter difference from the motion / disparity compensation unit 106 and the intra difference from the intra prediction unit 107. The inter difference and the intra difference are sums of absolute differences between respective predicted images and encoded images. The intra difference is multiplied by a predetermined intra coefficient and input to the intra / inter determination unit 401 as an intra evaluation value. Here, the intra coefficient is 2.0. Note that the value of the intra coefficient only needs to be larger than 1.0. One inter difference is directly input to the intra / inter determination unit 401 as an inter evaluation value. The intra / inter determination unit 401 determines the smaller of the inter evaluation value and the intra evaluation value as a prediction method, and transmits the prediction method to the entropy encoding unit 110.

  As described above, it is difficult to select intra prediction, encoding deterioration that is likely to occur in the case of intra prediction and extending in the horizontal direction is suppressed, and inhibition of correct stereoscopic effect can be reduced.

(Third embodiment)
Hereinafter, a stereoscopic video encoding apparatus according to the third embodiment of the present invention will be described. Since the configuration of the stereoscopic video encoding apparatus according to the third embodiment of the present invention is substantially the same as that of the first embodiment, the description thereof is omitted.

  By replacing the intra prediction unit 107 in FIG. 1 with the intra prediction 500 in FIG. 5, a block diagram of the stereoscopic video encoding apparatus according to the third embodiment of the present invention is obtained. The intra prediction unit 500 has substantially the same configuration as the intra prediction unit 107, and the prediction mode coefficient generation unit 505 is different. The prediction mode coefficient generation unit 505 receives a coding bit rate set by a user or the like, and generates a prediction mode coefficient according to the coding bit rate. As described in the first embodiment, the prediction mode coefficient only needs to be larger as the prediction direction has a higher horizontal component. However, the higher the encoding bit rate, the smaller the difference between the maximum and minimum prediction mode coefficients for each prediction mode. Here, as an example, changes in the prediction mode coefficient of Mode 0 and the prediction mode coefficient of Mode 1 according to the encoding bit rate will be described with reference to FIG.

  FIG. 6 is a graph showing the prediction mode coefficient of Mode 0 and the prediction mode coefficient of Mode 1 with the encoding bit rate on the horizontal axis. When the encoding bit rate is high, the prediction mode coefficient of Mode 0 is small and is close to the prediction mode coefficient of Mode 1. When the coding bit rate is low, the prediction mode coefficient of Mode 1 is large and becomes a value different from the prediction mode coefficient of Mode 0. That is, when the encoding bit rate is low, the prediction mode that is a prediction image extended in the horizontal direction is strongly suppressed.

  As described above, it is possible to suppress the deterioration of encoding such as extending in the horizontal direction, which becomes remarkable particularly when the encoding bit rate is low, and to reduce the inhibition of the correct stereoscopic effect.

101 Main Image Imaging Unit 102 Sub Image Imaging Unit 103 Frame Memory 104 Filtered Reference Frame Memory 105 Motion / Parallax Prediction Unit 106 Parallax Compensation Unit 107 Intra Prediction Unit 108 Orthogonal Transform Unit 109 Quantization Unit 110 Entropy Coding Unit 111 Inverse Quantization Unit 112 inverse orthogonal transform unit 113 intra / inter determination unit 114 subtractor 115 adder 116 pre-filter reference frame memory 117 loop filter

Claims (5)

In a stereoscopic video encoding device that encodes a stereoscopic video including a right-eye image and a left-eye image,
A prediction image is generated using already encoded pixel values adjacent to the encoding target block, and intra-screen encoding is performed to encode a difference between the pixel value of the encoding target block and the pixel value of the prediction image. Encoding means;
There are a plurality of prediction modes of the intra-picture encoding, and it comprises an intra-screen prediction mode selection means for selecting one prediction mode from the plurality of prediction modes,
The intra prediction mode selection means selects a prediction mode based on a prediction mode coefficient set for each prediction mode,
The prediction mode coefficient for a prediction mode for generating a prediction image using pixels adjacent in the horizontal direction of the encoding target block is weighted so as to be less likely to be selected than prediction mode coefficients for other prediction modes. Stereoscopic video encoding device. Each prediction mode coefficient is weighted such that the larger the ratio of the number of pixels adjacent in the horizontal direction of the encoding target block to the number of pixels adjacent in the vertical direction, the more difficult it is to select. The stereoscopic video encoding apparatus according to claim 1. The encoding means can select an encoding bit rate,
The lower the encoding bit rate, the greater the difference between the prediction mode coefficient for the prediction mode for generating a prediction image using pixels adjacent in the horizontal direction of the encoding target block and the prediction mode coefficient for other prediction modes. 3. The stereoscopic video encoding apparatus according to claim 1 or 2, wherein weighting is performed as described above. The encoding by the encoding means includes motion compensation encoding using an already encoded image that is temporally different from the encoding target image as a reference image, and the intra-frame encoding in units of the encoding block. The stereoscopic video encoding apparatus according to any one of claims 1 to 3, wherein the motion compensation encoding can be selected. In a stereoscopic video encoding method for encoding a stereoscopic video including a right eye image and a left eye image,
A prediction image is generated using already encoded pixel values adjacent to the encoding target block, and intra-screen encoding is performed to encode a difference between the pixel value of the encoding target block and the pixel value of the prediction image. Encoding process;
There are a plurality of prediction modes of the intra-picture encoding, and it has an intra-screen prediction mode selection step of selecting one prediction mode from the plurality of prediction modes,
In the intra prediction mode selection step, select a prediction mode based on the prediction mode coefficient set for each prediction mode,
The prediction mode coefficient for a prediction mode for generating a prediction image using pixels adjacent in the horizontal direction of the encoding target block is weighted so as to be less likely to be selected than prediction mode coefficients for other prediction modes. A stereoscopic video encoding method.