JPWO2012172634A1 - Image encoding apparatus, image decoding apparatus, method, and program - Google Patents

Abstract

The image encoding device according to the embodiment includes a determination unit, an acquisition unit, a generation unit, and an encoding unit. The determining unit assigns a corresponding block corresponding to the encoding target block in the first parallax image of the viewpoint in the encoding target image in one or a plurality of second parallax images of other viewpoints different from the viewpoint. The corresponding block in each of the one or the plurality of second parallax images, determined based on the depth information of the parallax image and the positional relationship between the viewpoint of the first parallax image and the viewpoint of the second parallax image; In addition, a shared block that shares encoding information is determined from among blocks in a small area including the vicinity of the corresponding block, and shared block designation information that indicates a positional relationship between the corresponding block and the shared block is generated. The acquisition unit acquires the encoding information shared by the shared block based on the shared block designation information. The generation unit generates a predicted image based on the acquired encoded information. The encoding unit generates encoded data from the image and the predicted image.

Description

  Embodiments described herein relate generally to an image encoding device, an image decoding device, a method, and a program.

  Conventionally, as a multi-parallax image encoding / decoding device, a block corresponding to a block to be encoded is determined from encoded images of other viewpoints by projective transformation using camera parameters and depth information. A device that shares information specifying a motion vector and a reference image of a block is known.

  However, in such a conventional technique, there is a possibility of erroneous determination of a block corresponding to a block to be encoded due to an estimation error of depth information, compression distortion added thereto, and an error due to projective transformation. In addition, encoding information that can be shared is limited to only motion information. For this reason, it is difficult for the conventional multi-parallax image encoding / decoding device to improve the encoding efficiency.

Special table 2010-515400

Jacek Konieczny, Marek Domanski: Depth-based Inter-view Prediction of Motion Vectors for Improved Multiview Video Coding, 3DTV-Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON), pp.1-4, 2010.

  The problem to be solved by the present embodiment is to provide an image encoding device, an image decoding device, a method, and a program capable of improving encoding efficiency.

  The image encoding device according to the embodiment includes a determination unit, an acquisition unit, a generation unit, and an encoding unit. The determining unit assigns a corresponding block corresponding to the encoding target block in the first parallax image of the viewpoint in the encoding target image in one or a plurality of second parallax images of other viewpoints different from the viewpoint. The corresponding block in each of the one or the plurality of second parallax images, determined based on the depth information of the parallax image and the positional relationship between the viewpoint of the first parallax image and the viewpoint of the second parallax image; In addition, a shared block that shares encoding information is determined from among blocks in a small area including the vicinity of the corresponding block, and shared block designation information that indicates a positional relationship between the corresponding block and the shared block is generated. The acquisition unit acquires the encoding information shared by the shared block based on the shared block designation information. The generation unit generates a predicted image based on the acquired encoded information. The encoding unit generates encoded data from the image and the predicted image.

1 is a diagram of an image encoding device according to Embodiment 1. FIG. FIG. 4 is a diagram illustrating an example of encoding according to the first embodiment. FIG. 6 is a diagram illustrating an example of determining a corresponding macroblock according to the first embodiment. FIG. 6 is a diagram illustrating an example of determining a shared block according to the first embodiment. FIG. 4 is an example of shared block designation information according to the first embodiment. FIG. 5 is a diagram illustrating an example of sharing encoded information according to the first embodiment. 5 is a flowchart of encoding processing according to the first embodiment. 5 is a flowchart of prediction processing according to the first embodiment. The figure of the image decoding apparatus of Embodiment 2. FIG. 10 is a flowchart of decoding processing according to the second embodiment. 10 is a flowchart of prediction processing according to the second embodiment.

(Embodiment 1)
In the first embodiment, an input image that is an encoding target image is input, the input image is divided into macro blocks, and the decoded disparity of the viewpoint different from the viewpoint of the input image with respect to the encoding target macro block It is an image encoding apparatus that performs encoding processing by sharing image encoding information.

  FIG. 1 is a block diagram illustrating a functional configuration of the image coding apparatus according to the first embodiment. As shown in FIG. 1, the image coding apparatus 100 according to the present embodiment includes a control unit 116 and a coding unit 117.

  The encoding unit 117 receives the input image I (v) 101 and divides the input image I (v) 101 into macroblocks. The encoding unit 117 generates a predicted image using the encoded parallax image information of a viewpoint different from the viewpoint of the input image for the divided macroblock. The encoding unit 117 outputs encoded data S (v) 104 obtained by encoding the residual information between the predicted image and the input image I (v) 101. The encoding control unit 116 controls the entire image encoding process by the encoding unit 117.

  FIG. 2 is an explanatory diagram of an example of the image encoding process. In FIG. 2, when the viewpoint of the input image is 2 and the other viewpoints are 0, the encoding unit 117 inputs the encoded data S (0) of the other viewpoint 0 and the corresponding depth information D (2). A predicted image corresponding to viewpoint 2 of the image is generated.

  As shown in FIG. 1, the encoding unit 117 includes a subtractor 111, a transform / quantization unit 115, a variable length encoding unit 118, an inverse transform / inverse quantization unit 114, an adder 113, and a prediction. Unit 112 and buffer 105.

  An input image I (v) 101 is input to the encoding unit 117. The subtractor 111 calculates a difference between the predicted image generated by the prediction unit 112 and the input image I (v) 101, and generates a residual that is this difference.

  The transform / quantization unit 115 obtains transform coefficients by orthogonally transforming the residuals, and obtains residual information by quantizing the transform coefficients. Here, as the orthogonal transformation, for example, discrete cosine transformation can be used. The residual information is input to the variable length coding unit 118 and the inverse transform / inverse quantization 114.

  Inverse transform / inverse quantization 114 performs a process of inverse quantization and inverse orthogonal transform on the residual information to reproduce a locally decoded image. The adder 113 adds the reproduced local decoded image and the predicted image to generate a decoded image. This decoded image is stored in the buffer 105 as a reference image.

  Here, the buffer 105 is a storage medium such as a frame memory. In the buffer 105, the above-described decoded image is stored as a reference image or the like, and the decoded parallax image R (v ') of another viewpoint is stored.

  The prediction unit 112 generates a prediction image from the decoded parallax image R (v ′) of another viewpoint stored in the buffer 105. The prediction unit 112 includes a determination unit 106, an acquisition unit 107, and a generation unit 108.

  The determination unit 106 stores the shared block that shares the encoding information when the encoding target macroblock is encoded as one or a plurality of parallax images (that is, stored in the buffer 105) of other viewpoints different from the viewpoint of the input image. The depth information D (v) of the input image I (v) 101, the viewpoint position of the input image I (v) 101, and decoding from among the decoded other viewpoint parallax images R (v ′)) It is determined based on the positional relationship between the viewpoint positions of the parallax images R (v ′) of other viewpoints that have already been completed. Here, the positional relationship between the viewpoint positions is the positional relationship between the camera that captured the input image I (v) 101 and the camera that captured the decoded parallax image R (v ′) of another viewpoint. It may be. More specifically, the shared block determination unit 105 determines a shared block as follows.

  First, the determination unit 106 adds the decoded other viewpoint parallax image R (v ′) stored in the buffer 105 to the macroblock to be encoded in the viewpoint parallax image in the input image I (v) 101. ) To determine a corresponding macroblock which is a corresponding macroblock.

  Specifically, the determination unit 106 inputs depth information D (v) 103 of the input image I (v) 101, and determines a shared block using the depth information D (v) 103 and camera parameters.

  FIG. 3 is a diagram for explaining the corresponding macroblock determination process. The determination unit 106 calculates the coordinates of the corresponding macroblock corresponding to the encoding target macroblock of the input image I (v) 101 by projective transformation using the following equations (1) and (2).

  Here, R, A, and T are all camera parameters, R represents a camera rotation matrix, A represents an internal camera matrix, and T represents a parallel progression sequence of the cameras. Z indicates a depth value which is depth information D (v) of the input image I (v) 101.

  In other words, the determination unit 106 projects the encoding target macroblock in the parallax image in the camera Ci at the viewpoint of the encoding target macroblock to the three-dimensional space coordinates by the expression (1). After that, the determination unit 106 performs back projection on the parallax image of the camera Cj of another viewpoint according to the equation (2), so that the parallax image R (v ′) (reference image) of the other viewpoint has been decoded. Determine the corresponding macroblock. If no corresponding macroblock is found, the projective transformation process is performed on the next reference image in the next buffer 105.

  Then, the determination unit 106 selects, as a search range, a small area including a corresponding macroblock of a parallax image R (v ′) of another decoded viewpoint and a neighboring macroblock centered on the corresponding macroblock.

  Then, the determination unit 106 obtains encoding information from the macroblocks in each small area of the reference image (the decoded parallax image R (v ′) of another viewpoint) stored in the buffer 105. A shared block that is a macroblock to be shared is determined.

  More specifically, the determination unit 106 calculates the RD cost when the encoding information of each macroblock in the small area is shared.

  FIG. 4 is a diagram for explaining shared block determination processing. The determination unit 106 calculates the RD cost by the following equation (3).

  Here, “Distortion” indicates a residual generated when the encoding information of the macroblock is shared. “Rate” indicates the amount of code generated when the encoding information of the macroblock is shared. λ represents a coupling coefficient. Details of the RD cost are described in the above-mentioned non-patent document and patent document.

  As illustrated in FIG. 4, the determination unit 106 calculates the RD cost for all the reference images stored in the buffer 105 and all the macroblocks in the small area in each reference image using the equation (3). , The macroblock with the lowest RD cost is determined as a shared block.

  In addition, when the shared block is determined, the determination unit 106 generates shared block designation information indicating the positional relationship between the corresponding macro block and the shared block. Here, in the present embodiment, as shared block designation information, view_index indicating the viewpoint of the reference image in which the shared block exists by limiting the sharing of the encoded information only from the same time, and the corresponding block on the small area Blk_index indicating the relative position of the shared block with respect to is used.

  FIG. 5 is a diagram illustrating an example of shared block designation information. In FIG. 5, as an example, the position of the corresponding block is set to “0” in the small area in the reference image of each viewpoint, and the macroblock numbers are assigned in order from the upper left to the lower right in the small area. In such a case, as shown in FIG. 5, assuming that the RD cost of the macroblock whose relative position from the corresponding block is “3” in the small area of the reference image of viewpoint 0 is the smallest, The shared block designation information is generated as “view = index = 0, blk_index = 3”.

  Returning to FIG. 1, the acquisition unit 107 inputs encoded data S (v ′) of another viewpoint including the shared block determined by the determination unit 106 (that is, the shared block indicated by the shared block designation information in the buffer 105). Then, the encoding information when the shared block is encoded is acquired from the encoded data S (v ′) of the other viewpoint or the shared block. In the present embodiment, the acquisition unit 107 acquires, for example, an encoding mode (PredMode), a motion vector (Mv), an index (Ref_index) specifying a reference image, and the like as encoding information.

  The generation unit 108 generates a prediction image based on the encoded information acquired by the acquisition unit 107. The generation unit 108 outputs the generated predicted image. That is, in the encoding of the encoding target macroblock, the encoding information of the shared block in the parallax image R (v ′) of another viewpoint that has been decoded is shared and used.

  FIG. 6 is a diagram illustrating an example of encoding information sharing. As illustrated in FIG. 6, the generation unit 108 generates a predicted image from the encoding target macroblock, and encodes the shared block (macroblock) determined for the encoding target macroblock. As coding information, coding mode (PredMode): L0 prediction, motion vector (Mv): (10, 20), and index (ref_index) = 1 for designating a reference image are used.

  Here, although the encoding information used in the encoding target macroblock and the shared block is common, the reference image in the buffer 105 when encoding each macroblock is different, so the encoding target macroblock The reference image referred to by and the reference image referenced by the shared block are different.

  The variable length encoding unit 118 performs variable length encoding on the residual information output from the transform / quantization unit 115 to generate encoded data S (v) 104. In addition, the variable length coding unit 118 performs variable length coding processing on the shared block designation information output from the determination unit 106, and adds the encoded shared block designation information to the encoded data. That is, the variable length encoding unit 118 generates encoded data S (v) 104 including the encoded residual information and the encoded shared block designation information. Then, the variable length encoding unit 118 outputs the encoded data S (v) 104. The encoded data S (v) 104 is input to the image decoding apparatus via a network or a storage medium.

  Next, an image encoding process performed by the image encoding apparatus 100 according to the present embodiment configured as described above will be described. FIG. 7 is a flowchart showing the procedure of the image encoding process.

  The encoding unit 117 inputs the input image I (v) (step S101). Here, the inputted input image I (v) is divided into macroblocks of a predetermined size. On the other hand, the encoding unit 117 inputs a decoded parallax image R (v ′) of one or a plurality of other viewpoints different from the viewpoint of the input image I (v), and stores the decoded parallax image R (v ′) as a reference image in the buffer 105 (Step S1). S102).

  The prediction unit 112 receives the decoded depth information D (v) of the input image I (v), receives the decoded parallax image R (v ′) of another viewpoint and the depth information D (v). A prediction image is generated using the prediction image (step S103).

  FIG. 8 is a flowchart showing the procedure of the prediction process. First, the determination unit 106 selects a reference image from the buffer 105 (step S301). Then, the determination unit 106 determines a corresponding macroblock corresponding to the encoding target macroblock in the reference image by projective transformation using the above-described equations (1) and (2) (step S302).

  The determination unit 106 determines a small region including a corresponding macroblock and a neighboring macroblock centered on the corresponding macroblock in the reference image (step S303). Then, the determination unit 106 selects one macro block from among a plurality of macro blocks constituting the small area (step S304). Then, the determination unit 106 calculates the RD cost of the selected macroblock using the above-described equation (3) (step S305). The determination unit 106 temporarily stores the calculated RD cost in a memory (not shown) or the like.

  The determination unit 106 determines whether the RD cost has been calculated for all macroblocks in the small area (step S306). If the determining unit 106 has not yet calculated the RD cost for all macroblocks in the small area (step S306: No), the determining unit 106 is the small area and has not yet calculated the RD cost. A macroblock is selected (step S307), the process returns to step S305, and the RD cost of the selected macroblock is calculated (step S305). Thereby, the RD cost is calculated for all macroblocks in the small area.

  In step S306, when the determination unit 106 calculates the RD cost for all the macroblocks in the small region (step S306: Yes), the determination unit 106 selects all the reference images (all other images) in the buffer 105. It is determined whether the corresponding macroblock determination process and the RD cost calculation process have been completed for the viewpoint-decoded parallax image R (v ′)) (step S308).

  If the determination unit 106 has not yet performed the corresponding macroblock determination process and the RD cost calculation process for all the reference images (step S308: No), the determination unit 106 The next reference image that has not yet been subjected to the corresponding macroblock determination process and the RD cost calculation process is selected (step S309), and the corresponding macroblock determination process and the RD cost calculation process from step S302 to S307 are repeated. Do. As a result, in all reference images in the buffer 105, RD costs for all macroblocks in a small area centered on the corresponding block are calculated and stored in a memory (not shown).

  When the determination unit 106 completes the corresponding macroblock determination processing and the RD cost calculation processing for all reference images (step S308: Yes), the determination unit 106 selects the RD cost stored in the memory from The macroblock for which the minimum RD cost is calculated is determined as a shared block (step S310).

  The determination unit 106 uses each value of view_index specifying the viewpoint of the reference image in which the determined shared block exists and blk_index specifying the relative position of the shared block to the corresponding block in the small area of the reference image as shared block specification information. Setting is made (step S311). The determination unit 106 outputs the shared block designation information to the acquisition unit 107 and the variable length encoding unit 118.

  The acquisition unit 107 acquires the encoding information of the shared block indicated by the shared block designation information from the encoded data S (v ′) of the reference image including the shared block or the shared block (step S312). The generation unit 108 generates a predicted image according to the acquired encoding information (encoding mode, motion vector, reference index of reference image) (step S313). The generation unit 108 outputs the generated predicted image and ends the prediction process.

  Returning to FIG. 7, when the prediction process in step S103 is completed, the subtractor 111 performs a subtraction process between the input image I (v) 101 and the predicted image, and outputs a residual (step S104). The transform / quantization unit 115 orthogonally transforms the residual to obtain a transform coefficient, and quantizes the transform coefficient to obtain residual information (step S105).

  The variable length coding unit 118 performs variable length coding on the residual information and the shared block designation information output from the determination unit 106 of the prediction unit 112 to generate coded data S (v) 104 ( Step S106). The variable length encoding unit 118 outputs the encoded data S (v) 104 (step S107).

  As described above, in the present embodiment, in the decoded parallax image R (v ′) having a viewpoint different from the viewpoint of the input image, RD is selected from the small regions centering on the corresponding block corresponding to the encoding target macroblock. The macro block with the lowest cost is determined as a shared block, and the encoding information of the shared block is used when generating a predicted image from the encoding target macro block. Also, this encoded information is not limited to motion information. That is, in the present embodiment, encoding processing is performed by sharing encoding information that is not limited to motion information between parallax images of different viewpoints, and specification information indicating the position of the determined shared block is encoded. By adding to the data and outputting it, the influence of errors such as projective transformation can be reduced, and the coding efficiency can be improved.

  In the present embodiment, view_index indicating the viewpoint of the reference image including the shared block and blk_index indicating the relative position of the shared block with respect to the corresponding block on the small area are used as the shared block designation information. The information is not limited to these as long as the information can be designated.

  For example, a reference image can be configured by using a number ref_index for identifying the reference image itself as shared block designation information. Also, regarding the relative position from the corresponding block of the shared block, the shared block designation is made in such a format that the relative position in the horizontal direction is ± H (H ≧ 0) block and the relative position in the vertical direction is ± V (V ≧ 0) block. The determination unit 106 and the like may be configured to set the information.

  Also, for example, H. The shared block designation information for designating the shared block is used in a method of predicting from the surrounding blocks of the encoding target macroblock by the method of using the median value of adjacent blocks for the prediction as used in motion vector prediction in H.264. In addition, the determination unit 106 and the like may be configured.

  Further, when the intra prediction information as the encoding information is shared between the encoding target macroblock and the shared block, the generation unit 108 generates a prediction image from the macroblock in the parallax image in which the shared block exists. May be configured. In intra-screen prediction in which prediction is performed from adjacent macroblocks, for example, if the macroblock to be encoded is included in the background and the adjacent macroblock is included in the foreground at the object boundary, the pixel values of the macroblocks are different. Predictions are not likely to be accurate. For this reason, the generation unit 108 may be configured to perform intra prediction using a macroblock adjacent to a shared block in a parallax image of another viewpoint different from the viewpoint of the input image. From the viewpoint where the shared block exists, a hidden surface (background area) macroblock that does not exist in the viewpoint of the input image appears on the screen as a background that is hidden from the viewpoint of the input image. Thus, it is possible to predict a macroblock to be encoded, and it can be expected to improve the prediction efficiency at the object boundary.

(Embodiment 2)
In the second embodiment, the shared block added to the encoded data S (v) 104 when the encoded data S (v) 104 encoded by the image encoding apparatus 100 of the first embodiment is decoded. It is an image decoding apparatus that generates a predicted image by sharing and using encoded information of a shared block indicated by designation information. Here, the encoded data S (v) 104 to be decoded includes codes of residual information and shared block designation information.

  FIG. 9 is a block diagram illustrating a functional configuration of the image decoding apparatus according to the second embodiment. The image decoding apparatus 500 according to the present embodiment includes a control unit 501 and a decoding unit 502, as shown in FIG.

  The decoding unit 502 transmits encoded data S (v) 104 (hereinafter, “decoding target image”) as a decoding target image from the image encoding apparatus 100 according to the first embodiment via a network or a storage medium. The encoded data S (v) 104 is divided into macro blocks, and the divided macro block is different from the viewpoint parallax image in the encoded data S (v) 104. A shared block in the decoded parallax image R (v ′) of another viewpoint is identified from the shared block designation information added to the encoded data S (v) 104, and prediction is performed using the encoded information of the shared block An image is generated and the encoded data S (v) 104 is decoded. The decoding control unit 501 controls the entire decoding unit 502.

  As illustrated in FIG. 9, the decoding unit 502 includes a variable length decoding unit 504, an inverse transform / inverse quantization unit 514, an adder 515, a prediction unit 512, and a buffer 505. Here, the variable length decoding unit 504, the inverse transform / inverse quantization unit 514, and the adder 515 function as a decoding unit.

  The variable length decoding unit 504 receives encoded data S (v) 104 as a decoding target image, performs variable length decoding processing on the input encoded data S (v) 104, and encodes the encoded data S (V) Find residual information (quantized orthogonal transform coefficient information) and shared block designation information included in 104. The variable length decoding unit 504 outputs the decoded residual information to the inverse transform / inverse quantization unit 514, and outputs the decoded shared block designation information to the determination unit 506 of the prediction unit 512.

  Here, the details of the shared block designation information are the same as in the first embodiment, the decoded depth information D (v) of the viewpoint of the encoded data S (v), the camera that captured the decoding target image, and the like. Information indicating the positional relationship between the corresponding block determined based on the positional relationship with the camera that captured the decoded parallax image R (v ′) of the viewpoint and the shared block sharing the encoding information is there.

  The shared block designation information includes a view_index indicating the viewpoint of the reference image including the shared block, and a blk_index indicating the relative position of the shared block with respect to the corresponding block on the small area, as described in the first embodiment. Like, but not limited to.

  The inverse transform / inverse quantization unit 514 performs an inverse quantization process and an inverse orthogonal transform process on the residual information and outputs a residual signal. The adder 515 adds the residual signal and the prediction image generated by the prediction unit 512 to generate a decoded image, and outputs the decoded image as an output image 503.

  The buffer 505 is a storage medium such as a frame memory, and stores a decoded parallax image R (v ′) having a viewpoint different from the viewpoint of the decoding target image as a reference image.

  The prediction unit 512 generates a predicted image using the reference image stored in the buffer 505. As illustrated in FIG. 9, the prediction unit 512 includes a determination unit 506, an acquisition unit 507, and a generation unit 508.

  The determination unit 506 determines a shared block for sharing the encoding information with the decoding target macroblock based on the shared block designation information decoded by the variable length decoding unit 504.

  More specifically, the determination unit 506 determines the shared block as follows. First, the determination unit 506 confirms the content of the shared block designation information received from the variable length decoding unit 504, and decodes the parallax image R () of the other viewpoint specified by the shared block designation information (view_index). v ′) is read from the buffer 505.

  Then, the determination unit 506 determines a corresponding macroblock corresponding to the decoding target macroblock in the decoded parallax image R (v ′) (reference image) of the other viewpoint read from the buffer 505. Here, the corresponding macroblock determination method for the decoding target macroblock is performed using the projective transformation according to the above-described equations (1) and (2), as in the first embodiment. The corresponding block determination process is performed on the reference image specified by the shared block specification information.

  Then, the determination unit 506 determines the macroblock at the relative position (blk_index) from the corresponding macroblock of the shared block specified by the shared block specification information as the shared block.

  The acquisition unit 507 inputs the encoded data S (v ′) of another viewpoint including the shared block determined by the determination unit 506 (that is, the shared block indicated by the shared block designation information in the buffer 505). Encoding information when the shared block is encoded is acquired from the viewpoint encoded data S (v ′) or the shared block. In the present embodiment, as in the first embodiment, the acquisition unit 507 includes, for example, an encoding mode (PredMode), a motion vector (Mv), an index (Ref_index) for specifying a reference image, and the like as encoding information. get.

  The generation unit 508 generates a predicted image based on the encoding information acquired by the acquisition unit 507. The generation unit 508 outputs the generated predicted image. That is, in the encoding of the encoding target macroblock, the encoding information of the shared block in the parallax image R (v ′) of another viewpoint that has been decoded is shared and used. Here, details of the generation of the predicted image by the generation unit 508 are the same as those in the first embodiment.

  Next, the decoding process by the image decoding apparatus 500 according to the present embodiment configured as described above will be described. FIG. 10 is a flowchart showing the procedure of the decoding process according to the second embodiment.

  The variable length decoding unit 504 inputs encoded data S (v) 104 that is a decoding target image from the image encoding device 100 via a network or a storage medium (step S501). Here, the input encoded data S (v) 104 is divided into macroblocks of a predetermined size.

  The variable length decoding unit 504 performs variable length decoding processing on the input encoded data S (v) 104 and extracts residual information and shared block designation information included in the encoded data S (v) 104. (Step S502). The variable length decoding unit 504 sends the shared block designation information to the determination unit 506 of the prediction unit 512 (step S503).

  The decoded residual information is sent to an inverse transform / inverse quantization unit 514, and the inverse transform / inverse quantization unit 514 performs an inverse quantization process and an inverse orthogonal transform process on the residual information to obtain a residual signal. Is output (step S504).

  The decoding unit 502 inputs a decoded parallax image R (v ′) of one or more other viewpoints different from the viewpoint of the decoding target image (encoded data S (v)), and inputs the decoded parallax image R (v ′) to the buffer 505. Save (step S505).

  The prediction unit 512 performs a prediction process for generating a predicted image (step S506). FIG. 11 is a flowchart illustrating a procedure of the prediction process according to the second embodiment.

  The determination unit 506 acquires, from the buffer 505, the decoded parallax image R (v ′) of the other viewpoint specified by the shared block specification information received from the variable length decoding unit 504 (view_index) (Step S701). ). The determination unit 506 determines a corresponding macroblock corresponding to the decoding target macroblock in the decoded parallax image R (v ′) of another viewpoint read from the buffer 505 (step S702). The determination unit 506 determines the macro block at the relative position (blk_index) from the corresponding macro block of the shared block specified by the shared block specifying information as the shared block (step S703).

  The acquiring unit 507 acquires the determined coding information of the shared block from the coded data S (v ′) of the reference image including the shared block or the shared block (Step S704). The generation unit 508 generates a prediction image according to the acquired encoding information (encoding mode, motion vector, reference index of reference image) (step S705). The generation unit 508 outputs the generated predicted image and ends the prediction process.

  Returning to FIG. 10, when the prediction process in step S506 is completed, the adder 515 outputs the residual output from the inverse transform / inverse quantization unit 514 in step S504 and the prediction image generated by the prediction unit 512. Addition generates a decoded image, and outputs the decoded image as an output image signal R (v) 503 (step S507).

  As described above, in the present embodiment, a decoded parallax image R (v ′) having a viewpoint different from the viewpoint of the decoding target image is added to the encoded data S (v) input from the image encoding device 100. The encoding information of the shared block indicated by the shared block designation information is used when generating a predicted image from the decoding target macroblock. Also, this encoded information is not limited to motion information. That is, in this embodiment, by performing decoding processing by sharing encoding information that is not limited to motion information between parallax images of different viewpoints, it is possible to reduce the influence of errors such as projective transformation. The encoding efficiency can be improved.

(Modification)
Various modifications can be employed in the image coding apparatus 100 according to the first embodiment and the image decoding apparatus 500 according to the second embodiment.

(Modification 1)
In the image coding apparatus 100 according to the first embodiment and the image decoding apparatus 500 according to the second embodiment, information specifying a coding mode, a motion vector, and a reference image as coding information shared when generating a predicted image. However, the present invention is not limited to this as long as the information is necessary for decoding the image. For example, as the encoded information to be shared, only the information specifying the encoding mode and the reference image is used, or only the motion vector is used. You may comprise so that it can share as.

(Modification 2)
In addition, residual information can also be used as encoded information in the image encoding device 100. For example, a method of using the residual information in the shared block as it is without encoding the prediction image generated by sharing the encoded information in the image encoding device 100 and the residual information of the encoding target macroblock is employed. Also good. Moreover, you may employ | adopt the system which encodes the difference with the residual information in a shared block. When these methods are adopted, the image encoding device 100 encodes information on whether or not to share residual information, adds the information to the encoded data, and outputs the encoded data to the image decoding device 500. What is necessary is just to comprise. In this case, in the decoding process of the encoded data S (v) in the image decoding apparatus 500 according to the second embodiment, the inverse transformation process and the quantization process by the inverse transformation / quantization unit 514 are unnecessary. There are advantages.

(Modification 3)
Further, the acquisition unit 107 and the generation unit 108 of the image encoding device 100 and the acquisition unit 507 and the generation unit 108 of the image decoding device 500 may be configured to use the pixel value of the shared block as the shared encoding information. Good. For example, since the encoding target macroblock and the shared block are macroblocks obtained by imaging the same point, the pixel values of each macroblock have a high correlation. For this reason, the acquisition units 107 and 507 determine the encoding information as the pixel value of the shared block, and the generation units 108 and 508 copy the pixel values as they are to generate a predicted image, thereby generating the encoding target block. There is an advantage that the amount of code required for encoding can be reduced.

  As described above, when the pixel value itself of the shared block is used as the encoding information to be shared, in the image encoding device 100, the acquisition unit 107 acquires the pixel value of the shared block as the encoding information and generates it. The unit 108 copies the obtained pixel value of the shared block to generate a prediction image, and the variable-length encoding unit 118 stores the encoding information shared between the encoding target block and the shared block as the encoding mode, the motion What is necessary is just to comprise so that the discrimination | determination information which shows whether it is the information for encodings, such as designation | designated of a vector and a reference image, or the pixel value of a shared block, may be encoded and added to encoded data.

  On the other hand, in the image decoding apparatus 500, the discrimination information included in the encoded data S (v) input by the variable length decoding unit 504 is decoded, and in the acquisition unit 507, the encoding information is included in the shared block. In the case of indicating that it is a pixel value, the pixel value of the shared block is acquired as the encoding information, and the generation unit 508 is configured to copy the acquired pixel value of the shared block and generate a predicted image. That's fine.

(Modification 4)
Also, the method for specifying the shared block is not limited to a method using shared block specifying information including information indicating the viewpoint of the reference image, information indicating the relative position of the shared block from the corresponding block, and the like. For example, a method may be adopted in which a processing procedure for determining a shared block is previously determined in common between the image encoding device 100 and the image decoding device 500, and the shared block is specified in accordance with this processing procedure. . As a result, it is possible to reduce the amount of code necessary for designating the shared block.

  For example, as an example of such a processing procedure, based on the tendency of the encoded information in the encoded macroblock near the encoding target macroblock, the correlation of the encoded information from the small area centering on the corresponding block is The determination units 106 and 506 can be configured to search for the highest position and determine the macroblock at that position as a shared block.

  That is, in the image encoding device 100, between the cameras that have captured the decoded depth information D (v) of the input image I (v) and the input image and the decoded parallax image R (v ′) of another viewpoint. The determination unit 106 is configured to determine a shared block from the search range determined based on the positional relationship based on the tendency of the encoding information in the macroblocks in the small area. Further, the determination unit 106 may be configured to generate, as shared block designation information, discrimination information for determining a shared block based on the tendency of encoded information in a macroblock of a small area from within the search range. .

  On the other hand, in the image decoding apparatus 500, when the discrimination information in the shared block designation information indicates that the shared block is determined based on the tendency of the encoded information in the small area block from within the search range, What is necessary is just to comprise the determination part 506 so that a shared block may be determined based on the tendency of encoding information from within the search range.

  The image coding apparatus 100 according to the first embodiment and the modification, and the image decoding apparatus 500 according to the second embodiment and the modification are a control device such as a CPU, a storage device such as a ROM (Read Only Memory) and a RAM, The hardware configuration includes an external storage device such as an HDD and a CD drive device, a display device such as a display device, and an input device such as a keyboard and a mouse.

  Note that the image encoding program executed by the image encoding device 100 according to the first embodiment and the modification example, and the image decoding program executed by the image decoding device 500 according to the second embodiment and the modification example are stored in a ROM or the like. Provided in advance.

  The image encoding program executed by the image encoding device 100 according to the first embodiment and the modified example, and the image decoding program executed by the image decoding device 500 according to the second embodiment and the modified example are in an installable format or An executable file may be recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital Versatile Disk).

  Furthermore, an image encoding program executed by the image encoding device 100 according to the first embodiment and the modification example, an image decoding program executed by the image decoding device 500 according to the second embodiment and the modification example, and the like. You may comprise so that it may provide by storing on the computer connected to the network and downloading via a network. In addition, the image encoding program executed by the image encoding device 100 according to the first embodiment and the modification, and the image decoding program executed by the image decoding device 500 according to the second embodiment and the modification are connected to a network such as the Internet. You may comprise so that it may provide or distribute via.

  The image encoding program executed by the image encoding apparatus 100 according to the first embodiment and the modified example includes the above-described units (subtractor, conversion / quantization unit, variable length encoding unit, inverse conversion / inverse quantization unit, The module configuration includes an adder, a shared block determination unit, an encoded information acquisition unit, and a predicted image generation unit). As actual hardware, a CPU (processor) reads out and executes an image encoding program from the ROM. Thus, the above-described units are loaded on the main memory, and a subtracter, transform / quantization unit, variable length coding unit, inverse transform / inverse quantization unit, adder, shared block determination unit, encoded information acquisition unit The predicted image generation unit is generated on the main storage device. Note that each unit of the image encoding device 100 may be configured by hardware such as a circuit.

  The image decoding program executed by the image decoding apparatus 500 according to the second embodiment and the modified example includes the above-described units (variable length decoding unit, inverse transform / inverse quantization unit, adder, shared block determination unit, code A module configuration including a computerized information acquisition unit and a predicted image generation unit). As actual hardware, a CPU (processor) reads out and executes an image decoding program from the ROM so that each unit is a main storage device. The variable length decoding unit, the inverse transform / inverse quantization unit, the adder, the shared block determination unit, the encoded information acquisition unit, and the prediction image generation unit are generated on the main storage device. Yes. Note that each unit of the image decoding apparatus 500 may be configured by hardware such as a circuit.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 100 Image coding apparatus 101 Input image 102 Decoded parallax image 103 Decoded depth information / disparity information 104,124 Encoded data 105,505 Buffer 106,506 Determination unit 107,507 Acquisition unit 108,508 Generation unit 111 Subtraction Unit 112,512 Prediction unit 113,515 Adder 114,514 Inverse transformation / inverse quantization unit 115 Transformation / quantization unit 116 Control unit 117 Encoding unit 118 Variable length coding unit 500 Image decoding device 501 Control unit 502 Decoding 503 output image 504 variable length decoding unit

Claims (12)

The corresponding block corresponding to the encoding target block in the first parallax image of the viewpoint in the encoding target image in one or a plurality of second parallax images of other viewpoints different from the viewpoint is set to the depth of the first parallax image. The corresponding block in each of the one or the plurality of second parallax images, and the corresponding block determined based on information and a positional relationship between the viewpoint of the first parallax image and the viewpoint of the second parallax image Determining a shared block that shares encoding information from among the blocks of the small area including the neighborhood of, and generating shared block designation information indicating a positional relationship between the corresponding block and the shared block;
An acquisition unit that acquires the encoding information shared by the shared block based on the shared block designation information;
Based on the acquired encoded information, a generating unit that generates a predicted image;
An encoding unit that generates encoded data from the image and the predicted image;
An image encoding device comprising: The encoding unit further encodes the shared block designation information and adds the encoded information to the encoded data, and outputs the encoded data to which the encoded shared block designation information is added.
The image encoding device according to claim 1. The acquisition unit acquires a pixel value of the shared block as the encoding information,
The generation unit generates the prediction image based on the acquired pixel value of the shared block,
The encoding unit further encodes discrimination information indicating that the encoding information is a pixel value of the shared block and adds the encoded discrimination information to the encoded data, and the encoded discrimination information is added. Output encoded data,
The image encoding device according to claim 2. The determining unit determines the shared block based on a tendency of the encoding information in the block of the small region from within a search range determined based on the positional relationship between the depth information and the imaging device, As the shared block designation information, generating a statement to determine the shared block based on the tendency of the encoded information in the small area block from within the search range,
The image encoding device according to claim 2. The corresponding block corresponding to the decoding target block in the first parallax image of the viewpoint in the decoding target image in the second parallax image of the other viewpoint different from the viewpoint, the depth information of the first parallax image, and the Based on the shared block designation information indicating the positional relationship between the shared block that shares the encoding information and the corresponding block, determined based on the positional relationship between the viewpoint of the first parallax image and the viewpoint of the second parallax image A determination unit for determining the shared block;
An acquisition unit that acquires the encoding information shared by the shared block based on the shared block designation information;
Based on the acquired encoded information, a generation unit that generates a predicted image;
A decoding unit that decodes input encoded data and generates an output image from the decoded encoded data and the predicted image;
An image decoding apparatus comprising: The encoded data includes the encoded shared block designation information,
The decoding unit further receives the encoded data from an image encoding device, decodes the shared block designation information included in the encoded data,
The determination unit determines the shared block based on the decoded shared block designation information.
The image decoding device according to claim 5. The encoded data is further encoded, and includes discrimination information indicating whether the encoded information is a pixel value of the shared block,
The decoding unit decodes the discrimination information,
When the determination information indicates that the encoding information is a pixel value of the shared block, the acquisition unit acquires the pixel value of the shared block as the encoding information;
The generation unit generates the prediction image based on the acquired pixel value of the shared block.
The image decoding device according to claim 6. The shared block designation information further includes the coding information of a block in a small area including a block in the vicinity of the corresponding block, within a search range determined based on the depth information and a positional relationship between the imaging devices. Including discriminating information indicating whether to determine the shared block based on a trend,
When the determination information indicates that the shared block is determined based on a tendency of the encoding information in a small area block including a block near the corresponding block from within a search range, Determining the shared block from within a range based on a tendency of the encoded information;
The image decoding device according to claim 6. The corresponding block corresponding to the encoding target block in the first parallax image of the viewpoint in the encoding target image in one or a plurality of second parallax images of other viewpoints different from the viewpoint is set to the depth of the first parallax image. The corresponding block in each of the one or the plurality of second parallax images, and the corresponding block determined based on information and a positional relationship between the viewpoint of the first parallax image and the viewpoint of the second parallax image Determining a shared block that shares encoding information from among the blocks in the small area including the neighborhood of the block, and generating shared block designation information indicating a positional relationship between the corresponding block and the shared block,
Based on the shared block designation information, obtain the encoding information shared by the shared block,
Based on the obtained encoded information, generate a prediction image,
Generating encoded data from the image and the predicted image;
Image coding method. The corresponding block corresponding to the decoding target block in the first parallax image of the viewpoint in the decoding target image in the second parallax image of the other viewpoint different from the viewpoint, the depth information of the first parallax image, and the Based on the shared block designation information indicating the positional relationship between the shared block that shares the encoding information and the corresponding block, determined based on the positional relationship between the viewpoint of the first parallax image and the viewpoint of the second parallax image To determine the shared block,
Based on the shared block designation information, obtain the encoding information shared by the shared block,
Based on the acquired encoded information, generate a prediction image,
Decoding input encoded data and generating an output image from the decoded encoded data and the predicted image;
Image decoding method. Computer
The corresponding block corresponding to the encoding target block in the first parallax image of the viewpoint in the encoding target image in one or a plurality of second parallax images of other viewpoints different from the viewpoint is set to the depth of the first parallax image. The corresponding block in each of the one or the plurality of second parallax images, and the corresponding block determined based on information and a positional relationship between the viewpoint of the first parallax image and the viewpoint of the second parallax image Means for determining a shared block that shares encoding information from among blocks in a small area including the neighborhood of the block, and generating shared block designation information indicating a positional relationship between the corresponding block and the shared block;
Means for acquiring the encoding information shared by the shared block based on the shared block designation information;
Means for generating a predicted image based on the obtained encoded information;
Means for generating encoded data from the image and the predicted image;
Program to make it function. Computer
The corresponding block corresponding to the decoding target block in the first parallax image of the viewpoint in the decoding target image in the second parallax image of the other viewpoint different from the viewpoint, the depth information of the first parallax image, and the Based on the shared block designation information indicating the positional relationship between the shared block that shares the encoding information and the corresponding block, determined based on the positional relationship between the viewpoint of the first parallax image and the viewpoint of the second parallax image Means for determining the shared block;
Means for acquiring the encoding information shared by the shared block based on the shared block designation information;
Means for generating a predicted image based on the obtained encoded information;
Means for decoding input encoded data and generating an output image from the decoded encoded data and the predicted image;
Program to make it function.

Images