JP2014112749A - Image coding device and image decoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a coding device and a decoding device for coding and decoding a distance image.SOLUTION: There is provided a coding device for performing coding by dividing a distance image into a plurality of blocks and performing in-screen prediction on the basis of characteristics of adjacent blocks. There is provided a coding device comprising: selection means for selecting a prediction mode to be applied to each block of a distance image from a plurality of prediction modes; first determination means for determining whether a plurality of depth values are included in adjacent coded blocks; second determination means for determining whether a block determined as including a plurality of depth values by the first determination means has a prediction mode corresponding to a direction toward a block to be coded; prediction means for setting the same value as a prediction value of the prediction mode of the block determined as having the prediction mode by the second determination means as a prediction value of a prediction mode of a block; and coding means for coding the block to be coded using the prediction value of the prediction mode and transmitting the coded block. A decoding device performs decoding on the basis of a coding scheme used in the coding device.

Description

  The present invention relates to an image encoding device and an image decoding device.

  Accurate and efficient recording of the three-dimensional shape of the subject is an important theme, and various methods have been proposed. As one of the methods, a texture image that is a general two-dimensional image that represents the subject space with the color of each subject and the background, and an image that represents the subject space with the distance from the viewpoint to each subject and the background. There is a method of recording in association with two types of image data (hereinafter referred to as “distance image”). A distance image is an image that expresses a distance value (depth value) from a viewpoint to a corresponding point in a subject space for each pixel. This distance image can be acquired, for example, by a distance measuring device such as a depth camera installed in the vicinity of the camera that records the texture image. Alternatively, a distance image can be acquired by analyzing a plurality of texture images obtained by photographing with a multi-viewpoint camera, and many analysis methods have been proposed.

  In addition, distance values are expressed in 256 levels (8-bit luminance values) in the Moving Picture Experts Group (MPEG), which is a working group of the International Organization for Standardization / ISO / IEC, as a standard for distance images. The standard MPEG-C part3 is defined, and the standard distance image is an 8-bit grayscale image. In addition, since it is defined that a higher luminance value is assigned as the distance from the viewpoint is shorter, in a standard distance image, a subject located in front is expressed as white and a subject located in the back is expressed in black. As a feature of the distance image, it can be said that a single pixel value tends to appear in a wider area than the texture image. For example, even if a person wearing a fancy pattern is drawn on the texture image, the distance value of the clothes portion is almost constant in the distance image.

  If a texture image and a distance image representing the same subject space are obtained, the distance from the viewpoint of each pixel constituting the subject image drawn in the texture image is known from the distance image, so that the subject has the maximum depth. It can be restored as a three-dimensional shape expressed in 256 stages. Furthermore, by projecting the 3D shape onto the 2D plane geometrically, the original texture image is converted into a texture image in the subject space when the subject is photographed from another angle within a certain range from the original angle. It is possible to convert. That is, since a 3D shape can be restored when viewed from an arbitrary angle within a certain range by a set of texture images and distance images, a free viewpoint image of 3D shapes can be obtained by using multiple sets of texture images and distance images. Can be expressed with a small amount of data.

  By the way, the video compression standard H.264. As in the case of H.264, a technique for compressing and encoding video by efficiently eliminating temporal or spatial redundancy in the video is known (for example, Non-Patent Document 1). When each video of a texture video (video having a texture image as each frame) and a distance video (video having a distance image as each frame) is encoded by an encoding device using this technology, the redundancy that each video has Can be eliminated, and the data amount of each video transmitted to the decoding device can be further reduced.

  This H. In the H.264 standard, information compression is performed using a method called intra prediction encoding. In-screen predictive encoding is a group of pixels included in an encoded block around an encoding target block when one image to be encoded is divided into square blocks and encoded in, for example, raster scan order. To predict the encoding target block in advance. By performing orthogonal transform on the difference signal obtained by subtracting the prediction block from the encoding target block, the energy of the frequency spectrum after the orthogonal conversion is concentrated in the low-order region compared to the case where the encoding target block is directly orthogonally converted. Therefore, information can be efficiently compressed.

  This intra prediction encoding can be performed on the luminance signal in units of 4 × 4 pixel sub-blocks or 16 × 16 pixel macroblocks. There are nine types of prediction modes for sub-blocks and four types of prediction modes for macroblocks. For color difference signals, there are four types of prediction modes for the 8 × 8 pixel block, the same as in the case of the luminance macroblock.

  19 and 20 are diagrams schematically showing nine types of prediction modes for sub-blocks. For the 4 × 4 pixel encoding target sub-block B1 shown in FIG. 19, prediction is performed using the surrounding pixels A to M. FIG. 20 shows the direction in which these pixels are used. For example, in the case of mode 1, since the copying direction is the horizontal direction from left to right, the pixels I, J, K, and L are directed to the right. A block in which copying is repeated becomes a prediction block. Mode 2 is called a DC mode, which does not create a pixel group by copying it in a specified direction, but creates a prediction block based on the average value of eight pixels A to D and I to L. In modes 3 to 8, as shown in FIG. 20, a prediction block is obtained by repeating copying in the direction of the arrow.

  FIG. 21 and FIG. 22 are diagrams similarly showing four types of prediction modes for macroblocks. The encoding target macroblock is predicted using the surrounding pixels 00 to 0F and 10 to 1F. As shown in FIG. 22, there are only two types of prediction directions, the vertical direction (mode 0) and the horizontal direction (mode 1). In addition, there are the DC mode (mode 2) and the Plane mode (mode 3) described above. . In the Plane mode, a prediction block is obtained by interpolating between pixel groups so that they are smoothly connected. The four types of prediction modes for color difference signals are the prediction modes having the same contents except for the number of surrounding pixel groups.

  When encoding the prediction mode for the sub-block, the prediction mode of the prediction mode of the encoding target block is set as the prediction value of the prediction mode of the encoding target block among the prediction modes of the blocks adjacent to the left and above the encoding target block. In the case of the same prediction mode as the value, the compression rate is further improved by omitting the encoding of the prediction mode number.

“ITU-T Recommendation H.264”, International Telecommunication Union-Telecommunication Standardization Sector, March 2009

  By the way, since the distance image represents the distance to the subject, the group of the same depth value is generally much larger than the group of the same depth value of the texture image. Have. In the distance image, it is rare that the distance depth value changes abruptly in units of pixels other than the contour portion of the subject. That is, the probability that adjacent blocks have the same depth value is very high. From these features, the correlation between blocks over a wide range is high, and in particular, there is a high probability that the same depth value is continuous. Furthermore, since the contour of the subject is continuous as long as it does not overlap with other subjects, the correlation between the prediction directions in the screen is high between the blocks along one contour line. In addition, distance images tend to have a simpler screen structure than texture images, so not only sub-blocks but also the correlation between blocks in large units such as macroblocks can be expected to be very high. .

  However, H.C. When the H.264 standard is applied to a distance video, there is a problem that information compression becomes inefficient in the above-described intra prediction. For the distance image having the above-described features, the above-described intra-screen prediction method includes a mode that is not so effective for the distance image, such as DC prediction and Plane prediction. Waste occurs. This is because, as described above, the distance image has a high probability that the same depth value continues in a block over a wide range, but in the DC prediction and the Plane prediction, an intermediate value of the actual depth value is created. This is because it is not suitable for accurate prediction in a distance image. Furthermore, as described above, although the correlation is high with respect to the prediction direction with the adjacent block, since the bit cannot be omitted unless the correlation is the same, the correlation is not fully utilized. In addition, since there are only four types of macroblocks, and there are only two types other than the DC mode and the Plane mode, the macroblock is not suitable for a simple image such as a distance image. There is a problem.

  The present invention has been made in view of such circumstances, and is supplied from an image encoding device capable of reducing the amount of code of encoded data of a distance image as compared with the conventional image encoding device. An object of the present invention is to provide a decoding device that decodes a distance image from encoded data.

  The present invention is an image encoding device that encodes by dividing a distance image into blocks and performing intra-screen prediction based on features of adjacent blocks, and each block of the distance image is selected from prediction modes. Selection means for selecting a prediction mode to be applied to, a first determination means for determining whether or not adjacent coded blocks include a plurality of depth values, and a plurality of the first determination means, The second determination means for determining whether or not the block determined to include the depth value has a prediction mode corresponding to the direction toward the encoding target block, and the second determination means. A prediction unit that uses a prediction value of the prediction mode of the block that is the same as the prediction mode of the block determined to be present, and the prediction value of the prediction mode. Characterized by comprising an encoding means for.

  According to the present invention, the plurality of adjacent encoded blocks are adjacent to the upper and left blocks, and when the predicted value cannot be obtained from any of the blocks, the blocks adjacent to the upper left and the upper right are It is characterized by doing.

  The present invention is characterized in that the prediction mode includes only prediction modes corresponding to eight directions.

  The present invention is characterized in that, when there are two blocks from which the predicted value is obtained, a prediction mode corresponding to an intermediate direction of each prediction direction is used as the predicted value.

  The present invention is characterized in that, when the selected one mode is encoded, the selected one mode is encoded by encoding a difference of a direction of the predicted value from a prediction direction.

  The present invention is characterized in that the encoding target block is any one of 4 × 4 pixels, 8 × 8 pixels, 16 × 16 pixels, or a combination thereof.

  The present invention is an image decoding device that decodes a distance image encoded by the image encoding device according to any one of claims 1 to 6, wherein a plurality of adjacent images are adjacent to each block of the distance image. A first determination unit that determines whether or not the decoded block includes a plurality of depth values in the block, and a block that is determined to include a plurality of depth values by the first determination unit, A second determination unit that determines whether or not a prediction mode corresponding to a direction toward the block is present; and a prediction mode that is the same as the prediction mode of the block determined to be possessed by the second determination unit. , And a prediction means for making a prediction value of the prediction mode of the block, and a decoding means for decoding the prediction mode of the received encoded block using the prediction value.

  The present invention causes a computer to function as the image encoding apparatus according to any one of claims 1 to 6.

  According to the present invention, a computer is caused to function as the image decoding device according to claim 7.

  The present invention is encoded data of a distance image, and for each block of the image, one mode is selected from prediction modes composed of only a plurality of prediction directions, and a plurality of adjacent encodings are selected. Whether or not a block that has been determined to include a plurality of depth values has a prediction mode corresponding to a direction toward the block. The prediction mode of the block determined to have the same prediction mode as the prediction mode of the block is used, and the prediction mode is encoded using the prediction value.

  According to the present invention, an encoding device capable of reducing the code amount of encoded data of a distance image and a decoding device that decodes a distance image from encoded data supplied from the encoding device are realized. The effect that it can be obtained.

It is a block diagram which shows the structure of one Embodiment of this invention. It is explanatory drawing which shows the pixel group around the block of an encoding target. It is explanatory drawing which shows the pixel group around the block of an encoding target. It is explanatory drawing which shows the pixel group around the block of an encoding target. It is explanatory drawing which shows prediction mode. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which shows an example of the kind of copy format. It is explanatory drawing which extracted and showed one of the arrow groups shown in FIG. It is explanatory drawing which shows the state which performed the copy of a pixel. It is a flowchart which shows the processing operation of the image coding apparatus shown in FIG. It is explanatory drawing which shows the processing operation of the image coding apparatus shown in FIG. It is explanatory drawing which shows an example of a code word. It is explanatory drawing which shows an encoding object block periphery pixel group. It is explanatory drawing which represented typically nine types of prediction modes with respect to a subblock. It is explanatory drawing which shows an encoding object block periphery pixel group. It is explanatory drawing which represented similarly 4 types of prediction modes with respect to a macroblock.

  Hereinafter, an image encoding device and an image decoding device according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. In this figure, reference numeral 1 indicates that a distance image is input, the input distance image is divided into blocks each having a predetermined number of pixels, encoded for each block, and encoded block data is transmitted via a transmission path. An image encoding device. Code 2 receives the encoded block data transmitted from the image encoding device via the transmission path, decodes the received encoded block data, restores the distance image, and outputs the restored distance image. An image decoding device.

  Next, the intra prediction encoding process will be described with reference to FIGS. The peripheral pixel group used for the intra prediction of the luminance signal sub-block is H.264. Similarly to the H.264 standard, as shown in FIG. 2, there are 13 pixel groups A to M. And the surrounding pixel group used for the prediction in a screen of the block of a color difference signal is 25 pixel groups of AY as shown in FIG. Further, the peripheral pixel groups used for the intra prediction of the macro block of the luminance signal are 49 pixel groups of 00 to 2F, 10 to 1F, and 30, as shown in FIG. As shown in FIG. 5, the prediction mode is prediction for eight directions of modes 0 to 7.

  6 to 13 are examples of the types of pixel copying formats. 6 to 13, a 16 × 16 pixel block located at the lower right is an encoding target block, and the other blocks are encoded adjacent blocks. 6 to 13, each grid in each block represents a pixel, and a line with an arrow represents a copy destination of the pixel. For example, in FIG. 6, the encoding target block is created by copying the pixel in the lowermost row of the encoded block adjacent thereto. Specifically, in the encoding target block, all the pixel groups located in the nth column from the left copy the nth pixel from the left in the bottom row of the adjacent block above. The same applies to the other drawings. The meaning of the arrow will be further described. For example, FIG. 14 shows one of the arrows in FIG. In this case, as shown in FIG. 15, the pixel shown in black is copied as the ninth pixel from the left in the bottom row of the adjacent block.

  For sub-blocks, how to copy neighboring pixels in each prediction mode is described in H.264. The same as the H.264 standard. In this way, for both sub-blocks and macro-blocks, the DC and Plane blocks that do not contribute much to the coding efficiency in the distance image are not used, but instead, prediction modes in various directions are prepared. Can be predicted well. As to which prediction mode is selected, the distortion (sum of squares of differences) for all pixels is calculated for each mode, and the minimum one is selected.

  Next, a prediction mode encoding method in the image encoding device 1 shown in FIG. 1 will be described. When encoding the prediction mode, H.264 is used. As in the case of intra-frame prediction of H.264 standard sub-blocks, prediction mode prediction is performed from adjacent blocks. However, the prediction process is H.264. It is different from that of H.264 standard. The prediction processing operation will be described with reference to FIG.

  First, a prediction mode of a block including a contour is referred to among encoded blocks adjacent to the upper and left sides of the encoding target block. This is because the contour is often continuous from the block including the contour, and the direction changes along the contour, so that the amount of change from the adjacent block is often not large. Whether or not the contour is included is determined depending on whether or not the adjacent block includes a plurality of depth values. That is, when the adjacent block includes an outline, it always includes a plurality of depth values. Therefore, the adjacent block consisting of only a single depth value is not used for prediction mode prediction.

  Therefore, first, it is determined whether there is an encoded block including a plurality of depth values among the encoded blocks adjacent to the upper and left sides of the encoding target block (step S1). As a result of the determination, if there is an encoded block including a plurality of depth values, it is determined whether or not the contour of a block that may include the contour in the block extends in the direction of the encoding target block. (Step S2). Specifically, the block adjacent to the left is determined to exist when the prediction mode is any one of mode 1, mode 3, mode 4, mode 5, and mode 7. A block adjacent to the upper side is determined to exist when the prediction mode is any one of mode 0, mode 2, mode 3, mode 4, mode 5, and mode 6. As a result of the determination in steps S1 and S2, whether or not there is an encoded block including a plurality of depth values in the encoded block on the upper left and the upper right of the encoding target block, if neither exists. Determine (step S3). If this also does not exist, the prediction value of the prediction mode is set to “none”, and the prediction mode number shown in FIG. 5 is encoded as it is (step S4).

  On the other hand, if there is an encoded block including a plurality of depth values among the encoded blocks on the upper left and the upper right, the contour of the block that may include the contour in the block is encoded. It is determined whether or not it extends in the direction of the target block (step S5). It is determined that a block adjacent on the upper left is present when the prediction mode is mode 3. It is determined that a block adjacent to the upper right is present when the prediction mode is 2 or mode 6.

  Next, as a result of the determination in steps S2 and S5, if there is a corresponding block, whether or not the determined block is a block of both of the two blocks (upper and left, or upper left and upper right). Is determined (step S6). As a result of this determination, if both are applicable, the middle direction between the two prediction mode numbers is used as a reference (step S7). When the middle is not fixed in one direction, the smaller mode number is adopted. On the other hand, when only one of the blocks is applicable, the direction of the corresponding prediction mode is used as a reference (step S8).

  For example, when the mode 4 shown in FIG. 5 is the reference, the direction is set to 0, the next lower number is set to 1 and the larger number is set to 2, and then the reference direction is used as an axis. Assign a number to the outside. If there is no more number on the outside of either one, the numbers are sequentially assigned to the outside on the opposite side (see FIG. 17). Then, for example, an exponent Golomb codeword is assigned to each number as shown in FIG. In this method, when the prediction direction predicted from the adjacent block is almost the same as the prediction direction of the encoding target block, the codeword length is shortened, so that the efficiency of information compression can be expected. Alternatively, instead of such codeword allocation, a 4-bit fixed-length codeword b0 b1 b2 b3 is prepared, and whether or not the prediction direction predicted from the adjacent block and the prediction direction of the encoding target block are the same for b0 If they are different, the prediction mode number shown in FIG. 5 may be encoded as it is using the 3 bits b1 b2 b3. Alternatively, H. As in the prediction mode of the macro block for the luminance signal and the block for the color difference signal in the H.264 standard, a 3-bit fixed-length codeword b0 b1 b2 is prepared, and the 3 bits are used for all the blocks in FIG. The number of the prediction mode shown may be encoded as it is.

  In addition, when only one of the upper and left adjacent blocks exists, such as when the block included in the uppermost row or the leftmost column is an encoding target, it is naturally not possible to refer to the nonexistent block. In such a case, the above prediction process is performed using only blocks that can be referred to. That is, steps S6 and S7 in FIG. 16 are omitted, and step S8 is substituted for them, and the process is either step S4 or step S8.

  Next, the processing operation of the image decoding device 2 shown in FIG. 1 will be described. In the image decoding device 2, decoding is performed in block units in the order of encoding. At the time of encoding, the prediction value was calculated by referring to the pixel group included in the encoded block. On the decoding side, the prediction value was calculated in the same manner with reference to the pixel group included in the decoded block. calculate. Since the encoded block on the encoding side and the block when the block is decoded on the decoding side are the same, the same predicted value as that on the encoding side is obtained on the decoding side. As described above, when the prediction mode is encoded using the prediction value on the encoding side, the prediction mode can be restored using the prediction value in the image decoding apparatus 2.

  The method described above can be applied to each of the sub-macroblock and macroblock for the luminance signal and the block for the color difference signal. Through the processing operations described above, efficient intra prediction encoding can be performed on the distance image having the above-described features, and more efficient information compression can be performed.

  The program for realizing the functions of the image encoding device and the image decoding device 2 in FIG. 1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. By doing so, the image encoding process and the image decoding process may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  The present invention can be applied to applications where it is indispensable to encode / decode range images.

  DESCRIPTION OF SYMBOLS 1 ... Image coding apparatus, 2 ... Image decoding apparatus

Claims (10)

An image coding apparatus that divides a distance image into blocks and performs coding by performing intra prediction based on features of adjacent blocks,
Selecting means for selecting a prediction mode to be applied to each block of the distance image from among the prediction modes;
First determination means for determining whether or not to include a plurality of depth values in adjacent encoded blocks;
Second determination means for determining whether or not the block determined to include a plurality of depth values by the first determination means has a prediction mode corresponding to the direction toward the encoding target block;
Prediction means that uses the same prediction mode as the prediction mode of the block determined to have by the second determination means as a prediction value of the prediction mode of the block;
An image encoding apparatus comprising: encoding means for encoding and transmitting the encoding target block using a prediction value of the prediction mode.   The plurality of adjacent coded blocks are blocks adjacent to the upper and left sides, and if the predicted value cannot be obtained from any of them, the blocks are adjacent to the upper left and the upper right. The image encoding device according to claim 1.   The image encoding apparatus according to claim 1, wherein the prediction mode includes only prediction modes corresponding to eight directions.   The image according to any one of claims 1 to 3, wherein when there are two blocks from which the prediction value is obtained, a prediction mode corresponding to an intermediate direction of each prediction direction is set as the prediction value. Encoding device.   5. The method according to claim 1, wherein when the selected one mode is encoded, the selected one mode is encoded by encoding a difference between a direction of the prediction value and a prediction direction. The image encoding device according to any one of the above.   6. The image code according to claim 1, wherein the encoding target block is any one of 4 × 4 pixels, 8 × 8 pixels, 16 × 16 pixels, or a combination thereof. Device. An image decoding device that decodes a distance image encoded by the image encoding device according to any one of claims 1 to 6,
First determination means for determining whether or not a plurality of adjacent decoded blocks include a plurality of depth values in each block of the distance image;
Second determination means for determining whether or not the block determined to include a plurality of depth values by the first determination means has a prediction mode corresponding to the direction toward the block;
Prediction means that uses the same prediction mode of the block determined to be possessed by the second determination means as the prediction value of the prediction mode of the block;
An image decoding apparatus comprising: decoding means for decoding a prediction mode of a received encoded block using the prediction value.   An image encoding program for causing a computer to function as the image encoding apparatus according to any one of claims 1 to 6.   An image decoding program that causes a computer to function as the image decoding device according to claim 7.   It is encoded data of a distance image, and for each block of the image, one mode is selected from prediction modes composed of only a plurality of prediction directions, and a plurality of adjacent encoded blocks are selected. Determining whether a block includes a plurality of depth values, determining whether a block determined to include a plurality of depth values has a prediction mode corresponding to a direction toward the block; Encoded data obtained by encoding the prediction mode using a prediction value, which is the same as the prediction mode of the block determined to have the prediction value of the prediction mode of the block.

Images