JP2013157950A - Encoding method, decoding method, encoder, decoder, encoding program and decoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance encoding quality by filtering a depth map when the video for a depth map to be encoded can also be obtained on the decoding side, thereby reducing the encoding noise and restoring the edge, and to reduce the code amount required for encoding the predictive residual by enhancing the prediction efficiency of a depth map.SOLUTION: A degree of similarity calculation unit 1091 calculates the degree of similarity of a reference pixel for a pixel to be filtered, from a corresponding video signal. A reliability distribution estimation unit 1092 estimates the reliability distribution for the pixel value of a pixel to be filtered, from a pair of a pixel value obtained for every reference pixel and the degree of similarity. A pixel value determination unit 1093 outputs a pixel value giving a maximum reliability, as the pixel value of a pixel to be filtered.

Description

  The present invention relates to an encoding method, a decoding method, an encoding device, a decoding device, an encoding program, and a decoding program for encoding or decoding video information and the like.

  A free viewpoint video is a video that allows the user to freely specify the position and orientation (hereinafter referred to as the viewpoint) of the camera in the shooting space. In the free viewpoint video, since the user designates an arbitrary viewpoint, it is impossible to hold the video for all the possibilities. For this reason, the free viewpoint video is composed of a group of information necessary to generate a video of the designated viewpoint. Note that the free viewpoint video may also be referred to as a free viewpoint television, an arbitrary viewpoint video, an arbitrary viewpoint television, or the like.

  A free viewpoint video is expressed using various data formats. As a most general format, there is a method using a video and a depth map (distance image) for each frame of the video (for example, see Non-Patent Document 1). . Here, the depth map is a representation of the depth (distance) from the camera to the subject for each pixel, and represents the three-dimensional position of the subject. Since the depth is proportional to the reciprocal of the parallax between the two cameras, it is sometimes called a disparity map (parallax image). In the field of computer graphics, the depth is information stored in the Z buffer, so it is sometimes called a Z image or a Z map. In addition to the distance from the camera to the subject, a coordinate value with respect to the Z axis of the three-dimensional coordinate system stretched on the expression target space may be used as the depth.

  In general, since the horizontal direction is the X axis and the vertical direction is the Y axis with respect to the captured image, the Z axis coincides with the direction of the camera, but when a common coordinate system is used for a plurality of cameras, etc. In some cases, the Z-axis does not match the camera orientation. Hereinafter, the distance and the Z value are referred to as depth without distinction, and an image representing the depth as a pixel value is referred to as a depth map. However, strictly speaking, it is necessary to set a reference camera pair in the disparity map.

  When expressing the depth as a pixel value, the value corresponding to the physical quantity is directly used as the pixel value, the method using a value obtained by quantizing the value between the minimum value and the maximum value into a certain number, and the difference from the minimum value. There is a method of using a value obtained by quantizing with a step width. When the range to be expressed is limited, the depth can be expressed with higher accuracy by using additional information such as a minimum value. In addition, when quantizing at equal intervals, there are a method of quantizing a physical quantity as it is and a method of quantizing an inverse of a physical quantity. Since the reciprocal of the distance is a value proportional to the parallax, the former is used when the distance needs to be expressed with high accuracy, and the latter is used when the parallax needs to be expressed with high accuracy. Often. In the following description, everything in which depth is expressed as an image is referred to as a depth map regardless of the pixel value conversion method or the quantization method.

  The depth map can be regarded as a grayscale image because each pixel is expressed as an image having one value. In addition, it can be said that the subject has a spatial correlation and a temporal correlation as in the case of the image signal because the subject exists continuously in the real space and cannot move instantaneously to a distant position. Therefore, depending on the image coding method and video coding method used to encode normal image signals and video signals, images composed of depth maps and continuous depth maps can be spatially and temporally redundant. It is possible to efficiently encode while removing. Hereinafter, the depth map and its video are referred to as a depth map without distinction.

  Here, general video coding will be described. Standard H. In recent video coding represented by H.264 / AVC, in order to realize efficient coding using the feature that the subject is spatially and temporally continuous, each frame of the video is macroblocked. Are divided into processing unit blocks, and the video signal is predicted spatially or temporally for each macroblock, and prediction information indicating a prediction method and a prediction residual are encoded. When predicting a video signal spatially, for example, information indicating the direction of spatial prediction becomes prediction information, and when predicting temporally, for example, information indicating a frame to be referenced and information indicating a position in the frame Is prediction information. In normal video coding, in order to dramatically reduce the amount of codes, efficient coding is realized by lossy coding of this prediction residual in consideration of subjective quality (standard H.264). For details of / AVC, see Non-Patent Document 2, for example).

  By performing lossy encoding of the prediction residual, noise is superimposed on the encoded frame. When a video signal is predicted temporally, if a frame on which such encoding noise is superimposed is used as a reference frame, noise is also superimposed on the prediction signal, resulting in a decrease in prediction efficiency. Therefore, the prediction efficiency is improved by applying a filter that reduces noise to the encoded frame. In a new video encoding called HEVC, a deblocking filter and an adaptive in-loop filter are used (for details of HEVC, see Non-Patent Document 3, for example).

  However, since these filters do not distinguish between the differences in the photographed subject, there is a problem that appropriate noise removal cannot be realized in a frame in which a value such as a depth map greatly depends on the subject. In order to solve this problem, when video frames corresponding to the depth map are encoded together, Non-Patent Document 4 uses a similarity for each pixel in the video frame to distinguish an object captured for each pixel. However, by performing the filtering process, appropriate noise removal is realized even in a frame whose value greatly depends on the subject.

Y. Mori, N. Fukusima, T. Fuji, and M. Tanimoto, “View Generation with 3D Warping Using Depth Information for FTV”, In Proceedings of 3DTV-CON2008, pp. 229-232, May 2008. Rec. ITU-T H.264, “Advanced video coding for generic audiovisual services”, March 2009. B. Boss, W.-J. Han, J.-R. Ohm, GJ Sullivan, and T. Wiegand, “WD5: Working Draft 5 of High-Efficiency Video Coding”, JCTVC-G1103_d0, pp.129-150, 7th JCT-VC Meeting, November 2011. S. Liu, P. Lai, D. Tian, and C. W. Chen, “New Depth Coding Techniques With Utilization of Corresponding Video”, IEEE Transactions on Broadcasting, vol.57, no.2, pp.551-561, June 2011.

  However, in the method of Non-Patent Document 4, the similarity to the filter target pixel is calculated for each of the plurality of reference pixels existing around the filter target pixel, and the filter is performed by the weighted sum of the pixel values of the reference pixel based on the similarity. In order to produce a result, the processing frame will be smoothed, and important strong edges cannot be preserved in the depth map. Also, it is impossible to restore strong edges that have been lost due to the encoding process.

  For this problem, the effect of smoothing can be weakened by narrowing down the reference pixels based on the similarity. In particular, by narrowing down the reference pixel to one, an edge depending on the subject can be generated. However, in such a method, when noise is included in a reference pixel having a high degree of similarity, there is a problem that not only the noise cannot be removed but also the noise is diffused.

  Further, since the video frame corresponding to the depth map is also encoded, the similarity calculated therefrom is also affected by the encoding noise. Therefore, it is not possible to narrow down an appropriate reference pixel by narrowing down directly using similarity. In particular, when an erroneous reference pixel is selected when narrowing down to one reference pixel, an edge that did not originally exist may be generated to increase noise.

  The present invention has been made in view of such circumstances, and an encoding method, a decoding method, an encoding device, a decoding device, and an encoding method that can reduce the amount of code by improving the quality of the encoding result and improving the prediction efficiency. An object is to provide an encryption program and a decryption program.

  The present invention is an encoding method for encoding a depth map corresponding to video information, a prediction step for generating prediction information of encoding target frame information of the depth map, the encoding target frame information, and the encoding information A prediction residual encoding step of encoding prediction residual information generated from the prediction information, and outputting the encoded prediction residual information and decoded prediction residual information obtained by decoding the encoded prediction residual information; Temporary decoding information generated from the information indicating the prediction method used in the prediction step, the encoding step for encoding and outputting the encoded prediction residual information, the prediction information, and the decoded prediction residual information. A temporary decoding step for outputting; a reference pixel setting step for setting a reference pixel for each pixel of the encoding target frame information; and the video corresponding to the encoding target frame information. Using the information, the reference pixel is set from the similarity calculation step for calculating the similarity between the reference pixel and the pixel in which the reference pixel is set, the similarity, and the temporary decoding information A reliability distribution estimation step for estimating reliability distribution information with respect to decoding information of a pixel, and a pixel value that gives maximum reliability in the reliability distribution information for each pixel of the encoding target frame information; A decoding step that outputs the prediction information, and the prediction step outputs the prediction information based on the decoding information output by the decoding step.

  In the present invention, the reliability distribution estimation step sets a reliability distribution initialization step for initializing the reliability distribution information for each pixel of the encoding target frame information, and sets a range for updating the reliability distribution information. An updating range setting step, a weighting factor setting step for setting a weighting factor of the reference pixel for each pixel value included in the updating range, and the range to be updated using the similarity and the weighting factor. An update reliability calculation step for calculating an update reliability of the pixel value included in the image, and a reliability information update step for updating the reliability information based on the update reliability.

  The present invention further includes a domain setting step for setting the domain of the reliability from the maximum value and the minimum value of the pixel value of the reference pixel for each pixel of the encoding target frame information, In the reliability distribution estimation step, the reliability distribution information is estimated only for the domain defined for each pixel of the encoding target frame information.

  The present invention is a decoding method for decoding code data in which depth map information corresponding to video information is encoded, and a decoding step for decoding information indicating a prediction method and decoded prediction residual information from the code data; A prediction step of outputting prediction information of decoding target frame information according to the decoded information indicating the prediction method; a temporary decoding step of generating temporary decoding information from the prediction information; and the decoded prediction residual information; A reference pixel setting step for setting a reference pixel for each pixel of the decoding target frame information, and the reference pixel and a pixel in which the reference pixel is set using the video information corresponding to the decoding target frame information The similarity calculation step of calculating the similarity of the image, the similarity, and the temporary decoding information, the decoding information of the pixel in which the reference pixel is set A reliability distribution estimation step for estimating reliability distribution information, and a decoding step for outputting, as the decoding information, a pixel value giving the maximum reliability in the reliability distribution information for each pixel of the decoding target frame information; In the prediction step, the prediction information is output based on the decoded information output in the decoding step.

  In the present invention, the reliability distribution estimation step sets a reliability distribution initialization step for initializing the reliability distribution information for each pixel of the decoding target frame information, and a range for updating the reliability distribution information. An update range setting step, a weighting factor setting step for setting a weighting factor of the reference pixel for each pixel value included in the updating range, and the range to be updated using the similarity and the weighting factor. An update reliability calculation step for calculating an update reliability of the included pixel value and a reliability information update step for updating the reliability information based on the update reliability are included.

  The present invention further includes a domain setting step for setting a domain of the reliability from the maximum value and the minimum value of the pixel value of the reference pixel for each pixel of the decoding target frame information, In the degree distribution estimation step, the reliability degree distribution information is estimated only for the domain defined for each pixel of the decoding target frame information.

  The present invention is an encoding device that performs encoding of a depth map corresponding to video information, a prediction unit that generates prediction information of encoding target frame information of the depth map, the encoding target frame information, and the encoding information Prediction residual encoding means for encoding prediction residual information generated from prediction information, and outputting the encoded prediction residual information and decoded prediction residual information obtained by decoding the encoded prediction residual information; Temporary decoding information generated from information indicating a prediction method used in the prediction means, encoding means for encoding and outputting the encoded prediction residual information, the prediction information, and the decoded prediction residual information Using the temporal decoding means for outputting, reference pixel setting means for setting a reference pixel for each pixel of the encoding target frame information, and the video information corresponding to the encoding target frame information, the reference A degree of similarity with respect to the decoding information of the pixel for which the reference pixel is set from the similarity calculation means for calculating the degree of similarity between the element and the pixel for which the reference pixel is set, the similarity and the temporary decoding information Reliability distribution estimation means for estimating distribution information; and decoding means for outputting, as the decoding information, a pixel value giving the maximum reliability in the reliability distribution information for each pixel of the encoding target frame information. And the prediction means outputs the prediction information based on the decoded information output by the decoding means.

  In the present invention, the reliability distribution estimation means sets reliability distribution initialization means for initializing the reliability distribution information for each pixel of the encoding target frame information, and sets a range for updating the reliability distribution information. Update range setting means, weight coefficient setting means for setting a weight coefficient of the reference pixel for each pixel value included in the update range, and the range to be updated using the similarity and the weight coefficient Update reliability calculation means for calculating the update reliability of the pixel value included in the image data, and reliability information update means for updating the reliability information based on the update reliability.

  The present invention further comprises domain setting means for setting the domain of the reliability from the maximum value and the minimum value of the pixel value of the reference pixel for each pixel of the encoding target frame information, The reliability distribution estimation means estimates the reliability distribution information only for the definition area set for each pixel of the encoding target frame information.

  The present invention is a decoding device for decoding code data in which depth map information corresponding to video information is encoded, the decoding means for decoding information indicating a prediction method and decoded prediction residual information from the code data, Prediction means for outputting prediction information of decoding target frame information according to the decoded information indicating the prediction method; temporary decoding means for generating temporary decoding information from the prediction information; and the decoded prediction residual information; Reference pixel setting means for setting a reference pixel for each pixel of decoding target frame information, the reference pixel, and a pixel in which the reference pixel is set using the video information corresponding to the decoding target frame information The degree of reliability distribution information for the decoding information of the pixel in which the reference pixel is set is deduced from the similarity calculation means for calculating the similarity of the reference pixel, the similarity, and the temporary decoding information. A reliability distribution estimation unit that performs decoding, and a decoding unit that outputs, as the decoding information, a pixel value that gives the maximum reliability in the reliability distribution information for each pixel of the decoding target frame information, and the prediction unit Is characterized in that the prediction information is output based on the decoded information output by the decoding means.

  In the present invention, the reliability distribution estimation means sets reliability distribution initialization means for initializing the reliability distribution information for each pixel of the decoding target frame information, and sets a range for updating the reliability distribution information. Update range setting means, weight coefficient setting means for setting a weight coefficient of the reference pixel for each pixel value included in the range to be updated, and the range to be updated using the similarity and the weight coefficient Update reliability calculation means for calculating the update reliability of the pixel value included, and reliability information update means for updating the reliability information based on the update reliability.

  The present invention further comprises a domain setting means for setting a domain of the reliability from the maximum value and the minimum value of the pixel value of the reference pixel for each pixel of the decoding target frame information, The degree distribution estimation means estimates the degree of reliability distribution information only for the domain defined for each pixel of the decoding target frame information.

  The present invention is an encoding program for causing a computer to function as the encoding device.

  The present invention is a decoding program for causing a computer to function as the decoding device.

  According to the present invention, when data having a value that greatly depends on a subject, such as a depth map, is transmitted together with a corresponding video, noise generated by encoding is reduced and smoothed by encoding. It becomes possible to restore the edge that has been closed. For this reason, it is possible to achieve an effect that it is possible to achieve a reduction in code amount by improving the quality of encoding results and improving prediction efficiency.

It is a block diagram which shows the structure of the depth map encoding apparatus by 1st Embodiment of this invention. It is a block diagram which shows the structure of a decoding frame filter part. It is a process flowchart of the depth map encoding apparatus shown in FIG. It is a flowchart which shows the detail of a filter process. It is a flowchart which shows the detail of the filter process which reduced the computational complexity by restrict | limiting the domain of reliability. It is a flowchart in the case of performing filter processing by setting the update range of the reliability distribution for each reference pixel and estimating the reliability distribution by updating only a part of the reliability. An example of more specific processing when filtering is performed by setting the update range of the reliability distribution for each reference pixel and estimating the reliability distribution by updating only part of the reliability. This is a flowchart. It is a block diagram which shows the structure of the depth map decoding apparatus by 2nd Embodiment of this invention. It is a process flowchart of the depth map decoding apparatus shown in FIG. It is a figure which shows the hardware structural example in the case of comprising a depth map encoding apparatus by a computer and a software program. It is a figure which shows the hardware structural example in the case of comprising a depth map decoding apparatus by a computer and a software program.

  Hereinafter, a depth map encoding apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a depth map encoding apparatus according to the embodiment. The depth map encoding apparatus 100 includes an encoding target depth map input unit 101, a depth map memory 102, a video input unit 103, a video memory 104, a depth prediction unit 105, a prediction residual encoding unit 106, and a variable length encoding unit 107. A bit stream output unit 108, a decoded frame filter unit 109, and a reference frame memory 110.

  The encoding target depth map input unit 101 inputs depth map information to be encoded. In the following description, the depth map information to be encoded is referred to as an encoding target depth map, and a frame to be processed in particular is referred to as an encoding target frame. The depth map memory 102 stores the input encoding target depth map. The video input unit 103 inputs video information corresponding to the encoding target depth map. The depth of the subject shown in each pixel of each frame of the video is represented by the encoding target depth map. In the following description, this video information is referred to as an auxiliary video, and the auxiliary video frame corresponding to the encoding target frame is referred to as an auxiliary video frame. The video memory 104 stores the input video.

  The depth prediction unit 105 generates a prediction signal for the encoding target frame. The prediction residual encoding unit 106 encodes the prediction residual signal of the difference between the encoding target frame (original signal) and the prediction signal, and decodes the prediction residual corresponding to the encoding target prediction residual value and the decoded value thereof. Output a signal. The variable length coding unit 107 performs variable length coding on the information indicating the prediction method and the encoding target prediction residual value to generate a bitstream. The bit stream output unit 108 outputs a bit stream that is an encoding result. The decoded frame filter unit 109 filters the temporary decoded signal obtained by the sum of the prediction signal and the decoded prediction residual signal while using the auxiliary video frame (auxiliary video signal). The decoded signal of the filter result is stored in the reference frame memory 110 and used for video prediction.

  Next, the detailed configuration of the decoded frame filter unit 109 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration of the decoded frame filter unit 109 shown in FIG. The decoded frame filter unit 109 includes a similarity calculation unit 1091, a reliability distribution estimation unit 1092, and a pixel value determination unit 1093.

  The similarity calculation unit 1091 calculates the similarity to the filter target pixel for each reference pixel using the temporary decoded signal and the auxiliary video signal. The reliability distribution estimation unit 1092 estimates a probability distribution as a true value for the pixel value from the similarity of the reference pixel and the pixel value. In the following description, the certainty as the true value is referred to as reliability. The pixel value determination unit 1093 determines the pixel value of the filter result for the filter target pixel based on the similarity distribution, and outputs it as a decoded signal.

  Next, the operation of the depth map encoding apparatus 100 shown in FIG. 1 will be described with reference to FIG. FIG. 3 is a flowchart showing the operation of the depth map encoding apparatus 100 shown in FIG. Here, a process of encoding one frame in the encoding target depth map will be described. By repeating the processing to be described for each frame, it is possible to realize encoding of a depth map composed of a plurality of frames.

  First, the encoding target depth map input unit 101 inputs an encoding target frame and stores it in the depth map memory 102. On the other hand, the video input unit 103 inputs an auxiliary video frame and stores it in the video memory 104 (step S101). Note that although the input encoding target frames are sequentially encoded here, the input order and the encoding order do not necessarily match. When the input order and the encoding order are different, the input depth map and auxiliary video frame are stored in the depth map memory 102 and the video memory 104 until the next frame to be encoded is input. The stored encoding target frame and auxiliary video frame may be deleted from each memory when encoding by the encoding process described below is completed.

  Further, the auxiliary video frame input in step S101 is a frame of auxiliary video that can be obtained on the decoding side, such as that obtained by decoding already encoded auxiliary video. This is to suppress the occurrence of coding noise such as drift by using exactly the same information as the information obtained by the decoding device. However, when the generation of such encoding noise is allowed, the original one before encoding may be input. As an auxiliary video obtained on the other decoding side, an auxiliary video synthesized by decoding an encoded auxiliary video of another viewpoint may be used.

  When the storage of the encoding target frame and the auxiliary video frame is completed, the depth prediction unit 105 generates a prediction signal for the encoding target frame (step S102). Any method may be used to generate the prediction signal. In general encoding, using a spatial correlation or a temporal correlation, a decoded signal of an already encoded portion of a frame to be encoded stored in the reference frame memory 110 or a previously encoded frame. To generate a prediction signal. As typical methods, there are intra prediction using the spatial continuity of the subject, and motion compensation prediction which uses the temporal continuity of the subject depth while compensating for the motion of the subject. Details of these prediction methods are described in, for example, Non-Patent Document 2 and Non-Patent Document 3, and are well known, and thus detailed description thereof is omitted.

  When the prediction signal is generated, the prediction method and the prediction residual are encoded (step S103). The prediction method is information for specifying the method for generating the prediction signal. When intra prediction or motion compensation prediction is available, which prediction is used, what kind of prediction is used when intra prediction is used? Information indicating whether intra prediction has been used or what motion is compensated by using which reference frame in the case of motion compensation prediction is the prediction method. Any method may be used to encode the prediction method. For example, in Non-Patent Document 2 and Non-Patent Document 3, a binary sequence obtained by binarizing a prediction method is encoded by context adaptive entropy encoding. Here, the variable length coding unit 107 performs variable length coding.

  The prediction residual is a difference signal for each pixel between the encoding target frame and the prediction signal. The prediction residual may be encoded in any way. In general coding represented by MPEG and JPEG, first, a prediction residual signal is converted into coefficient information in the frequency domain by using orthogonal transformation or the like.

  Next, each coefficient information is quantized based on the target bit rate and quality. Then, the obtained quantized representative value is encoded using variable length encoding such as entropy encoding. The prediction residual signal is transformed and quantized by the prediction residual coding unit 106, and the variable length coding unit 107 performs variable length coding using the obtained quantized representative value as a coding target prediction residual value.

  The bit stream output unit 108 outputs the bit stream that is the output of the variable length encoding unit 107 as an output result of the depth map encoding apparatus 100. If the encoding order does not match the output order, the bit stream memory is further provided and the output of the variable length encoding unit 107 is temporarily stored, and then the bit stream output unit 108 outputs from the bit stream memory. It doesn't matter.

  When encoding of the prediction method and the prediction residual is completed, the encoded prediction residual is decoded (step S104). The decoding performed here may be performed by decoding the bitstream, or may be performed by decoding information before lossless encoding such as entropy encoding. Here, a decoded prediction residual signal is obtained by performing inverse quantization / inverse transformation on the quantized representative value obtained by transform / quantization in the prediction residual signal encoding unit 106.

  When decoding of the prediction residual is completed, a temporary decoded signal is generated by generating a sum signal of the obtained decoded prediction residual signal and the prediction signal (step S105). Then, the decoded frame filter unit 109 performs filtering on the temporary decoded signal to generate a decoded signal (Step S106), and uses the obtained decoded signal for generating a prediction signal for the subsequent depth map. Store in the memory 110 (step S107). This process (steps S101 to S107) is performed for all frames (step S108).

  Next, the filtering process (step S106) in which the decoded frame filter unit 109 shown in FIG. 1 filters the temporary decoded signal shown in FIG. 3 will be described in detail with reference to FIG. FIG. 4 is a flowchart showing the detailed operation of the filter process. Although the case where the filtering process is performed after the temporary decoded signal for one frame is obtained will be described here, the filtering process may be performed for each part. In that case, a temporary decoded signal cannot be obtained for the filtered region and the region before the filter, but in that case, these regions may not be selected as a reference pixel group to be described later, The corresponding decoded signal stored in the reference frame memory 110 may be used instead, or a temporary decoded signal may be separately stored and used.

  First, a temporarily decoded signal of a processing target frame and an auxiliary video signal corresponding thereto are input (step S1611). Subsequently, one pixel to be filtered is selected and set (step S1612). Then, a pixel group existing within a certain distance from the filter target pixel is set as a reference pixel group (step S1613). Note that the filter target pixel itself may be included in the reference pixel. Further, the distance may be set in any way, but it is necessary to use the same distance as that on the decoding side. However, this is not the case when coding noise such as drift may be generated. For example, the condition may be set according to the Manhattan distance or the Euclidean distance in the image space.

  Next, when the setting of the reference pixel group is completed, the similarity calculation unit 1091 obtains the similarity with the processing target pixel for each reference pixel using the auxiliary video signal (step S1614). Any definition may be used for the degree of similarity, but a definition is used such that the closer the distance to the pixel to be processed and the closer the auxiliary video signal, the greater the degree of similarity. In addition to these conditions, the closer the temporary decoded signal is, the higher the similarity may be set.

For example, the similarity S _p (q) of the reference pixel q with respect to the processing target pixel p is given by Equation (1).

  Here, f, g, and h are non-negative functions that take a maximum value when the arguments match, and I and D are the pixel value of the auxiliary video signal at the pixel position given by the argument and the temporary decoded signal, respectively. Represents the pixel value. For example, a Gaussian function for the norm of the argument difference may be used as f, g, and h.

In addition, since the depth map is information indicating the distance from the camera to the subject, as shown in equations (2) to (4), a three-dimensional coordinate is constructed by combining it with the position of the reference pixel, and the three-dimensional coordinate is obtained. You may use it to define similarity.

  Next, when the calculation of the similarity for each reference pixel is completed, the reliability distribution estimation unit 1092 estimates the reliability distribution for the pixel value using a set of the pixel value and the similarity of the temporarily decoded signal for the reference pixel. (Step S1615). Any reliability distribution may be assumed, and the reliability distribution may be estimated from given information.

  For example, the reliability distribution may be approximated by a polynomial, and the polynomial may be obtained using the least square method. Further, assuming that the reliability distribution is a Gaussian distribution or a mixed Gaussian distribution, maximum likelihood estimation may be performed by an EM algorithm or the like.

  If the reliability distribution can be estimated, the pixel value determination unit 1093 finds a pixel value that gives the maximum reliability from the obtained reliability distribution, and outputs it as a filter result of the pixel to be filtered (step S1616). Until all the pixels are processed, the processing from step S1612 to step S1616 is repeated for each pixel (step S1617).

  For example, the degree of similarity may be used as the reliability as it is. In this case, since pixel value and similarity data are obtained for each reference pixel, a plurality of samples with different reliability may be obtained for one pixel value when estimating the reliability distribution. At this time, either reliability (similarity) may be selected.

In addition, since there is a high possibility that there is a plurality of samples, the reliability may be redefined by the sum. Further, when the similarity is obtained with a value of 0 to 1 like a probability, the value R obtained using the equation (5) may be set as a new reliability.

ここで、ＲＥＦ（ｐ）が画素ｐに対する参照画素群の集合であり、ｗは引数が０の時に最大値となる非負の関数である。 In addition, when redefining the reliability using the sum, not only the similarity to the same pixel value but also the similarity to a close pixel value may be added with a weight. In this case, the reliability is expressed by equation (6).

Here, REF (p) is a set of reference pixel groups for the pixel p, and w is a non-negative function having a maximum value when the argument is zero.

  The reliability distribution may be obtained for the entire range of pixel values. However, since the pixel value space is a discrete space, it is possible to speed up the processing by obtaining it by limiting the definition range. Next, filter processing when the domain is limited will be described. FIG. 5 is a flowchart showing the operation of the filter processing when the domain is limited. In FIG. 5, the same parts as those shown in FIG. 4 are denoted by the same reference numerals, and the description thereof is omitted. The operation shown in FIG. 5 differs from the operation shown in FIG. 4 in that a step of setting a domain (step S1625) is added, and a step of obtaining a reliability distribution (step S1626) obtains a distribution only for the domain. This is the point.

  That is, after calculating the similarity for each reference pixel, the maximum value and the minimum value of the pixel value in the reference pixel group are obtained and set as the reliability definition area (step S1625). The reliability may increase or decrease monotonously with respect to the pixel value. In addition, even if it is not monotonically increasing or decreasing as a whole, it may become an increasing function near the maximum pixel value of the reference pixel or a decreasing function near the minimum pixel value of the reference pixel. is there. For this reason, in the case of limiting using the minimum and maximum pixel values of the reference pixel group as in this method, there is a possibility that a correct reliability distribution cannot be obtained.

  However, in general, the depth map has a high correlation with surrounding pixels and rarely has a value greatly different from the surrounding pixels. On the other hand, when the reliability distribution is an increase function near the maximum value of the reference pixel or a decrease function near the minimum value, the value selected as the filtered value in the subsequent step S1616 has a value greatly different from any reference pixel. It will be. For this reason, a depth map that cannot be generally generated is generated by the filtering process, and noise is newly generated. That is, by using the maximum value and the minimum value as described above, it is possible not only to reduce the calculation amount but also to suppress the generation of noise.

  Note that it is fully possible to slightly increase or decrease the maximum or minimum value of the reference pixel. In order to consider such a case, the definition area obtained with the maximum and minimum values is slightly expanded. You may do it.

  In the estimation of the reliability distribution assuming the model, the accuracy of the obtained reliability distribution largely depends on the probability of the assumption. In addition, when a complicated model is used, there is a problem that the amount of calculation for estimating the distribution increases. Therefore, since the pixel value space is a discrete space, it is possible to consider that the distribution is estimated by obtaining the reliability for each pixel value. In this way, a reliability distribution of an arbitrary shape can be obtained without assuming a complicated model. As a specific method, there is a method of calculating the reliability with respect to all pixel values according to the above-described equation (6).

  In addition, when obtaining the reliability for each pixel value, the similarity obtained for each reference pixel is stored, and instead of repeating the process of recalculating the reliability for each pixel value, some pixel values are obtained for each reference pixel. The reliability distribution may be obtained by repeatedly updating the reliability. This operation will be described with reference to FIG. FIG. 6 is a flowchart showing an operation of obtaining a reliability distribution by repeatedly updating the reliability of some pixel values for each reference pixel. In FIG. 6, the same parts as those shown in FIG. 4 are denoted by the same reference numerals, and the description thereof is omitted.

  The operation shown in FIG. 6 is different from the operation shown in FIG. 4 in that after the reliability distribution is initialized (step S1634), the processing for calculating the similarity (step S1636) and the vicinity of the pixel value of the reference pixel according to the similarity The process of updating the reliability distribution (step S1637) is repeated for each reference pixel (steps S1635 and S1638), thereby generating the reliability distribution. In this processing operation, the pixel value whose reliability is updated is set to the pixel value ± L of the reference pixel to be noted.

  By doing this, in addition to reducing the memory for storing the degree of similarity obtained for each reference pixel, it is only necessary to update the reliability of a limited number of pixel values for each reference pixel. The amount of calculation can be reduced. Compared to updating the reliability for all pixel values, it is considered that the estimation accuracy of the reliability distribution is reduced. However, the further away from the pixel value of the reference pixel, the less influence the reference pixel has on the reliability. Therefore, the accuracy of identifying the pixel value that gives the maximum reliability is not lowered even if the update is terminated within a certain range. That is, this method does not reduce the noise reduction and edge restoration effects of the filter.

Next, the update operation will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of the update operation. First, the reliability distribution R _p initialized to 0 (step S1654), it may be repeated updating of similarity distribution for each reference pixel. That is, if the index of the reference pixel is i and the number of reference pixels is numRefs, after i is initialized to 0 (step S1655), 1 is added to i (step S1661), while i is smaller than numRefs (step S1661). In step S1662, the calculation process (step S1656) of the similarity S _p (q _i ) with respect to the reference pixel qi of the filter target pixel p and the reliability distribution update process (steps S1657 to S1660) are alternately repeated.

In the reliability distribution update process, after initializing the pixel value offset j with the minimum value −L of the update range (step S1657), j is incremented by 1 (step S1659), and j is the maximum value L of the update range. Until it becomes larger (step S1660), the pixel value obtained by adding the pixel value offset to the pixel value of the reference pixel qi, that is, the reliability with respect to D (q _i ) + j, and the magnitude of the offset value in S _p (q _i ) This is performed by repeating the process (step S1658) of adding the value multiplied by the weighting factor w (j) corresponding to. w (j) is a non-negative function that becomes maximum when j = 0, and a Gaussian function or the like can be used.

When the similarity S _p (q _i ) represents the probability that the pixel value of the filter target pixel estimated from the reference pixel q _i is D (q _i ), the reliability distribution in step S1658 is expressed as (7 You may make it update with a formula.

At this time, in order to reduce the number of computations, the dissimilarity distribution R _p ′ is initialized with 1 in step S 1654, and then the dissimilarity distribution is updated with equation (8) in step S 1658, and the minimum dissimilarity is determined in step S 1616. May be set as a decoded signal at the same position as the pixel to be filtered.

  When the similarity of a certain reference pixel to the filter target pixel is high, the true pixel value of the filter target pixel is considered to be very close to the pixel value of the reference pixel. Therefore, a correct reliability distribution is obtained by updating the reliability with a weighting factor that attenuates according to the distance from the pixel value of the reference pixel within the range centered on the pixel value of the reference pixel as described above. It is done. In addition, it is highly possible that the true pixel value of the pixel to be filtered is not significantly different from the pixel value of the reference pixel, and it is considered that the attenuation of the weight coefficient is very fast. On the other hand, when the degree of similarity is low, it can be said that there is a high possibility that it differs from the pixel value of the reference pixel to some extent, and therefore the attenuation of the weighting factor is relatively gradual.

  In order to reflect this, the function w of the weight coefficient may be changed according to the similarity of the reference pixel. That is, when the similarity is large, a weighting function with a fast decay is used, and when the similarity is small, a weighting function with a slow decay is used. When a Gaussian function is used as the weighting function, it corresponds to using a Gaussian function having a smaller variance as the degree of similarity increases.

  Even when a common weight function is used, the influence range may be controlled by changing the update range L according to the similarity of the reference pixels. That is, a smaller value is set in the update range L as the similarity of the reference pixels is larger. Compared with the method of making the weighting function variable according to the similarity of the reference pixels, the accuracy is lowered, but it is possible to have a table of weighting coefficients with a small amount of memory.

  Next, a depth map decoding apparatus according to an embodiment of the present invention will be described. FIG. 8 is a block diagram showing the configuration of the depth map decoding apparatus according to the embodiment. The depth map decoding apparatus 200 includes a bit stream input unit 201, a bit stream memory 202, a video input unit 203, a video memory 204, a variable length decoding unit 205, a depth map prediction unit 206, a prediction residual decoding unit 207, and a decoded frame filter unit 208. A reference frame memory 209 and a depth map output unit 210.

  The bit stream input unit 201 inputs a bit map of a depth map to be decoded. In the following description, the depth map to be decoded is referred to as a decoding target depth map, and a frame to be processed in particular is referred to as a decoding target frame. The bit stream memory 202 stores the input bit stream.

  The video input unit 203 inputs video corresponding to the decoding target depth map. The depth of the subject shown in each pixel of each frame of this video is represented by a decoding target depth map. In the following description, this video is referred to as an auxiliary video, and the auxiliary video frame for the decoding target frame is referred to as an auxiliary video frame. The video memory 204 stores the input auxiliary video.

  The variable length decoding unit 205 decodes information indicating a prediction method that is variable length encoded in the input bitstream and a decoding target prediction residual value. The depth map prediction unit 206 generates a prediction signal for the decoding target frame according to the decoded prediction method. The prediction residual decoding unit 207 decodes and outputs a prediction residual signal of a difference between the decoding target frame and the prediction signal from the decoding target prediction residual value.

  The decoded frame filter unit 208 filters the temporary decoded signal obtained by the sum of the prediction signal and the decoded prediction residual signal while using the auxiliary video frame (auxiliary video signal). The decoded signal of the filter result is stored in the reference frame memory 209, used for video prediction, and output from the depth map output unit 210. Note that the decoded frame filter unit 208 has the same configuration as that shown in FIG.

  Next, the operation of the depth map decoding apparatus 200 shown in FIG. 8 will be described with reference to FIG. FIG. 9 is a flowchart showing the operation of the depth map decoding apparatus 200 shown in FIG. Here, a process of decoding one frame in the decoding target depth map will be described. Depth map decoding composed of a plurality of frames can be realized by repeating the processing described for each frame.

  First, the bitstream input unit 201 inputs a bitstream of a decoding target depth map and stores the bitstream in the bitstream memory 202. On the other hand, the video input unit 203 inputs an auxiliary video frame and stores it in the video memory 204 (step S201). Here, it is assumed that the decoding target frames are sequentially decoded from the input bit stream and output, but the input order and the output order are not necessarily the same. When the input order and the output order are different, the decoded frame is stored in the reference frame memory 209 until the next output frame is decoded, and the depth map output unit 210 performs the depth map from the reference frame memory 209 according to the output order. Is read and output. The stored auxiliary video frame may be deleted from each memory when the corresponding frame is decoded by the decoding process described below.

  In addition, the auxiliary video frame input in step S201 is a frame of the auxiliary video used at the time of encoding, such as that obtained by decoding the encoded auxiliary video. This is to suppress the occurrence of encoding noise such as drift by using exactly the same information as that used in the encoding apparatus. However, when allowing the generation of such encoding noise, a different one from that used at the time of encoding may be input. In addition to those decoded directly, those synthesized using an auxiliary video decoded for another viewpoint may be used.

  When the storage of the code data and the depth map is completed, the variable length decoding unit 205 decodes information indicating a method for generating a prediction signal for the decoding target frame from the bitstream, and based on the information, the variable length decoding unit 205 decodes the decoding target frame. A prediction signal is generated (step S202). Note that any method may be used to generate the prediction signal. In general encoding, a prediction signal is obtained from a part of an already decoded frame of a decoding target frame stored in the reference frame memory 209 or a decoded signal of a previously decoded frame using a spatial correlation or a temporal correlation. Is generated. However, the prediction signal is generated using the same method as the generation method of the prediction signal used at the time of encoding based on the decoded information.

  Next, the prediction residual signal is decoded from the bit stream (step S203). The decoding process uses a method corresponding to the encoding process. Here, the variable-length encoding unit 205 decodes the decoding target prediction residual value that is the quantized representative value in the frequency domain of the prediction residual signal from the bitstream, and predicts the decoded decoding target prediction residual value. Assume that the residual decoding unit 207 decodes the prediction residual signal by inverse quantization and inverse transformation.

  When the prediction signal and the prediction residual signal are obtained, a temporary decoded signal represented by the sum signal is generated (step S204). Then, the decoded frame filter unit 208 generates a decoded signal by performing filter processing on the temporary decoded signal (step S205). The processing performed here is the same as step S106 shown in FIG. 3, and is the same as that described above with reference to FIGS.

  Next, the depth map output unit 210 outputs the obtained decoded signal as an output result of the depth map decoding apparatus 200, and stores it in the reference frame memory 209 for use in generating a prediction signal for the subsequent depth map. (Step S206). This process (steps S201 to S206) is performed for all frames (step S207).

  In the above description, the prediction signal is generated and the filtering process is performed on the entire encoding target frame and the decoding target frame at once. However, the processing target frame is divided and processed for each part. It doesn't matter. Also, encoding / decoding of information indicating prediction signal generation and prediction method, encoding / decoding of prediction residual, etc. are performed for each part, and filtering processing may be performed after a temporary decoded signal is generated for the entire frame. I do not care. In that case, it is necessary to temporarily store the temporarily decoded signal. In addition, the temporary decoded signal may be referred to when generating the prediction signal.

  In the above description, only pixels that are spatially close to each other are set as reference pixels. However, a pixel in a processed frame may be set as a reference pixel in consideration of a distance in the time direction. . In that case, when a motion vector is obtained for the pixel to be processed, the temporal distance may be defined in consideration of the motion vector. When a pixel in a processed frame is used as a reference pixel, it is necessary to store an auxiliary video frame for that frame, and a temporary decoded signal is stored to set a similarity to the reference pixel. Alternatively, it is necessary to use a decoded signal instead of the temporary decoded signal.

  The video encoding and video decoding processes described above can also be realized by a computer and a software program. The program can be provided by being recorded on a computer-readable recording medium or provided via a network. Is possible.

  Next, a hardware configuration when the depth map encoding apparatus is configured by a computer and a software program will be described with reference to FIG. FIG. 10 is a block diagram showing a hardware configuration when the depth map encoding apparatus is configured by a computer and a software program. The system shown in FIG. 10 includes a CPU 50 that executes a program, a memory 51 such as a RAM that stores programs and data accessed by the CPU 50, and an encoding target that receives a depth map signal to be encoded from a camera or the like. Depth map input unit 52 (may be a storage unit that stores a signal of a depth map by a disk device or the like), and a video input unit 53 (video signal by the disk device or the like) that inputs a video for a depth map to be encoded via, for example, a network It may also be a storage unit that stores. Also, the program storage device 54 in which the depth map encoding program 541 which is a software program for causing the CPU 50 to execute the processing operation shown in FIG. 3 is executed, and the CPU 50 executes the depth map encoding program 541 loaded in the memory 51. For example, a bit stream output unit 55 (which may be a storage unit that stores a bit stream by a disk device or the like) that outputs the bit stream generated through the network is connected via a bus.

  In addition, although illustration is omitted, other hardware such as a reference frame storage unit is provided and used to implement this method. In addition, a temporary decoded signal storage unit, a bit stream storage unit, or the like may be used.

  Next, a hardware configuration when the depth map decoding apparatus is configured by a computer and a software program will be described with reference to FIG. FIG. 11 is a block diagram showing a hardware configuration when the depth map decoding apparatus is configured by a computer and a software program. The system shown in FIG. 11 includes a CPU 60 that executes a program, a memory 61 such as a RAM that stores programs and data accessed by the CPU 60, and a bit stream that is input by the depth map encoding apparatus according to the present technique. A stream input unit 62 (may be a storage unit that stores a bit stream by a disk device or the like), and a video input unit 63 (stores a video signal by the disk device or the like) that inputs video for a depth map to be decoded, for example, via a network. It may be a storage unit). Further, FIG. 9 shows a program storage device 64 that stores a depth map decoding program 641 that is a software program that causes the CPU 60 to execute processing operations, and the CPU 60 executes a depth map decoding program 641 that is loaded into the memory 61. A decoding depth map output unit 65 that outputs a decoding depth map obtained by decoding the bitstream to a playback device or the like is connected by a bus.

  In addition, although illustration is omitted, other hardware such as a reference frame storage unit is provided and used to implement this method. Further, a temporary decoded signal storage unit, a bit stream storage unit, or the like may be used.

  In addition, it is obvious that the present embodiment is not limited to only encoding and decoding of a depth map. That is, it is obvious that the present invention can be applied to encoding and decoding of data (for example, temperature information) that greatly depends on a subject when encoding and decoding together with corresponding video information.

  As described above, in the encoding of free viewpoint video data having video and depth maps as components, the distribution of the probability as the true value for the pixel value is estimated from the similarity calculated for each reference pixel, A new video coding technology that improves the prediction efficiency of inter-frame prediction by performing filtering based on the estimated distribution to achieve both suppression of coding noise generated in the depth map and restoration of the edge depending on the subject. Can be provided.

  1 and 8 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed to execute the depth map. An encoding process and a depth map decoding process may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Accordingly, additions, omissions, substitutions, and other changes of the components may be made without departing from the technical idea and scope of the present invention.

  The present invention can be applied to applications where it is essential to encode and decode data (for example, temperature information) that greatly depends on a subject when encoding and decoding together with corresponding video information.

  DESCRIPTION OF SYMBOLS 100 ... Depth map encoding apparatus, 101 ... Decoding target depth map input part, 102 ... Depth map memory, 103 ... Video input part, 104 ... Video memory, 105 ... Depth Prediction unit 106 ... Prediction residual encoding unit 107 ... Variable length encoding unit 108 ... Bit stream output unit 109 ... Decoded frame filter unit 110 ... Reference frame memory DESCRIPTION OF SYMBOLS 200 ... Depth map decoding apparatus, 201 ... Bit stream input part, 202 ... Bit stream memory, 203 ... Video input part, 204 ... Video memory, 205 ... Variable length decoding part, 206 ... Depth map prediction unit, 207 ... Prediction residual decoding unit, 208 ... Decoded frame filter unit, 209 ... Reference frame memory, 210 ... Depth SMAP output unit, 1091 ... similarity calculation unit, 1092 ... reliability distribution estimating unit, 1093 ... pixel value determining unit

Claims (14)

An encoding method for encoding a depth map corresponding to video information,
A prediction step of generating prediction information of encoding target frame information of the depth map;
A prediction that encodes prediction residual information generated from the encoding target frame information and the prediction information, and outputs the encoded prediction residual information and decoded prediction residual information obtained by decoding the encoded prediction residual information A residual encoding step;
An encoding step for encoding and outputting information indicating a prediction method used in the prediction step and the encoded prediction residual information;
A temporary decoding step of outputting temporary decoding information generated from the prediction information and the decoded prediction residual information;
A reference pixel setting step of setting a reference pixel for each pixel of the encoding target frame information;
A similarity calculation step of calculating a similarity between the reference pixel and a pixel in which the reference pixel is set, using the video information corresponding to the encoding target frame information;
A reliability distribution estimation step of estimating reliability distribution information for the decoding information of the pixel in which the reference pixel is set, from the similarity and the temporary decoding information;
A decoding step of outputting, as the decoding information, a pixel value that gives the maximum reliability in the reliability distribution information for each pixel of the encoding target frame information, and
In the prediction step, the prediction information is output based on the decoding information output in the decoding step. The reliability distribution estimation step includes:
A reliability distribution initialization step of initializing the reliability distribution information for each pixel of the encoding target frame information;
An update range setting step for setting a range for updating the reliability distribution information;
A weighting factor setting step for setting a weighting factor of the reference pixel for each pixel value included in the range to be updated;
An update reliability calculation step of calculating an update reliability of a pixel value included in the range to be updated using the similarity and the weighting factor;
The encoding method according to claim 1, further comprising: a reliability information update step of updating the reliability information based on the update reliability. A definition area setting step of setting a definition area of the reliability from the maximum value and the minimum value of the pixel value of the reference pixel for each pixel of the encoding target frame information;
3. The reliability distribution estimation step estimates the reliability distribution information only for the domain defined for each pixel of the encoding target frame information. 4. The encoding method described. A decoding method for decoding code data in which depth map information corresponding to video information is encoded,
A decoding step of decoding information indicating a prediction method and decoded prediction residual information from the code data;
A prediction step of outputting prediction information of decoding target frame information according to the decoded information indicating the prediction method;
A temporary decoding step of generating temporary decoding information from the prediction information and the decoded prediction residual information;
A reference pixel setting step of setting a reference pixel for each pixel of the decoding target frame information;
A similarity calculation step of calculating a similarity between the reference pixel and a pixel in which the reference pixel is set, using the video information corresponding to the decoding target frame information;
A reliability distribution estimation step of estimating reliability distribution information for the decoding information of the pixel in which the reference pixel is set, from the similarity and the temporary decoding information;
A decoding step of outputting, as the decoding information, a pixel value giving the maximum reliability in the reliability distribution information for each pixel of the decoding target frame information;
In the prediction step, the prediction information is output based on the decoding information output in the decoding step. The reliability distribution estimation step includes:
A reliability distribution initialization step of initializing the reliability distribution information for each pixel of the decoding target frame information;
An update range setting step for setting a range for updating the reliability distribution information;
A weighting factor setting step for setting a weighting factor of the reference pixel for each pixel value included in the range to be updated;
An update reliability calculation step for calculating an update reliability of a pixel value included in the update range using the similarity and the weighting factor, and a reliability for updating the reliability information based on the update reliability The decoding method according to claim 4, further comprising: an information updating step. A domain setting step for setting a domain of the reliability from the maximum value and the minimum value of the pixel value of the reference pixel for each pixel of the decoding target frame information;
The reliability distribution estimation step estimates the reliability distribution information only for the domain defined for each pixel of the decoding target frame information. Decryption method. An encoding device that encodes a depth map corresponding to video information,
Prediction means for generating prediction information of encoding target frame information of the depth map;
A prediction that encodes prediction residual information generated from the encoding target frame information and the prediction information, and outputs the encoded prediction residual information and decoded prediction residual information obtained by decoding the encoded prediction residual information Residual encoding means;
Encoding means for encoding and outputting information indicating a prediction method used in the prediction means and the encoded prediction residual information;
Temporary decoding means for outputting temporary decoding information generated from the prediction information and the decoded prediction residual information;
Reference pixel setting means for setting a reference pixel for each pixel of the encoding target frame information;
Similarity calculation means for calculating a similarity between the reference pixel and a pixel in which the reference pixel is set, using the video information corresponding to the encoding target frame information;
Reliability distribution estimation means for estimating reliability distribution information for the decoding information of the pixel in which the reference pixel is set from the similarity and the temporary decoding information;
Decoding means for outputting, as the decoding information, a pixel value giving the maximum reliability in the reliability distribution information for each pixel of the encoding target frame information,
The encoding device, wherein the prediction means outputs the prediction information based on the decoded information output by the decoding means. The reliability distribution estimation means includes
Reliability distribution initialization means for initializing the reliability distribution information for each pixel of the encoding target frame information;
Update range setting means for setting a range for updating the reliability distribution information;
Weight coefficient setting means for setting a weight coefficient of the reference pixel for each pixel value included in the range to be updated;
Update reliability calculation means for calculating update reliability of pixel values included in the range to be updated using the similarity and the weighting factor;
The encoding apparatus according to claim 7, further comprising: reliability information updating means for updating the reliability information based on the update reliability. A definition area setting unit that sets a definition area of the reliability from the maximum value and the minimum value of the pixel value of the reference pixel for each pixel of the encoding target frame information,
9. The reliability distribution estimation unit estimates the reliability distribution information only for the domain defined for each pixel of the encoding target frame information. The encoding device described. A decoding device that decodes code data in which depth map information corresponding to video information is encoded,
Decoding means for decoding information indicating a prediction method and decoded prediction residual information from the code data;
Prediction means for outputting prediction information of decoding target frame information according to the decoded information indicating the prediction method;
Temporary decoding means for generating temporary decoding information from the prediction information and the decoded prediction residual information;
Reference pixel setting means for setting a reference pixel for each pixel of the decoding target frame information;
Similarity calculation means for calculating a similarity between the reference pixel and a pixel in which the reference pixel is set, using the video information corresponding to the decoding target frame information;
Reliability distribution estimation means for estimating reliability distribution information for the decoding information of the pixel in which the reference pixel is set from the similarity and the temporary decoding information;
Decoding means for outputting, as the decoding information, a pixel value giving the maximum reliability in the reliability distribution information for each pixel of the decoding target frame information;
The decoding device, wherein the prediction means outputs the prediction information based on the decoding information output by the decoding means. The reliability distribution estimation means includes
Reliability distribution initialization means for initializing the reliability distribution information for each pixel of the decoding target frame information;
Update range setting means for setting a range for updating the reliability distribution information;
Weight coefficient setting means for setting a weight coefficient of the reference pixel for each pixel value included in the range to be updated;
Update reliability calculation means for calculating update reliability of pixel values included in the range to be updated using the similarity and the weighting factor, and reliability for updating the reliability information based on the update reliability The decoding apparatus according to claim 10, further comprising: information updating means. A definition area setting unit that sets a definition area of the reliability from the maximum value and the minimum value of the pixel value of the reference pixel for each pixel of the decoding target frame information,
12. The reliability distribution estimation means estimates the reliability distribution information only for the domain set for each pixel of the decoding target frame information. Decoding device.   An encoding program for causing a computer to function as the encoding device according to any one of claims 7 to 9.   A decoding program for causing a computer to function as the decoding device according to any one of claims 10 to 12.

Images