JP2009159466A - Method, apparatus and program for decoding multi-viewpoint image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate an encoded bit stream by performing encoding while omitting redundant viewpoint dependent information on an encoding side, and to derive the encoding-omitted viewpoint dependent information when decoding the encoded bit stream. <P>SOLUTION: Since a syntax element whose index (i) is "0" is not encoded for an encoded bit stream to be supplied, in a reference viewpoint number information decoding section 404, the syntax element the index (i) of which is "0" is not decoded, and syntax elements the index (i) of which is "1" or more are decoded. A reference viewpoint information decoding section 406 decodes a syntax element indicating a viewpoint ID of a viewpoint to be used as reference of inter-viewpoint prediction. In such a case, however, a syntax element the (i) of which is "0" or "1" is not decoded in the reference viewpoint information decoding section 406, and syntax elements the (i) of which is "2" or more are decoded. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a multi-view image decoding method, a multi-view image decoding apparatus, and a multi-view image decoding program, and more particularly to decoding multi-view image encoded data obtained by encoding multi-view images taken from different viewpoints. The present invention relates to a viewpoint image decoding method, a multi-view image decoding apparatus, and a multi-view image decoding program.

<Video coding system>
Currently, moving images on the time axis are handled as digital signal information. At that time, motion compensated prediction is used using redundancy in the time direction for the purpose of broadcasting, transmitting or storing information with high efficiency. Devices and systems that are compliant with a coding scheme such as MPEG (Moving Picture Experts Group) that performs coding compression using orthogonal transform such as discrete cosine transform using redundancy in the spatial direction have become widespread.

  The MPEG-2 video (ISO / IEC 13818-2) encoding system established in 1995 is defined as a general-purpose moving image compression encoding system, and supports interlaced scanned images in addition to progressive scanned images. Supports not only SDTV (standard definition images) but also HDTV (high definition images), and storage of DVDs (Digital Versatile Disks), which are optical discs, and magnetic tapes using D-VHS (registered trademark) digital VTRs. It is widely used as an application for media and digital broadcasting.

  In addition, MPEG-4 visual (ISO / IEC 14496-2) encoding method was standardized, aiming at higher encoding efficiency in applications such as network transmission and portable terminals, and was established as an international standard in 1998. It was done.

  In addition, the Joint Technical Committee (ISO / IEC) of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) and the International Telecommunication Union Telecommunication Standardization Department (ITU-T) jointly jointly developed JVT (Joint Video Team). In 2003, the MPEG-4 AVC / H.264 encoding method (14496-10 for ISO / IEC and H.264 for ITU-T was assigned in collaboration. This is called the AVC / H.264 encoding method). This AVC / H.264 encoding method achieves higher encoding efficiency than conventional encoding methods such as MPEG-2 video and MPEG-4 visual.

  In a P picture (forward prediction encoded image) of an encoding method such as MPEG-2 video or MPEG-4 visual, motion compensation prediction is performed only from the immediately preceding I picture or P picture in the display order. On the other hand, in the AVC / H.264 encoding method, a plurality of pictures can be used as P pictures and B pictures as reference pictures, and an optimal one is selected for each block to perform motion compensation. be able to. Further, in addition to the preceding picture in the display order, a subsequent picture can be referred to in the already encoded display order. In addition, a B picture of an encoding method such as MPEG-2 video or MPEG-4 visual refers to one reference picture in the display order, one reference picture in the rear, or two reference pictures at the same time. The average value of the two pictures is a predicted picture, and the difference data between the target picture and the predicted picture is encoded.

  On the other hand, in the AVC / H.264 coding system, a B picture can be referred to for prediction regardless of the forward or backward, regardless of the forward or backward, without being restricted by the restriction of one forward and one backward in the display order. It was. Furthermore, it is possible to refer to the B picture as a reference picture. In inter prediction (motion compensation prediction) in the temporal direction of P pictures and B pictures, a reference picture list is defined to designate which reference picture is actually referred from a plurality of reference picture candidates. The reference picture is registered in the reference picture list, and its specification is specified by an index. This index is called the reference index. Further, the reference picture list defines the reference picture list 0 and the reference picture list 1, and the P slice can perform inter prediction with reference to only the reference pictures registered in the reference picture list 0. The B slice can perform inter prediction with reference to reference pictures registered in both the reference picture list 0 and the reference picture list 1. The prediction that refers to the reference picture registered in the reference picture list 0 is distinguished by calling the list 0 prediction and the prediction that refers to the reference picture registered in the reference picture list 1 as list 1 prediction.

  Furthermore, the encoding mode for each picture (VOP) has been determined using a picture in MPEG-2 video and a video object plane (VOP) in MPEG-4 as one unit, but AVC / H.264 encoding is used. In the system, a slice is used as an encoding unit, and it is also possible to have a configuration in which different slices such as an I slice, a P slice, and a B slice are mixed in one picture.

  Further, in the AVC / H.264 encoding method, a VCL (Video Coding Layer) that performs encoding / decoding processing of video pixel signals (encoding mode, motion vector, DCT coefficient, etc.), NAL ( Network Abstraction Layer) is defined.

  An encoded bit string encoded by the AVC / H.264 encoding method is configured in units of NAL units that are a delimiter of NAL. The NAL unit includes a VCL NAL unit including data (encoding mode, motion vector, DCT coefficient, etc.) encoded by VCL, and a non-VCL NAL unit not including data generated by VCL. The non-VCL NAL unit includes an SPS (sequence parameter set) that includes parameter information related to coding of the entire sequence, and a PPS (picture parameter parameter) that includes parameter information related to picture coding. Set), SEI (supplementary additional information) and the like which are not essential for decoding data encoded by VCL.

  A header (head part) of each NAL unit always has a flag (forbidden_zero_bit) having a value of “0”, an SPS or PPS, or an identifier (nal_ref_idc) for identifying whether or not a slice serving as a reference picture is included. An identifier (nal_unit_type) for identifying the type of NAL unit is included. The nal_unit_type is defined to have any value from “1” to “5” in the case of the VCL NAL unit. For the non-VCL NAL unit, for example, the SEI is “6” and the SPS is “7”. , PPS is defined to have a value of “8”. On the decoding side, the type of the NAL unit can be identified by “nal_unit_type” which is an identifier for identifying the type of the NAL unit included in the header part of the NAL unit.

  The basic unit of encoding in the AVC / H.264 encoding method is a slice obtained by dividing a picture, and the VCL NAL unit is a slice unit. Therefore, a unit called an access unit in which several NAL units are combined is defined, and one encoded picture is included in one access unit.

<Multi-view image coding method>
On the other hand, in a twin-lens stereoscopic television, a left-eye image and a right-eye image captured from two different directions by two cameras are generated and displayed on the same screen to show a stereoscopic image. I have to. In this case, the left eye image and the right eye image are separately transmitted or recorded as independent images. However, this requires about twice as much information as a single two-dimensional image.

  Therefore, a method has been proposed in which one of the left and right images is used as a main image, and the other image (sub-image) information is information-compressed by a general compression encoding method to suppress the amount of information (for example, Patent Document 1). reference). In the stereoscopic television image transmission method described in Patent Document 1, a relative position with high correlation in the other image is obtained for each small region, and the position shift amount (parallax vector) and a difference signal (prediction residual signal) are obtained. ). The difference signal is also transmitted and recorded because the image close to the sub-image can be restored using the main image and the amount of disparity and position shift, which is parallax information, but there is no main image such as the shadow of the object. This is because the sub-image information cannot be restored.

  In 1996, a stereo image encoding method called a multi-view profile was added to the MPEG-2 video (ISO / IEC 13818-2) encoding method, which is an international standard for single-view image encoding (ISO / IEC). IEC 13818-2 / AMD3). The MPEG-2 video multi-view profile is a two-layer encoding method that encodes the image for the left eye with the base layer and the image for the right eye with the enhancement layer, and motion compensated prediction using redundancy in the time direction In addition to discrete cosine transformation using redundancy in the spatial direction, encoding compression is performed using disparity compensation prediction using redundancy between viewpoints.

  In addition, a technique has been proposed for reducing the amount of information using motion compensation prediction and parallax compensation prediction for multi-viewpoint images captured by three or more cameras (see, for example, Patent Document 2). The high-efficiency image coding method described in Patent Document 2 performs pattern matching with reference pictures of a plurality of viewpoints, and selects a motion compensation / disparity compensation predicted image that minimizes an error, thereby improving coding efficiency. It is improving.

  In JVT, the standardization work of multi-view video coding (MVC: Multiview Video Coding (hereinafter referred to as MVC method)) in which the AVC / H.264 coding method is extended to a multi-view image is progressing. JD4.0 (Joint Draft 4.0) is issued as the latest version (see Non-Patent Document 1, for example). Similar to the MPEG-2 video multi-view profile described above, this MVC method also improves encoding efficiency by incorporating prediction between viewpoints.

  Here, with respect to the reference dependency relationship between the viewpoints when the images of the respective viewpoints of the multi-viewpoint image are encoded by the MVC method, and the encoded encoded bit string is decoded, and between the encoding target images constituting the viewpoint image The case of 8 viewpoints will be described as an example.

  FIG. 10 is a diagram illustrating an example of the reference dependency relationship between images when a multi-view image including eight viewpoints is encoded. The vertical axis indicates the direction of the dimension of the space at the viewpoint (in this specification, the viewpoint The horizontal axis indicates the time dimension direction in the shooting (display) order (in this specification, the time dimension direction is the time direction). Yes. P (v, t) (viewpoint v = 0, 1, 2,...; Time t = 0, 1, 2,...) Is an image of the viewpoint v at time t.

  Also, the image indicated by the end point of the arrow is an image to be encoded / decoded. When encoding / decoding the image to be encoded / decoded, temporal inter prediction (motion compensation prediction) or inter-view prediction (disparity compensation prediction) is performed. The reference picture referred to in () is an image indicated by the start point of the arrow. Further, when encoding / decoding an image to be encoded / decoded, a reference picture referred to by temporal inter prediction is an image pointed by the start point of a horizontal arrow, and a reference picture referred to by inter-view prediction is a vertical direction It is an image pointed by the starting point of the arrow.

  Here, inter prediction in the time direction (motion compensation prediction) refers to prediction that refers to an image at another time, and inter-view prediction (disparity compensation prediction) refers to prediction that refers to an image at another viewpoint. . Also, only the image preceding in the encoding / decoding order in the temporal direction can be used as a reference for temporal inter prediction, and the encoding in the viewpoint direction can be used as a reference for inter-view prediction. / Only the image preceding in the decoding order (encoding / decoding / order in the dimension of the viewpoint space).

  Here, when the viewpoint encoding / decoding order in the viewpoint direction is the order of the viewpoint 0, the viewpoint 2, the viewpoint 1, the viewpoint 4, the viewpoint 3, the viewpoint 6, the viewpoint 5, and the viewpoint 7, the image P of the viewpoint 0 ( 0, t) are all encoded / decoded in the same manner as normal AVC / H.264 using inter prediction (motion compensation prediction) in the time direction without referring to images of other viewpoints. Further, viewpoints other than viewpoint 0 (viewpoints 1 to 7) use inter-view prediction (disparity compensation prediction) predicted from decoded images of other viewpoints. For example, the image P (2, 0) of the viewpoint 2 is encoded / decoded using inter-view prediction using the decoded image of the image P (0, 0) of the viewpoint 0 as another viewpoint as a reference picture.

  The viewpoint 1 image P (1, 0) is a decoded reference image of the viewpoint 0 image P (0, 0) and the viewpoint 2 image P (2, 0), which are other viewpoints. Encode / decode using prediction. The images of the respective viewpoints at which t is 0 at the same time are P (0,0), P (2,0), P (1,0), P (4) in the viewpoint encoding / decoding order in the viewpoint direction. , 0), P (3,0), P (6,0), P (5,0), P (7,0) in this order, and then images of each viewpoint with t = 4 are obtained. Similarly, P (0,4), P (2,4), P (1,4), P (4,4), P (3,4), P in the viewpoint encoding / decoding order in the viewpoint direction. Encoding is performed in the order of (6, 4), P (5, 4), P (7, 4). Subsequently, the encoding / decoding of the image of each viewpoint with t = 2 is followed.

  In incorporating predictions between viewpoints, AVC / H. The reference picture list already defined in the H.264 system is supported by extending the reference picture used for inter-view prediction in addition to the reference picture used for temporal inter prediction (motion compensation prediction). .

  Further, the MVC scheme encodes the number of viewpoints of the multi-view image to be encoded, the encoding / decoding order in the inter-view direction, and the reference dependency relationship between the viewpoints caused by the inter-view prediction as a whole sequence. And encoding is performed by extending SPS (sequence parameter set) which is a parameter set of sequence information. In addition, an SPS MVC extension syntax structure defined in MVC JD4.0 is improved in order to reduce the code amount (see Non-Patent Document 2). The syntax structure of the SVC MVC extension will be described with reference to FIG.

  In FIG. 22, “num_views_minus1” is a parameter for encoding the number of viewpoints encoded in the encoded bit string, and is a value obtained by subtracting “1” from the number of viewpoints. Subsequently, “view_id [i]” has a structure in which each viewpoint is repeatedly and repeatedly encoded in the encoding / decoding order in the viewpoint direction. “View_id [i]” indicates the viewpoint ID of the viewpoint when the encoding / decoding order in the viewpoint direction is indicated by the index i. That is, “view_id [i]” indicates the viewpoint ID of the i-th viewpoint in the encoding / decoding order in the viewpoint direction. Here, in the description of the present specification, the array index (subscript) starts from 0. For example, the top of the array “view_id [i]” is “view_id [0]”, and the next is “view_id [1]”. Also, when expressing the order, the first is 0th and the next is 1st. That is, the viewpoint that is first encoded / decoded in the viewpoint direction is 0th, and the viewpoint that is encoded / decoded next is 1st.

  Subsequent syntax elements "num_anchor_refs_l0 [i]", "anchor_ref_l0 [i] [j]", "num_anchor_refs_l1 [i]", "anchor_ref_l1 [i] [j]", "num_non_anchor_refs_l0 [i]", "_ref" “[j]”, “num_non_anchor_refs_l1 [i]”, and “non_anchor_ref_l1 [i] [j]” are view dependency information indicating reference dependency relationships between views.

  “Num_anchor_refs_l0 [i]” is the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 0 for the anchor picture of the i-th view in the encoding / decoding order in the view direction. “Num_anchor_refs_l0 [i]” exists for each viewpoint.

  Here, an anchor picture is an image that can be decoded without referring to an image at a different display time as a reference picture at the time of decoding. Only the anchor picture of another viewpoint at the same time can be used as the reference picture when the anchor picture is decoded. Therefore, anchor picture cannot use inter prediction in the temporal direction. For example, in the case of encoding with the reference dependency shown in FIG. 10, P (0,0), P (1,0), P (2,0), P (0,4), P (1,4) , P (2, 4), etc. are anchor pictures.

  Here, the viewpoint with the 0th encoding / decoding order in the viewpoint direction (the viewpoint to be encoded / decoded first) is always encoded without using inter-view prediction, that is, without referring to other viewpoints. . Here, in each viewpoint, when encoding is performed without using inter-view prediction, that is, without referring to another viewpoint, the number of viewpoints that can be referred to is represented by zero. Accordingly, in the viewpoint with the 0th encoding / decoding order in the viewpoint direction, the number of viewpoints used as a reference for inter-view prediction is always “0”, and therefore “num_anchor_refs_l0 [0]” is omitted. . Therefore, encoding is performed from “num_anchor_refs_l0 [i]” in which the first viewpoint (viewpoint to be encoded / decoded next) in the encoding / decoding order in the viewpoint direction, that is, index i is “1”.

  Further, “anchor_ref_l0 [i] [j]” in FIG. 22 is used as a reference for the j-th inter-view prediction in the reference picture list 0 for the anchor picture of the i-th view in the encoding / decoding order in the view direction. The value of the viewpoint ID of the selected viewpoint is shown. There are as many “anchor_ref_l0 [i] [j]” as “num_anchor_refs_l0 [i]” for each viewpoint.

  “Num_anchor_refs_l1 [i]” is the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 1 for the anchor picture of the i-th view in the encoding / decoding order in the view direction. “Num_anchor_refs_l1 [i]” exists for each viewpoint. Here, for the same reason as described above, since the number of viewpoints that can be referred to in inter-view prediction is always 0 in the viewpoint with the 0th encoding / decoding order in the viewpoint direction, “num_anchor_refs_l1 [0]” is It is omitted. Therefore, encoding is performed from “num_anchor_refs_l1 [i]” in which the first viewpoint (the viewpoint to be encoded / decoded next) in the encoding / decoding order in the viewpoint direction, that is, the index i is 1. “Anchor_ref_l1 [i] [j]” is the viewpoint ID of the viewpoint used as a reference for the j-th inter-view prediction in the reference picture list 1 for the anchor picture of the i-th viewpoint in the encoding / decoding order in the viewpoint direction. Indicates the value of. There are as many “anchor_ref_l1 [i] [j]” as “num_anchor_refs_l1 [i]” for each viewpoint.

  Further, “num_non_anchor_refs_l0 [i]” is the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 0 for the non-anchor picture of the i-th view in the encoding / decoding order in the view direction. “Num_non_anchor_refs_l0 [i]” exists for each viewpoint.

  Here, the non-anchor picture is an image excluding the anchor picture. When decoding a non-anchor picture, an image at a different display time can be referred to as a reference picture. That is, inter prediction in the time direction can be used. For example, in FIG. 10, P (0,1), P (1,1), P (2,1), P (0,2), P (1,2), P (2,2), etc. are non- It is an anchor picture. Here, for the same reason as described above, since the number of viewpoints that can be referred to in inter-view prediction is always “0” in the viewpoint with the 0th encoding / decoding order in the viewpoint direction, num_non_anchor_refs_l0 [0] is It is omitted. Accordingly, encoding is performed from “num_non_anchor_refs_l0 [i]” in which the first viewpoint (the viewpoint to be encoded / decoded next) in the encoding / decoding order in the viewpoint direction, that is, the index i is “1”.

  Further, “non_anchor_ref_l0 [i] [j]” in FIG. 22 is used as a reference for the j-th inter-view prediction in the reference picture list 0 for the non-anchor picture of the i-th view in the encoding / decoding order in the view direction. Indicates the value of the viewpoint ID of the viewpoint used. There are the same number of “non_anchor_ref_l0 [i] [j]” as “num_non_anchor_refs_l0 [i]” for each viewpoint.

  “Num_non_anchor_refs_l1 [i]” is the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 1 for the non-anchor picture of the i-th view in the encoding / decoding order in the view direction. “Num_non_anchor_refs_l1 [i]” exists for each viewpoint. Here, for the same reason as described above, since the number of viewpoints that can be referred to in inter-view prediction is always “0” in the viewpoint with the 0th encoding / decoding order in the viewpoint direction, “num_non_anchor_refs_l1 [0] "Is omitted. Therefore, encoding is performed from “num_non_anchor_refs_l1 [i]” in which the first viewpoint (viewpoint to be encoded / decoded next) in the encoding / decoding order in the viewpoint direction, that is, the index i is “1”.

  Further, “non_anchor_ref_l1 [i] [j]” is a view used as a reference for the j-th inter-view prediction in the reference picture list 0 for the non-anchor picture of the i-th view in the encoding / decoding order in the view direction. Indicates the value of the viewpoint ID. There are the same number of “non_anchor_ref_l1 [i] [j]” as “num_non_anchor_refs_l1 [i]” for each viewpoint. Each syntax element is encoded without a code by a technique called exponential Golomb coding.

  Exponential Golomb coding used here is a kind of universal coding, which is a variable length coding method without using a conversion table. In the exponent Golomb code, a bit sequence of “0” called “prefix” is followed by “1” of 1 bit, and a bit sequence of the same number of bits as the prefix of “0” or “1” called suffix is followed. . If the number of prefix bits is n and the suffix value is s, the value ν of the bit string encoded by the unsigned exponential Golomb code is derived by the following equation.

ν = 2 ⁿ -1 + s (1)
FIG. 23 shows the relationship between the bit string encoded by the unsigned exponential Golomb code and the code number. For example, when the bit string to be decoded is “0001010”, three “0” s are consecutive first, so the number of prefix bits n is “3”. The subsequent “1” is omitted, and the suffix bit string corresponding to the prefix bit number of 3 bits is “010”. Therefore, the suffix value s is “2” in decimal. Therefore, according to equation (1), the code number ν of this bit string is 9 (= 2 ³ -1 + 2).

  Further, according to the syntax structure shown in FIG. 22 defined in the MVC method, each syntax element of the MVC extension portion of the SPS when encoding a multi-view image consisting of eight viewpoints with reference dependency shown in FIG. An example of the value is shown in FIG. Here, the viewpoint encoding / decoding order in the viewpoint direction is the order of viewpoint 0, viewpoint 2, viewpoint 1, viewpoint 4, viewpoint 3, viewpoint 6, viewpoint 5, viewpoint 7, and the viewpoint ID of viewpoint 0 is “8”. ", The viewpoint ID of the viewpoint 1 is" 9 ", the viewpoint ID of the viewpoint 2 is" 10 ", the viewpoint ID of the viewpoint 3 is" 11 ", the viewpoint ID of the viewpoint 4 is" 12 ", and the viewpoint ID of the viewpoint 5 is" 13 " ”, The viewpoint ID of the viewpoint 6 is“ 14 ”, and the viewpoint ID of the viewpoint 7 is“ 15 ”.

  First, since the number of viewpoints of the multi-viewpoint image shown in FIG. 10 is eight, “7” is encoded with an unsigned exponential Golomb code for “num_views_minus1”. The bit string at that time is “0001000”, which is 7 bits. Next, the value of “view_id [0]” is “8” which is the viewpoint ID of the viewpoint 0 which is the 0th viewpoint (first viewpoint) in the encoding / decoding order of the viewpoint in the viewpoint direction. The bit string at that time is “0001001”, which is 7 bits. Similarly, the value of “view_id [1]” is “10” which is the viewpoint ID of the viewpoint 2 which is the first viewpoint (the 0th next viewpoint) in the encoding / decoding order of the viewpoint in the viewpoint direction. The bit string becomes “0001011”, and the value of “view_id [2]” is “9” which is the viewpoint ID of the viewpoint 1 that is encoded second in the viewpoint encoding / decoding order in the viewpoint direction. The bit string becomes “0001010”. The following “view_id [3]” to “view_id [7]” are similarly encoded.

  Subsequently, the syntax element of the view-dependent information is encoded. Here, since view 0 that is the 0th view (first view) in the encoding / decoding order in the view direction does not always refer to another view, “num_anchor_refs_l0 [0]” and “num_anchor_refs_l1 [0]” are Do not encode. When encoding the anchor picture of the viewpoint 2 that follows the viewpoint 0 in the encoding / decoding order in the viewpoint direction, only the viewpoint 0 is referred to as the reference picture of the reference picture list 0, and the reference picture list 1 is not used. Turn into. In the reference picture list 0, since the number of viewpoints referred to in inter-view prediction is one, “1” is encoded as the value of “num_anchor_refs_l0 [1]”, and “anchor_ref_l0 [1] [0]” is referred to. The viewpoint ID value “8” of viewpoint 0 is encoded with an unsigned exponential Golomb code, and the bit string at that time is “0001001”, which is 7 bits.

  Subsequently, “0” is encoded as the value of “num_anchor_refs_l1 [1]”. Next, when encoding the anchor picture of the viewpoint 1 to be encoded, the viewpoint 0 is referred to as the reference picture of the reference picture list 0, the viewpoint 2 is referred to as the reference picture of the reference picture list 1, and is referred to by inter-view prediction. Since the number of viewpoints is one, “1” is encoded as the value of “num_anchor_refs_l0 [2]”, and “anchor_ref_l0 [2] [0]” is the value of the viewpoint ID of the viewpoint 0 to be referred to. 8 "is encoded. Further, “1” is encoded as the value of “num_anchor_refs_l1 [2]”, and “10” that is the value of the viewpoint ID of the viewpoint 2 to be referenced is encoded as “anchor_ref_l1 [2] [0]”. The following syntax elements of the viewpoint-dependent information are encoded in the same manner.

  By encoding the parameters, that is, the number of viewpoints and the view dependency information of each viewpoint, on the encoding side, it is possible to determine the reference dependency of each viewpoint as the entire sequence on the decoding side. The reference dependency information of each viewpoint is used for decoding processing such as initialization of a reference picture list for inter-view prediction pictures.

JP-A 61-144191 JP-A-6-98312 Joint Draft 4.0 on Multiview Video Coding, Joint Video Team of ISO / IEC MPEG & ITU-T VCEG, JVT-X209, July 2007 Comments on MVC high level syntax, J.H.Yang et al., Joint Video Team of ISO / IEC MPEG & ITU-T VCEG, JVT-Y061, October 2007

  In the MVC method, when encoding a multi-viewpoint image having a large number of viewpoints, temporal direction inter prediction (motion compensation prediction) using temporal direction redundancy and orthogonal transformation using spatial direction redundancy are used. In addition, encoding efficiency can be further improved by performing encoding compression using inter-view prediction (parallax compensation prediction) using redundancy between viewpoints.

  The MVC system has a mechanism for encoding view dependency information indicating reference dependency relationships between viewpoints as a whole sequence for inter-view prediction, and an SPS (sequence parameter set) that is a parameter set of sequence information. Encoding is performed by extending. However, since SPS is an important parameter for the entire sequence, it is necessary to reduce the amount of codes as much as possible while satisfying the function.

  The present invention has been made in view of the above points. An encoding bit sequence is generated by encoding without omitting redundant view-dependent information on the encoding side, and the encoded encoding bit sequence is decoded. An object of the present invention is to provide a multi-view image decoding method, a multi-view image decoding apparatus, and a multi-view image decoding program for deriving view-dependent information in which encoding is omitted.

In order to achieve the above object, the first invention is a multi-view image signal including image signals of respective viewpoints respectively obtained from a plurality of set viewpoints, and the image signal of one viewpoint is actually transmitted from one viewpoint. The encoded data to be decoded in the encoded bit string obtained by encoding the multi-view image signal that is an image signal obtained by photographing the image signal or the image signal generated as a virtual image taken from one viewpoint. A multi-view image decoding method for decoding, comprising:
The encoded data to be decoded includes a plurality of first encoded data obtained by encoding information for specifying an encoding / decoding order between a plurality of viewpoints of each viewpoint in encoding of an image signal of each viewpoint; I-th viewpoint (where i is a natural number) when the first viewpoint is the first viewpoint in the encoding / decoding order between the two viewpoints and the next viewpoint is the first viewpoint. , Information on the number of viewpoints of zero or more that can be referred to in inter-view prediction that refers to decoded image signals of other viewpoints when coding the image signal of each viewpoint (where 0 is a viewpoint that can be referred to in inter-view prediction) In this viewpoint, encoding is performed without using inter-view prediction.) The second encoded data obtained by encoding for each of the first and subsequent viewpoints, and the jth in the encoding / decoding order. (Where j is a natural number greater than or equal to 2) In this case, when the number of viewpoints that can be referred to in inter-view prediction is 1 or more, the third encoding obtained by encoding the information for specifying the viewpoint to be referred to in the inter-view prediction in the relevant viewpoint for each viewpoint Data and
A first step of decoding first encoded data to obtain information for specifying an encoding / decoding order between a plurality of viewpoints, and a second step of decoding the second encoded data for reference in inter-view prediction In the second step of obtaining information on the number of possible viewpoints, and in the first viewpoint, when the number of viewpoints that can be referred to in the inter-view prediction obtained by decoding in the second step is 1, in the inter-view prediction A third step in which information for specifying a viewpoint to be referred to is information for specifying a zeroth viewpoint obtained from information for specifying an encoding / decoding order among a plurality of viewpoints decoded in the first step; And decoding the third encoded data for each viewpoint to obtain information for identifying the viewpoint to be referred to in the inter-view prediction.

In order to achieve the above object, the second invention is a multi-viewpoint image signal including image signals of respective viewpoints respectively obtained from a plurality of set viewpoints, and an image signal of one viewpoint is Decoding target code in an encoded bit string obtained by encoding an image signal obtained by actually photographing from a viewpoint or a multi-viewpoint image signal that is generated as a virtual photograph from one viewpoint A multi-viewpoint image decoding device for decoding the digitized data,
The encoded data to be decoded includes a plurality of first encoded data obtained by encoding information for specifying an encoding / decoding order between a plurality of viewpoints of each viewpoint in encoding of an image signal of each viewpoint; I-th viewpoint (where i is a natural number) when the first viewpoint is the first viewpoint in the encoding / decoding order between the two viewpoints and the next viewpoint is the first viewpoint. , Information on the number of viewpoints of zero or more that can be referred to in inter-view prediction that refers to decoded image signals of other viewpoints when coding the image signal of each viewpoint (where 0 is a viewpoint that can be referred to in inter-view prediction) In this viewpoint, encoding is performed without using inter-view prediction.) The second encoded data obtained by encoding for each of the first and subsequent viewpoints, and the jth in the encoding / decoding order. (Where j is a natural number greater than or equal to 2) In this case, when the number of viewpoints that can be referred to in inter-view prediction is 1 or more, the third encoding obtained by encoding the information for specifying the viewpoint to be referred to in the inter-view prediction in the relevant viewpoint for each viewpoint Data and
In inter-view prediction, a first decoding unit that decodes the first encoded data to obtain information for specifying an encoding / decoding order between a plurality of viewpoints, and decodes the second encoded data The second decoding means for obtaining information on the number of view points that can be referred to, and the first view point when the number of view points that can be referred to in the inter-view prediction obtained by decoding in the second step is 1. Third decoding means that uses information specifying a viewpoint to be referred to in prediction as information specifying a zeroth viewpoint obtained from information specifying an encoding / decoding order between a plurality of viewpoints decoded in the first step And fourth decoding means for decoding the third encoded data for each viewpoint and obtaining information for identifying the viewpoint to be referred to in inter-view prediction.

  Furthermore, in order to achieve the above object, the third aspect of the invention is a multi-viewpoint in which each step of the first aspect of the invention is decoded by the computer with respect to the same encoded data as the first aspect of the invention. It is an image decoding program.

  According to the present invention, when multi-viewpoint images are decoded, the encoding is performed by omitting the information specifying the viewpoint used as the reference for inter-view prediction of the first viewpoint in the encoding / decoding order in the viewpoint direction. From this encoded bit string, it is possible to derive information for identifying a viewpoint used as a reference for inter-view prediction of the first viewpoint in the encoding / decoding order in the viewpoint direction, and to decode a multi-viewpoint image. It is possible to reduce the generated code amount of the SPS (sequence parameter set) encoded bit string generated on the encoding side.

  Further, according to the present invention, it is possible to reduce the transmission amount at the time of transmission by reducing the generated code amount of the encoded bit string, and it is possible to reduce the data amount when recording to a storage medium or the like. Thus, the processing amount of encoding / decoding can be reduced, and further, the error resistance of SPS can be improved.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Multi-view image encoding apparatus and multi-view image encoding method)
First, a multi-view image decoding method, a multi-view image decoding apparatus, a multi-view image encoding method and a multi-view image encoding apparatus for generating an encoded bit string to be decoded by a multi-view image decoding program according to the present invention will be described.

  FIG. 1 is a block diagram illustrating an example of a multi-view image encoding device. As shown in the figure, the multi-view image encoding apparatus includes an encoding management unit 101, a sequence information encoding unit 102, a picture information encoding unit 103, an image signal encoding unit 104, and a multiplexing 105, and an input The multi-view image signal to be encoded is encoded to output encoded data (encoded bit string). Here, the multi-view image signal is a multi-view image signal including the image signals of the respective viewpoints respectively obtained from the set two or three or more viewpoints. It is an image signal obtained by actually photographing from one viewpoint, or an image signal generated as a virtual photograph from one viewpoint.

  In the description of the multi-view image encoding apparatus, the multi-view image encoding apparatus according to the MVC method, which is an extension of the AVC / H.264 encoding method to a multi-view image, will be described.

  First, the syntax structure of an encoded bit string generated by encoding with the multi-view image encoding device in FIG. 1 will be described. FIG. 11 is a diagram showing the syntax structure of the MVC extension part in the SPS according to the present invention.

  In the syntax structure shown in FIG. 11, as in the conventional example shown in FIG. 22, first, “num_views_minus1”, which is a syntax element indicating a value obtained by subtracting 1 from the number of viewpoints encoded in the encoded bit string. "View_id [i]", which is a syntax element indicating the viewpoint ID of the i-th viewpoint in the encoding / decoding order in the viewpoint direction, is encoded for each viewpoint. It has a structure in which encoding is repeated continuously in the encoding / decoding order in the direction.

  Subsequent syntax elements "num_anchor_refs_l0 [i]", "anchor_ref_l0 [i] [j]", "num_anchor_refs_l1 [i]", "anchor_ref_l1 [i] [j]", "num_non_anchor_refs_l0 [i]", "_ref" “[j]”, “num_non_anchor_refs_l1 [i]”, and “non_anchor_ref_l1 [i] [j]” are view dependency information indicating reference dependency relationships between views. "Num_anchor_refs_l0 [i]", "num_anchor_refs_l1 [i]", "num_non_anchor_refs_l0 [i]", and "num_non_anchor_refs_l1 [i]" are encoded in the encoding / decoding order for each viewpoint except index i is "0" It has become a structure.

  Here, in the multi-view image encoding method, apparatus, and program, and the multi-view image decoding method, apparatus, and program of the present invention described later, for the first viewpoint in the encoding / decoding order in the view direction, It is defined that only the decoded image of the 0th viewpoint can be used as a reference for inter prediction.

  Further, the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 0 for the anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction, that is, “num_anchor_refs_l0 [1]” is “1”. ”, The viewpoint ID of the viewpoint used as a reference for inter-view prediction in the reference picture list 0 for the anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction, that is,“ anchor_ref_l0 [1] [0 The value of “]” is defined as the viewpoint ID of the 0th viewpoint in the encoding direction of the viewpoint direction, that is, the value of “view_id [0]”.

  Similarly, the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 1 for the anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction, that is, “num_anchor_refs_l1 [1]” is “ 1 ”, the viewpoint ID of the viewpoint used as a reference for inter-view prediction in the reference picture list 1 for the anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction, that is,“ anchor_ref_l1 [1] [ The value “0]” is defined as the viewpoint ID of the 0th viewpoint in the encoding direction of the viewpoint direction, that is, the value of “view_id [0]”.

  Similarly, the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 0 for the non-anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction, that is, “num_non_anchor_refs_l0 [1]” is “ 1 ”, the viewpoint ID of the viewpoint used as a reference for inter-view prediction in the reference picture list 0 for the non-anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction, that is,“ non_anchor_ref_l0 [1] The value of “[0]” is defined as the viewpoint ID of the 0th viewpoint in the encoding direction of the viewpoint direction, that is, the value of “view_id [0]”.

  Similarly, the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 1 for the non-anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction, that is, “num_non_anchor_refs_l1 [1]” is When “1”, the viewpoint ID of the viewpoint used as a reference for inter-view prediction in the reference picture list 1 for the non-anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction, that is, “non_anchor_ref_l1 [1 ] [0] ”is defined as the viewpoint ID of the 0th viewpoint in the encoding direction of the viewpoint direction, that is, the value of“ view_id [0] ”.

  By specifying in this way on both the encoding side and the decoding side, the syntax elements “anchor_ref_l0 [1] [0]”, “anchor_ref_l1 [1] [0]”, “non_anchor_ref_l0 [1] whose index i is“ 1 ”are defined. ] [0] ”and“ non_anchor_ref_l1 [1] [0] ”can be derived on the decoding side without encoding on the encoding side. Therefore, in the present invention, the syntax elements “anchor_ref_l0 [1] [0]”, “anchor_ref_l1 [1] [0]”, “non_anchor_ref_l0 [1] [0]”, “non_anchor_ref_l1 [1]” having the index i of “1” are used. [0] is always omitted.

  Further, according to the present invention, only “num_anchor_refs_l0 [i]” and “num_anchor_refs_l1 [i]” are used for syntax elements having an index i larger than “1”, that is, only the second and subsequent viewpoints in the encoding / decoding order in the viewpoint direction. "," Num_non_anchor_refs_l0 [i] "," anchor_ref_l0 [i] [j] "," anchor_ref_l1 [i] [j] "," non_anchor_ref_l0 [i] [j] [j] "depending on the value of" num_non_anchor_refs_l1 [i] " "," Non_anchor_ref_l1 [i] [j] "is encoded.

  When only the viewpoint that precedes the encoding / decoding order in the viewpoint direction can be used as a reference for inter-view prediction, the image of the first viewpoint is encoded in the encoding / decoding order in the viewpoint direction. In this case, only the decoded image of the 0th viewpoint can be used as a reference for inter-view prediction. Furthermore, "num_anchor_refs_l0 [i]", "num_anchor_refs_l1 [i]", "num_non_anchor_refs_l0 [i]", and "num_non_anchor_refs_l1 [i]" can take values of "0" or "1". I can't take it. Therefore, after applying the above definition of the present invention, the syntax elements “anchor_ref_l0 [1] [0]”, “anchor_ref_l1 [1] [0]”, “non_anchor_ref_l0 [1] [0] whose index i is“ 1 ”are used. ] ”And“ non_anchor_ref_l1 [1] [0] ”can be encoded with the same prediction structure as in the conventional example, even if encoding and decoding are always omitted.

  Furthermore, the syntax elements “anchor_ref_l0 [1] [0]”, “anchor_ref_l1 [1] [0]”, “non_anchor_ref_l0 [1] [0]”, “non_anchor_ref_l1 [1] [0]” with the index i being “1” ”Is always omitted, it is possible to reduce the generated code amount of the SPS (sequence parameter set) encoded bit string. In addition, by reducing the amount of generated code of the encoded bit string, it is possible to reduce the amount of transmission during transmission and the amount of data when recording to a storage medium or the like. The amount of processing can be reduced.

  On the other hand, on the decoding side of the present invention, as will be described later, when decoding a multi-viewpoint image, a viewpoint used as a reference for inter-view prediction of the first viewpoint in the encoding / decoding order in the viewpoint direction is specified. Deriving information for identifying the viewpoint used as a reference for inter-view prediction of the first viewpoint in the encoding / decoding order in the viewpoint direction from the encoded bit string encoded by omitting the information, the multi-view image Therefore, it is possible to reduce the generated code amount of the SPS (sequence parameter set) encoded bit string generated on the encoding side.

  In particular, since SPS is parameter information related to the encoding of the entire sequence, it is indispensable when decoding other data belonging to the sequence in the encoded bit string, compared to other data included in the encoded bit string. The most important information. Since only SPS is decoded to obtain system information of an encoded bit string, or SPS information needs to be decoded first when decoding picture information and slice header information, the effect of reducing the amount of processing is other than that. It will be larger than the data. Also, depending on the system, the SPS may be transmitted separately from the VCL NAL unit, which is an encoded bit sequence in which an encoding mode, a motion / disparity vector, an encoded residual signal, and the like are encoded. Also in this case, the effect of reducing the SPS code amount and the processing amount is greater than that of other data.

  Further, by reducing the amount of generated code of the SPS encoded bit string, the risk that an error occurs in the SPS encoded bit string during transmission is reduced, and error tolerance can be increased. SPS is parameter information related to the coding of the entire sequence, and if an error occurs in the coded bit string of the SPS, the entire sequence is fatally affected. Therefore, the effect of improving the error resistance of the SPS according to the present embodiment is very effective. Big.

  FIG. 12 shows an example of each syntax element and its value in the MVC extension part of SPS when an 8-view multi-view image is encoded with the reference dependency shown in FIG. 10 according to the syntax structure shown in FIG. . Here, as in the conventional example shown in FIG. 24, the encoding / decoding order of viewpoints in the viewpoint direction is the order of viewpoint 0, viewpoint 2, viewpoint 1, viewpoint 4, viewpoint 3, viewpoint 6, viewpoint 5, and viewpoint 7. The viewpoint ID of the viewpoint 0 is “8”, the viewpoint ID of the viewpoint 1 is “9”, the viewpoint ID of the viewpoint 2 is “10”, the viewpoint ID of the viewpoint 3 is “11”, and the viewpoint ID of the viewpoint 4 is “12”. ”, The viewpoint ID of the viewpoint 5 is“ 13 ”, the viewpoint ID of the viewpoint 6 is“ 14 ”, and the viewpoint ID of the viewpoint 7 is“ 15 ”.

  Compared with the syntax structure shown in FIG. 12 and the conventional syntax structure shown in FIG. 24, both the conventional example and the present embodiment are for the anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction. The value of the syntax element “num_anchor_refs_l0 [1]” indicating the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 0 is “1”.

  However, in the conventional syntax structure shown in FIG. 24, it is used as a reference for the 0th inter-view prediction in the reference picture list 0 for the anchor picture of the 1st view in the encoding / decoding order in the view direction. The syntax element “anchor_ref_l0 [1] [0]” indicating the viewpoint ID of the viewpoint is encoded with “8”, which is the viewpoint ID value of the viewpoint 0 to be referenced, with an unsigned exponential Golomb code. In the syntax structure shown in FIG. 12, the encoding of “anchor_ref_l0 [1] [0]” is omitted. The omitted “anchor_ref_l0 [1] [0]” value is derived on the decoding side.

  Therefore, according to the syntax structure shown in FIG. 12, the code amount of the SPS encoded bit string generated by the encoding can be reduced as compared with the conventional example, which is refined.

  Next, the operation of the multi-view image encoding device in FIG. 1 will be described. In FIG. 1, first, the encoding management unit 101 calculates a new parameter as necessary based on an encoding parameter set from the outside, and sets parameter information (SPS) related to the entire sequence and a picture. Management related to coding including related parameter information (PPS), header information related to a slice of a picture (slice header), and the like is performed. Furthermore, the encoding management unit 101 manages the reference dependency relationship and the encoding / decoding order of the encoding target image.

  Regarding the reference dependency, it is managed whether or not to refer to the decoded image of another viewpoint in units of viewpoints, and when the encoding target image is encoded in units of pictures or slices, the decoded images of other viewpoints are referred to Whether to perform inter-view prediction (disparity compensation prediction) to be used as an image, or a decoded image obtained by decoding an encoding target image after encoding an encoding target image of another viewpoint as a reference image It is managed whether or not it is used and which reference image is referred to from among a plurality of reference image candidates. As for the encoding / decoding order, the encoding / decoding order is managed so that the decoding side can start decoding after the reference image referenced by the image of the encoded bit string to be decoded is decoded, in the reference dependency relationship. To do.

  Next, the sequence information encoding unit 102 encodes parameter information (SPS) related to the entire sequence managed by the encoding management unit 101. Here, the MVC extension part of SPS is also encoded according to the syntax structure shown in FIG.

  FIG. 2 is a block diagram illustrating an example of the sequence information encoding unit 102. As shown in FIG. 2, the sequence information encoding unit 102 includes a sequence information encoding unit 201, a view number information encoding unit 202, a coding order information encoding unit 203, and a reference view number information encoding other than the MVC extension part. Unit 204 and a reference view information encoding unit 205. Also, the reference view number information encoding unit 204 and the reference view information encoding unit 205 constitute a view dependent information encoding unit 206. Furthermore, the viewpoint number information encoding unit 202 and the encoding order information encoding unit 203 together with the viewpoint dependent information encoding unit 206 constitute an SPS MVC extended partial encoding unit 207.

  The sequence information encoding unit 201 other than the MVC extension portion encodes sequence information other than the MVC extension portion, that is, SPS (sequence parameter set) in the AVC / H.264 system. On the other hand, the SPS MVC extended partial encoding unit 207, which includes the view number information encoding unit 202, the encoding order information encoding unit 203, the reference view number information encoding unit 204, and the reference view information encoding unit 205, According to the syntax structure shown in FIG. 11, the MVC extension part of the information (SPS) related to the entire sequence is encoded.

  First, the viewpoint number information encoding unit 202 encodes a syntax element “num_views_minus1” indicating a value obtained by subtracting “1” from the number of viewpoints encoded in the encoded bit string. Next, the encoding order information encoding unit 203 encodes / decodes the syntax element “view_id [i]” indicating the viewpoint ID of the i-th viewpoint in the encoding / decoding order in the viewpoint direction in the viewpoint direction. Encode in order.

  Next, the view dependent information is encoded by the reference view number information encoding unit 204 and the reference view information encoding unit 205 that constitute the view dependent information encoding unit 206. The view-dependent information to be encoded here includes the syntax elements “num_anchor_refs_l0 [i]”, “anchor_ref_l0 [i] [j]”, “num_anchor_refs_l1 [i]”, “anchor_ref_l1 [i] [j]”, “num_non_anchor_refs_0” [i] ”,“ non_anchor_ref_l0 [i] [j] ”,“ num_non_anchor_refs_l1 [i] ”,“ non_anchor_ref_l1 [i] [j] ”.

  The reference view number information encoding unit 204 can be used as a reference for inter-view prediction in the reference picture list 0 and the reference picture list 1 for the anchor picture and the non-anchor picture of each viewpoint in the view dependency information. The syntax elements “num_anchor_refs_l0 [i]”, “num_anchor_refs_l1 [i]”, “num_non_anchor_refs_l0 [i]”, and “num_non_anchor_refs_l1 [i]” indicating the number of viewpoints are encoded. However, as described above, in the present embodiment, the reference view number information encoding unit 204 does not encode the syntax element whose index i is “0”, and the index i is “1” or more. Encode syntax elements.

  The reference viewpoint information encoding unit 205 sets the viewpoint ID of the viewpoint used as a reference for the j-th inter-view prediction in the reference picture list 0 and the reference picture list 1 for the anchor picture and the non-anchor picture of each viewpoint. The syntax elements “anchor_ref_l0 [i] [j]”, “anchor_ref_l1 [i] [j]”, “non_anchor_ref_l0 [i] [j]”, and “non_anchor_ref_l1 [i] [j]” are encoded. However, as described above, in the present embodiment, the reference viewpoint information encoding unit 205 does not encode the syntax element whose index i is “0” or “1”, and the index i is “2” or more. Are encoded.

  Returning again to FIG. The picture information encoding unit 103 encodes information (PPS) related to a picture managed by the encoding management unit 101. Also, the image signal encoding unit 104 encodes information (slice header) related to the slice managed by the encoding management unit 101 and the supplied encoding target image signal in units of slices. When encoding an image signal, inter-view prediction may be used. In this case, a reference image for inter-view prediction is selected based on the viewpoint dependency information.

  The multiplexing unit 105 includes an encoded bit sequence of sequence information obtained by encoding by the sequence information encoding unit 102, an encoded bit sequence of picture information obtained by encoding by the picture information encoding unit 103, and an image signal Header information for handling the slice information obtained by encoding by the encoding unit 104 and the encoded bit sequence of the image signal in units of NAL units is added and multiplexed to obtain an encoded bit sequence of the multi-view image.

  Next, the multi-view image encoding processing procedure by the multi-view image encoding device shown in FIG. 1 will be described with reference to the flowcharts of FIGS. Since the processing operation of each step is the same as that described with reference to the block diagrams of FIG. 1 and FIG. 2, only the processing procedure will be described here in association with FIG. 1 and FIG.

  First, parameter information related to encoding of the entire sequence is encoded, and an encoded bit string of parameter information related to encoding of the entire sequence is generated (step S101 in FIG. 3). The processing in step S101 corresponds to the encoding operation in the sequence information encoding unit 102 in the multi-viewpoint image encoding device in FIG.

  An example of the sequence information encoding process procedure in step S101 will be described in more detail with reference to the flowchart of FIG. First, sequence information other than the MVC extension is encoded (step S111). The processing in step S111 corresponds to the encoding operation in the sequence information encoding unit 201 other than the MVC extension portion in FIG.

  Subsequently, the MVC extension portion of the information (SPS) related to the entire sequence is encoded according to the syntax structure shown in FIG. 11 (steps S112 to S114). First, information on the number of viewpoints to be encoded into the encoded bit string is encoded (step S112). The processing in step S112 corresponds to the encoding operation in the viewpoint number information encoding unit 202 in FIG. Subsequently, the viewpoint ID information of each viewpoint is encoded in the encoding / decoding order in the viewpoint direction (step S113). The processing in step S113 corresponds to the encoding operation in the encoding order information encoding unit 203 in FIG.

  An example of the viewpoint ID encoding processing procedure in the encoding / decoding order in the viewpoint direction in step S113 will be described in more detail with reference to the flowchart of FIG. First, an index i indicating the encoding / decoding order in the viewpoint direction is set to “0” (step S121). Subsequently, it is determined whether or not the value of index i is equal to or less than (number of viewpoints-1) (step S122). If the value of index i is not less than (number of viewpoints−1), the encoding process in step S113 is terminated. If the value of index i is (number of viewpoints-1) or less, the process proceeds to step S123, and the processing from step S122 to step S124 is repeated until the value of index i is not less than (number of viewpoints-1). In step S123, the syntax element “view_id [i]” indicating the viewpoint ID of the i-th viewpoint in the encoding / decoding order in the viewpoint direction is encoded. Subsequently, in step S124, “1” is added to the index i, and the process proceeds again to step S122.

  Returning to the flowchart of FIG. Subsequent to the processing in step S113 described above, in step S114, the view-dependent information is encoded. The processing in step S114 corresponds to the encoding operation in the view dependent information encoding unit 206 configured by the reference view number information encoding unit 204 and the reference view information encoding unit 205 in FIG.

  An example of the processing procedure for encoding the viewpoint dependent information in step S114 will be described in more detail with reference to the flowchart of FIG. In the encoding process of the view dependency information in step S114, the view dependency information of the anchor picture is encoded (step S131), and then the view dependency information of the non-anchor picture is encoded (step S132). When the process in step S132 is completed, the view-dependent information encoding process in FIG. 6 is completed.

  An example of the processing procedure for encoding the viewpoint dependent information of the anchor picture in step S131 will be described in more detail with reference to the flowchart of FIG. In the encoding process of the anchor picture view-dependent information in FIG. 7, the index i is “0”, that is, the viewpoint encoded / decoded in the encoding / decoding order in the viewpoint direction (first encoding / decoding). (Viewed viewpoint) is always encoded without using inter-view prediction, that is, without referring to other viewpoints, the number of viewpoints used as a reference for inter-view prediction is always “0”, and “num_anchor_refs_l0 [0 ] ”And“ num_anchor_refs_l0 [1] ”are not encoded, and the value is always“ 0 ”. Therefore, the index i indicating the encoding / decoding order in the viewpoint direction is set to “1” (step S141).

  Subsequently, it is determined whether or not the value of the index i is (the number of viewpoints−1) or less (step S142). If the value of the index i is not less than (number of viewpoints−1), the encoding process of the anchor-picture viewpoint-dependent information is terminated. When the value of index i is (number of viewpoints-1) or less, the process proceeds to step S143, and the processing from step S142 to step S155 is repeated until the value of index i is not less than (number of viewpoints-1).

  In step S143, the syntax element “num_anchor_refs_l0 [i] indicating the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 0 for the anchor picture of the i-th view in the encoding / decoding order in the view direction. Is encoded.

  Subsequently, in step S144, it is determined whether or not the index i is greater than “1”. When the index i is larger than “1”, the process proceeds to step S145, and the value of the index j is set to “0”. Subsequently, it is determined whether the value of index j is smaller than “num_anchor_refs_l0 [i]” (step S146). If the value of index j is smaller than the value of “num_anchor_refs_l0 [i]”, the value of index j is “num_anchor_refs_l0”. The process from step S146 to step S148 is repeated until the value is equal to or greater than the value of [i].

  In step S147, the syntax element “anchor_ref_l0” indicating the viewpoint ID of the viewpoint used as a reference for the j-th inter-view prediction in the reference picture list 0 for the anchor picture of the i-th viewpoint in the encoding / decoding order in the viewpoint direction. [i] [j] ”is encoded, and the process proceeds to step S148. In step S148, “1” is added to the index j, and the process proceeds again to step S146.

  On the other hand, when the index i is not larger than “1” in step S144, that is, when it is determined that the index i is “1”, or the value of the index j is greater than or equal to the value of “num_anchor_refs_l0 [i]” in step S146. If it is determined that there is, the process proceeds to step S149. In step S149, the syntax element “num_anchor_refs_l1 [i] indicating the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 1 for the anchor picture of the i-th view in the encoding / decoding order in the view direction. Is encoded.

  Subsequently, it is determined whether or not the index i is larger than “1” (step S150). When the index i is larger than “1”, the process proceeds to step S151, and the value of the index j is set to “0”. Subsequently, it is determined whether or not the value of index j is smaller than “num_anchor_refs_l1 [i]” (step S152). If the value of index j is smaller than the value of “num_anchor_refs_l1 [i]”, the value of index j is “num_anchor_refs_l1”. The process from step S152 to step S154 is repeated until the value of [i] "is reached.

  When the value of the index j is smaller than the value of “num_anchor_refs_l1 [i]”, reference to the j-th inter-view prediction in the reference picture list 1 for the anchor picture of the i-th view in the encoding / decoding order in the view direction The syntax element “anchor_ref_l1 [i] [j]” indicating the viewpoint ID of the viewpoint used as is encoded (step S153). Subsequently, “1” is added to the index j (step S154), and the process proceeds again to step S152.

  On the other hand, when the index i is not larger than “1” in step S150, that is, when it is determined that the index i is “1”, or the value of the index j is “num_anchor_refs_l1 [i]” or more in step S152. If so, the process proceeds to step S155. In step S155, “1” is added to the index i and the process proceeds to step S142 again.

  Next, an example of the processing procedure for encoding the viewpoint dependent information of the non-anchor picture in step S132 of FIG. 6 will be described in more detail with reference to the flowchart of FIG. In the non-anchor picture view-dependent information encoding process of FIG. 8, “num_non_anchor_refs_l0 [0]” and “num_non_anchor_refs_l0 [1]” are not encoded for the reasons described above, and the value is always “0”. Therefore, after setting index i to “1” (step S161), it is determined whether or not the value of index i is equal to or less than (number of viewpoints−1) (step S162).

  When the value of the index i is not less than (number of viewpoints−1), the encoding processing of the viewpoint dependent information of the non-anchor picture is finished. When the value of index i is (number of viewpoints-1) or less, the process proceeds to step S163, and the processing from step S162 to step S175 is repeated until the value of index i is not less than (number of viewpoints-1).

  In step S163, the syntax element “num_non_anchor_refs_l0 [i indicating the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 0 for the non-anchor picture of the i-th view in the encoding / decoding order in the view direction. ] ".

  Subsequently, in step S164, it is determined whether or not the index i is greater than “1”. When the index i is larger than “1”, the process proceeds to step S165, and the value of the index j is set to “0”. Subsequently, it is determined whether or not the value of the index j is smaller than “num_non_anchor_refs_l0 [i]” (step S166). If the value of the index j is smaller than the value of “num_non_anchor_refs_l0 [i]”, The processing from step S166 to step S168 is repeated until the value of [i] becomes equal to or greater than the value.

  In step S167, the syntax element “indicating the viewpoint ID of the viewpoint used as a reference for the j-th inter-view prediction in the reference picture list 0 for the non-anchor picture of the i-th viewpoint in the encoding / decoding order in the viewpoint direction. Non_anchor_ref_l0 [i] [j] ”is encoded, and the process proceeds to step S168. In step S168, “1” is added to the index j, and the process proceeds again to step S166.

  On the other hand, when the index i is not greater than “1” in step S164, that is, when it is determined that the index i is “1”, or the value of the index j is greater than or equal to the value of “num_non_anchor_refs_l0 [i]” in step S166. If it is determined that there is, the process proceeds to step S169. In step S169, the syntax element “num_non_anchor_refs_l1 [i] indicating the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 1 for the non-anchor picture of the i-th view in the encoding / decoding order in the view direction. ] ".

  Subsequently, in step S172, it is determined whether or not the index i is larger than “1”. When the index i is larger than “1”, the process proceeds to step S171, and the value of the index j is set to “0”. Subsequently, it is determined whether or not the value of the index j is smaller than “num_non_anchor_refs_l1 [i]” (step S 172). The processes from step S172 to step S174 are repeated until the value of [i] is equal to or greater.

  In step S173, the syntax element “indicating the viewpoint ID of the viewpoint used as the reference for the j-th inter-view prediction in the reference picture list 1 for the non-anchor picture of the i-th viewpoint in the encoding / decoding order in the viewpoint direction. Non_anchor_ref_l1 [i] [j] ”is encoded, and the process proceeds to step S174. In step S174, “1” is added to the index j, and the process proceeds again to step S172.

  On the other hand, when the index i is not larger than “1” in step S170, that is, when it is determined that the index i is “1”, or the value of the index j is greater than or equal to the value of “num_non_anchor_refs_l1 [i]” in step S172. If it is determined that there is, the process proceeds to step S175. In step S175, “1” is added to the index i, and the process proceeds to step S162 again.

  7 and FIG. 8, the processing in steps S143 and S149 in FIG. 7 and steps S163 and S169 in FIG. 8 are performed by the encoding operation of the reference viewpoint number information encoding unit 204 in FIG. The processing in steps S147 and S153 in FIG. 7 and steps S167 and S173 in FIG. 8 corresponds to the encoding operation of the reference viewpoint information encoding unit 205 in FIG.

  Returning to the flowchart of FIG. When the process of step S114 is completed, the sequence information encoding process of FIG. 4 is completed.

  Returning to the flowchart of FIG. When the process of step S101 described with the flowcharts of FIGS. 4 to 8 is completed, the process proceeds to step S102. In step S102, the encoded bit string of the parameter information relating to the encoding of the entire sequence is multiplexed to obtain a multiplexed encoded bit string. The processing in step S102 corresponds to the multiplexing operation in the multiplexing unit 105 in the multi-view image encoding device in FIG.

  In the next step S103, parameter information related to picture encoding is encoded, and an encoded bit string of parameter information related to picture encoding is generated. The processing in step S103 corresponds to the encoding operation in the picture information encoding unit 103 in the multi-view image encoding apparatus in FIG.

  Subsequently, in step S104, the encoded bit string of the parameter information related to the encoding of the picture is multiplexed to obtain a multiplexed encoded bit string. The processing in step S104 corresponds to the multiplexing operation in the multiplexing unit 105 in the multi-view image encoding device in FIG.

  Subsequently, in step S105, the slice information and the image signal are encoded. The processing in step S105 corresponds to the processing operation in the image signal encoding unit 104 in the multi-view image encoding device in FIG.

  Subsequently, in step S106, following the bit sequence multiplexed in step S102 and step S104, the decoded image output order number o, the encoding mode, and the motion vector or disparity vector, encoding residual signal, etc. are encoded. The bit string is appropriately multiplexed into one encoded bit string or a plurality of encoded bit strings as necessary. The processing in step S106 corresponds to the multiplexing operation in the multiplexing unit 105 in the multi-view image encoding device in FIG.

  After completion of the multiplexing operation in step S106, it is determined whether or not the encoding process has been completed for all images of the multi-viewpoint image to be encoded (step S107). If completed, the multi-viewpoint image encoding processing procedure ends. If not completed, the process proceeds to step S105, and the process from step S105 to step S106 is repeated until the encoding process is completed for all the images of the multi-viewpoint image to be encoded.

  Next, the multiplexing and transmission processing procedure in the multiplexing unit 105 when transmitting via a network will be described with reference to the flowchart of FIG. In FIG. 9, a multiplexing unit 105 multiplexes data obtained by multiplexing a coded bit sequence of sequence information, a coded bit sequence of picture information, and a coded bit sequence of slice information and an image signal, as necessary. Packetization is performed based on standards such as 2-system, MP4 file format, and RTP (step S181).

  Subsequently, the multiplexing unit 105 adds a packet header to the packet based on standards such as MPEG-2 system, MP4 file format, RTP, etc. as necessary (step S182), and then via the network. Transmit (step S183).

(Multi-view image decoding apparatus and multi-view image decoding method)
Next, the multi-view image decoding method according to the present invention for decoding encoded data generated by the multi-view image encoding method, multi-view image encoding device, and multi-view image encoding program described with reference to FIGS. A multi-viewpoint decoding apparatus and a multi-viewpoint image decoding program will be described with reference to the drawings.

  FIG. 13 shows a block diagram of an embodiment of a multi-view image decoding apparatus according to the present invention. As shown in FIG. 13, the multi-view image decoding apparatus according to the present embodiment includes a separation unit 301, a decoding management unit 302, a sequence information decoding unit 303, a picture information decoding unit 304, and an image signal decoding unit 305. An encoded bit string obtained by encoding an image signal is input, and this is decoded to output a multi-view image signal.

  Next, the operation of the multi-view image decoding apparatus shown in FIG. 13 will be described in association with the AVC / H.264 encoding method. First, the separation unit 301 receives an encoded bit string that is encoded by the multi-view image encoding apparatus illustrated in FIG. 1 and transmitted via the network. It should be noted that the encoded bit string supply form in this system is not only received via network transmission, but also a code bit string recorded on a storage medium such as a DVD, or a code broadcast on BS / terrestrial broadcasts. An encoded bit string can also be received.

  Also, the separation unit 301 removes the packet header from the supplied encoded bit string, and separates the NAL unit. Further, the separation unit 301 evaluates an identifier (nal_unit_type) that identifies the type of the NAL unit included in the header part of the separated NAL unit, and the NAL unit encodes parameter information related to the coding of the entire sequence. In the case of an encoded bit string, it is supplied to the sequence information decoding unit 303, and in the case of an encoded bit string in which parameter information relating to the encoding of a picture is encoded, it is supplied to the picture information decoding unit 304, and the NAL unit Is a coded bit string in which a coding mode, a motion / disparity vector, a coded residual signal, and the like are coded, is supplied to the image signal decoding unit 305.

  The sequence information decoding unit 303 decodes a coded bit string in which parameter information (SPS) related to coding of the entire sequence separated by the separation unit 301 is coded. Here, the MVC extension part of SPS is also decoded according to the syntax structure shown in FIG.

  FIG. 14 is a block diagram showing a configuration of an embodiment of the sequence information decoding unit 303. As illustrated in FIG. 14, the sequence information decoding unit 303 includes a sequence information decoding unit 401, a viewpoint number information decoding unit 402, a decoding order information decoding unit 403, a reference view number information decoding unit 404, and reference view information other than the MVC extension portion. A generation unit 405, a reference viewpoint information decoding unit 406, and a switch 407 are included. The switch 407 is switched according to the parameter information (SPS) related to the encoding of the entire sequence, the syntax structure of the MVC extension of the SPS shown in FIG. 11, and the decoding results of the decoding units 401 to 404, 406. The digitized bit string is sequentially supplied to the decoding units 401 to 404, 406.

  Further, the reference view number information decoding unit 404, the reference view information generation unit 405, and the reference view information decoding unit 406 constitute a view dependency information decoding unit 408. Furthermore, the viewpoint number information decoding unit 402 and the decoding order information decoding unit 403 constitute an SPS MVC extended partial decoding unit 409 together with the viewpoint dependent information decoding unit 408.

  The sequence information decoding unit 401 other than the MVC extension portion in FIG. 14 decodes sequence information other than the MVC extension portion, that is, SPS (sequence parameter set) in the AVC / H.264 system, from the encoded bit string. In addition, the SPS MVC extended partial decoding unit 409 including the view number information decoding unit 402, the decoding order information decoding unit 403, the reference view number information decoding unit 404, the reference view information generating unit 405, and the reference view information decoding unit 406 is: The MVC extension part of the information (SPS) related to the entire sequence is decoded from the encoded bit string according to the syntax structure shown in FIG.

  In this SPS MVC extended partial decoding unit 409, first, the number-of-views information decoding unit 402 has a syntax element “num_views_minus1” indicating a value obtained by subtracting “1” from the number of viewpoints encoded in the encoded bit sequence from the encoded bit sequence. "Is decrypted. Next, the decoding order information decoding unit 403 sequentially decodes the syntax element “view_id [i]” indicating the viewpoint ID of the i-th viewpoint in the encoding / decoding order in the viewpoint direction for each viewpoint from the encoded bit string. To do. Since “view_id_ [i]” indicating the view ID in the encoding / decoding order in the view direction is encoded in the supplied encoded bit string, each viewpoint is encoded in any encoding / decoding order You can know what is being done.

  Next, a view dependency information decoding unit 408 including a reference view number information decoding unit 404, a reference view information generation unit 405, and a reference view information decoding unit 406 decodes the view dependency information from the encoded bit string. The view-dependent information to be decoded here includes the syntax elements “num_anchor_refs_l0 [i]”, “anchor_ref_l0 [i] [j]”, “num_anchor_refs_l1 [i]”, “anchor_ref_l1 [i] [j]”, “num_non_anchor_refs_l0 [ i] ”,“ non_anchor_ref_l0 [i] [j] ”,“ num_non_anchor_refs_l1 [i] ”,“ non_anchor_ref_l1 [i] [j] ”.

  The reference view number information decoding unit 404 includes viewpoint views that can be used as references for inter-view prediction in the reference picture list 0 and the reference picture list 1 for the anchor picture and the non-anchor picture of the respective viewpoints in the view dependency information. The syntax elements “num_anchor_refs_l0 [i]”, “num_anchor_refs_l1 [i]”, “num_non_anchor_refs_l0 [i]”, “num_non_anchor_refs_l1 [i]” indicating the numbers are decoded. However, as described above, in the present invention, the syntax element whose index i is “0” is not encoded in the supplied encoded bit string. The syntax element “0” is not decoded, and the syntax element whose index i is “1” or more is decoded.

  The reference view information decoding unit 406 is used for anchor pictures of numbers corresponding to “num_anchor_refs_l0 [i]”, “num_anchor_refs_l1 [i]”, “num_non_anchor_refs_l0 [i]”, “num_non_anchor_refs_l1 [i]”, and Syntax elements “anchor_ref_l0 [i] [j]” and “anchor_ref_l1 [] indicating the viewpoint ID of the viewpoint used as a reference for the j-th inter-view prediction in the reference picture list 0 for the non-anchor picture and the reference picture list 1 i] [j] ”,“ non_anchor_ref_l0 [i] [j] ”, and“ non_anchor_ref_l1 [i] [j] ”are decoded. However, as described above, in the present invention, since the syntax element in which i is “0” or “1” is not encoded in the supplied encoded bit string, the reference view information decoding unit 406 does not specify i. The syntax element of “0” or “1” is not decoded, and the syntax element of which i is “2” or more is decoded.

  In the reference viewpoint information generation unit 405, for the reason described above, the reference viewpoint information generation unit 405 indicates the number of reference viewpoints for inter-view prediction in the reference picture list 0 for the anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction. When the value of the tax element “num_anchor_refs_l0 [1]” is “1”, the 0th inter-view prediction in the reference picture list 0 for the anchor picture of the first view in the encoding / decoding order in the view direction is performed. The value of the syntax element “anchor_ref_l0 [1] [0]” indicating the viewpoint ID of the viewpoint used as a reference is “view_id [0]” that is the viewpoint ID of the zeroth viewpoint in the encoding / decoding order in the viewpoint direction. And

  Similarly, when the value of “num_anchor_refs_l1 [1]” is “1”, the value of “anchor_ref_l1 [1] [0]” is set to “view_id [0]”, and the value of “num_non_anchor_refs_l0 [1]” is “1” "", The value of "non_anchor_ref_l0 [1] [0]" is "view_id [0]", and the value of "num_non_anchor_refs_l1 [1]" is "1", the value of "non_anchor_ref_l1 [1] [0]" Is “view_id [0]”.

  Returning to FIG. 13, the description will be continued. The management information of the entire sequence decoded by the sequence information decoding unit 303 is supplied to the decoding management unit 302 and used for decoding management. The picture information decoding unit 304 decodes a coded bit string in which parameter information (PPS) related to coding of the picture separated by the separation unit 301 is coded, and decodes the decoded parameter information (PPS) as picture management information. The data is supplied to the management unit 302 and used for decoding management.

  The image signal decoding unit 305 is supplied from the separation unit 301 based on the management information and the picture management information of the entire decoded sequence including the number-of-views information, the decoding order information, and the view dependency information supplied from the decoding management unit 302. The encoded bit string (encoded data) to be decoded is decoded to obtain an image signal. When decoding an image signal, it may be decoded using inter-view prediction, and in that case, a reference image for inter-view prediction is also determined using the view-dependent information.

  Next, the multi-view image decoding processing procedure by the multi-view image decoding apparatus of the present embodiment shown in FIG. 13 will be described with reference to the flowcharts of FIGS. Since the processing operation of each step is the same as that described with reference to the block diagrams of FIGS. 13 and 14, only the processing procedure will be described here in association with FIGS.

  First, in step S201 in FIG. 15, the encoded bit string that has been encoded is separated into NAL unit units. In this step S201, the reception and separation processing procedure when transmitting the encoded bit string via the network will be described in detail with reference to the flowchart of FIG. In the separation process in step S201, first, an encoded bit string is received via the network (step S281), and subsequently, the MPEG-2 system method, MP4 file format, RTP, etc. used for the received encoded bit string are received. The packet header added based on the standard is decoded and removed (step S282). Then, the encoded bit string is separated in units of NAL units (step S283).

  Again, referring back to FIG. The identifier (nal_unit_type) for identifying the type of the NAL unit included in the header part of the NAL unit separated in step S201 in FIG. 15 is evaluated, and the NAL unit is parameter information (SPS) related to encoding of the entire sequence, that is, the sequence It is determined whether or not the information is information (step S202). If it is sequence information, the process proceeds to step S205. If it is determined that the information is not picture information (PPS) but sequence information (step S203), the process proceeds to step S206.

  If the NAL unit is neither sequence information nor picture information, the process proceeds to step S204. In step S204, it is determined whether the NAL unit is a VCL NAL unit, that is, a coding bit string in which a coding mode, a motion vector or a disparity vector, a coded residual signal, and the like are coded, and the VCL NAL unit. If YES, the process proceeds to step S207. The processes in steps S201, S202, S203, and S204 correspond to the processing operation in the separation unit 301 in the multi-viewpoint image decoding apparatus in FIG.

  Next, in step S205, the encoded bit string in which the parameter information related to the encoding of the entire sequence is encoded is decoded to obtain the parameter information related to the encoding of the entire sequence. The processing in step S205 corresponds to the decoding operation in the sequence information decoding unit 303 in the multi-view image encoding device in FIG.

  An example of the sequence information decoding process procedure of step S205 will be described in more detail with reference to the flowchart of FIG. In the sequence information decoding process, first, sequence information other than the MVC extension is decoded (step S211). The processing in step S211 corresponds to the decoding operation in the sequence information decoding unit 401 other than the MVC extension portion in FIG.

  Subsequent to step S211, the MVC extension portion of the information (SPS) related to the entire sequence is decoded according to the syntax structure shown in FIG. 11 (steps S212 to S214). First, information on the number of viewpoints encoded in the encoded bit string is decoded (step S212). The processing in step S212 corresponds to the decoding operation in the viewpoint number information decoding unit 402 in FIG. Subsequent to step S212, the viewpoint ID information of each viewpoint encoded in the decoding order in the viewpoint direction is decoded (step S213). The decoding process in step S213 corresponds to the decoding operation in the decoding order information decoding unit 403 in FIG.

  An example of the decoding process procedure of the viewpoint ID of each viewpoint encoded in the decoding order in the viewpoint direction in step S213 will be described in more detail with reference to the flowchart of FIG. In the decoding process in step S213, first, the index i indicating the encoding / decoding order in the viewpoint direction is set to “0” (step S221), and then whether or not the value of the index i is (number of viewpoints−1) or less. Judgment is made (step S222). When the value of index i is not less than (number of viewpoints−1), the decoding process in step S213 is terminated. When the value of index i is (number of viewpoints-1) or less, the process proceeds to step S223, and the processing from step S222 to step S224 is repeated until the value of index i is not less than (number of viewpoints-1).

  In step S223, the syntax element “view_id [i]” indicating the viewpoint ID of the i-th viewpoint in the encoding / decoding order in the viewpoint direction is decoded. Subsequently, in step S224, “1” is added to the index i, and the process proceeds again to step S222.

  Returning to the flowchart of FIG. Following the decoding process of the viewpoint ID information in step S213 described above with reference to FIG. 17, the viewpoint dependent information is decoded (step S214). The processing in step S214 corresponds to the decoding operation in the view dependency information decoding unit 408 including the reference view number information decoding unit 404, the reference view information generation unit 405, and the reference view information decoding unit 406 in FIG.

  An example of the process of decoding the viewpoint dependent information in step S214 will be described with reference to the flowchart of FIG. In step S214, first, the view dependency information of the anchor picture is decoded (step S231), and then the view dependency information of the non-anchor picture is decoded (step S232), thereby ending the decoding process.

  Next, an example of the decoding processing procedure of the viewpoint dependent information of the anchor picture in step S231 in FIG. 18 will be described in more detail with reference to the flowchart in FIG.

  In the decoding processing procedure of the anchor-picture view-dependent information in FIG. 19, the 0th viewpoint (viewpoint that is first encoded / decoded in the encoding / decoding order) of the encoded bit string in the encoding / decoding order in the view direction. The syntax elements “num_anchor_refs_l0 [0]” and “num_anchor_refs_l1 [0]” indicating the number of viewpoints that can be used as references for inter-view prediction in the reference picture list 0 and the reference picture list 1 for anchor pictures are encoded. The values are always set to “0” (step S241). Therefore, the index i indicating the encoding / decoding order in the viewpoint direction is set to “1” (step S242).

  Subsequently, it is determined whether or not the value of the index i is (the number of viewpoints−1) or less (step S243). If the value of the index i is not less than (number of viewpoints−1), the decoding process of the viewpoint dependent information of the anchor picture is terminated. When the value of index i is (number of viewpoints-1) or less, the process proceeds to step S244, and the processing from step S243 to step S260 is repeated until the value of index i is not less than (number of viewpoints-1).

  In step S244, the syntax element “num_anchor_refs_l0 [i] indicating the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 0 for the anchor picture of the i-th view in the encoding / decoding order in the view direction. "Is decrypted. Subsequently, in step S245, it is determined whether or not the index i is greater than “1”. When the index i is greater than “1”, the process proceeds to step S248. When the index i is not greater than “1”, that is, when the index i is “1”, the process proceeds to step S246. In step S246, it is determined whether or not the value of “num_anchor_refs_l0 [1]” is “1”.

  Here, “num_anchor_refs_l0 [1]” has a value of “0” or “1” as described above. When the value of “num_anchor_refs_l0 [1]” is “0”, the process proceeds to step S252, and when the value is “1”, the process proceeds to step S247. As described above, when the index i is “1”, the viewpoint used as the reference for inter-view prediction in the reference picture list 0 for the anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction is used for the encoded bit string. The syntax element indicating the viewpoint ID is not encoded. Therefore, in step S247, the viewpoint ID of the 0th viewpoint (viewpoint to be encoded / decoded first) in the encoding / decoding order in the viewpoint direction decoded in step S223 of FIG. 17, that is, the value of “view_id [0]” The value of “anchor_ref_l0 [1] [0]” is set, and the process proceeds to step S252.

  On the other hand, if it is determined in step S245 that the index i is greater than “1”, the index j is set to “0” (step S248), and then it is determined whether the value of the index j is smaller than “num_anchor_refs_l0 [i]”. (Step S249). When the value of the index j is “num_anchor_refs_l0 [i]” or more, the process proceeds to step S252. On the other hand, when the value of index j is smaller than “num_anchor_refs_l0 [i]”, the process proceeds to step S250, and the processing from step S249 to step S251 is repeated until the value of index j becomes equal to or greater than “num_anchor_refs_l0 [i]”.

  In step S250, the syntax element “anchor_ref_l0” indicating the viewpoint ID of the viewpoint used as a reference for the j-th inter-view prediction in the reference picture list 0 for the anchor picture of the i-th viewpoint in the encoding / decoding order in the viewpoint direction. [i] [j] ”is decoded, and the process proceeds to step S251. Subsequently, in step S251, “1” is added to the value of the index j, and the process proceeds again to step S249.

  In step S252, the syntax element “num_anchor_refs_l1 [i] indicating the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 1 for the anchor picture of the i-th view in the encoding / decoding order in the view direction. "Is decrypted. Subsequently, it is determined whether or not the index i is larger than “1” (step S253). When the index i is larger than “1”, the process proceeds to step S256. When the index i is not larger than “1”, that is, when the index i is “1”, the process proceeds to step S254.

  In step S254, it is determined whether or not the value of “num_anchor_refs_l1 [1]” is “1”. Here, “num_anchor_refs_l1 [1]” has a value of “0” or “1” as described above. When the value of “num_anchor_refs_l1 [1]” is “0”, the process proceeds to step S260, and when the value is “1”, the process proceeds to step S255. As described above, when the index i is “1”, the viewpoint used as the reference for inter-view prediction in the reference picture list 1 for the anchor picture of the first viewpoint in the encoding / decoding order in the viewpoint direction is used for the encoded bit string. The syntax element indicating the viewpoint ID is not encoded. In step S255, the viewpoint ID of the 0th viewpoint (viewpoint to be encoded / decoded first) in the encoding / decoding order in the viewpoint direction decoded in step S223 of FIG. 17, that is, the value of “view_id [0]” is set to “ The value of “anchor_ref_l1 [1] [0]” is set, and the process proceeds to step S260.

  On the other hand, if it is determined in step S253 that the value of index i is greater than “1”, after index j is set to “0” (step S256), whether or not the value of index j is smaller than “num_anchor_refs_l1 [i]” Is determined (step S257). When the value of the index j is “num_anchor_refs_l1 [i]” or more, the process proceeds to step S260. On the other hand, when the value of the index j is smaller than “num_anchor_refs_l1 [i]”, the process proceeds to step S258, and the processing from step S257 to step S259 is repeated until the value of the index j becomes “num_anchor_refs_l1 [i]” or more.

  In step S258, the syntax element “anchor_ref_l1” indicating the viewpoint ID of the viewpoint used as a reference for the j-th inter-view prediction in the reference picture list 1 for the anchor picture of the i-th viewpoint in the encoding / decoding order in the viewpoint direction. [i] [j] ”is decoded, and the process proceeds to step S259. In step S259, “1” is added to the value of index j, and the process proceeds again to step S257. In step S260, “1” is added to the value of index i, and the process proceeds again to step S243.

  Next, an example of the decoding processing procedure of the viewpoint dependent information of the non-anchor picture in step S232 in FIG. 18 will be described in more detail with reference to the flowchart in FIG.

  In the decoding process of the view dependent information of the non-anchor picture in FIG. 20, the reference bit list 0 and the reference picture list 1 for the non-anchor picture of the 0th view in the coding / decoding order in the view direction are coded into the coded bit string. The syntax elements “num_non_anchor_refs_l0 [0]” and “num_non_anchor_refs_l1 [0]” indicating the number of viewpoints that can be used as a reference for inter-view prediction are not encoded, and their values are always “0” (step S261). ). Therefore, the index i is set to “1” (step S262).

  Subsequently, it is determined whether or not the value of the index i is (the number of viewpoints−1) or less (step S263). If the value of index i is not less than (number of viewpoints −1), the decoding process of the viewpoint dependent information of the non-anchor picture is terminated. When the value of index i is (number of viewpoints-1) or less, the process proceeds to step S264, and the processing from step S263 to step S280 is repeated until the value of index i is not less than (number of viewpoints-1).

  In step S264, the syntax element “num_non_anchor_refs_l0 [i] indicating the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 0 for the non-anchor picture of the i-th view in the encoding / decoding order in the view direction. ] ". Subsequently, in step S265, it is determined whether or not the index i is greater than “1”. When the index i is greater than “1”, the process proceeds to step S268. When the index i is not greater than “1”, that is, when the index i is “1”, the process proceeds to step S266.

  In step S266, it is determined whether or not the value of “num_non_anchor_refs_l0 [1]” is “1”. Here, “num_non_anchor_refs_l0 [1]” has a value of “0” or “1” as described above. When the value of “num_non_anchor_refs_l0 [1]” is “0”, the process proceeds to step S272, and when the value is “1”, the process proceeds to step S267.

  As described above, when the index i is “1”, the coded bit string is used as a reference for inter-view prediction in the reference picture list 0 for the non-anchor picture of the first view in the coding / decoding order in the view direction. A syntax element indicating the viewpoint ID of the viewpoint is not encoded. Accordingly, in step S267, the viewpoint ID of the 0th viewpoint (viewpoint to be encoded / decoded first) in the encoding / decoding order in the viewpoint direction decoded in step S223 of FIG. 17, that is, the value of “view_id [0]” Is set to a value of “non_anchor_ref_l0 [1] [0]”, and the process proceeds to step S272.

  On the other hand, if it is determined in step S265 that the value of index i is greater than “1”, after index j is set to “0” (step S268), whether the value of index j is smaller than “num_non_anchor_refs_l0 [i]” Is determined (step S269). When the value of the index j is “num_non_anchor_refs_l0 [i]” or more, the process proceeds to step S272. On the other hand, when the value of the index j is smaller than “num_non_anchor_refs_l0 [i]”, the process proceeds to step S270, and the processing from step S269 to step S271 is repeated until the value of the index j becomes “num_non_anchor_refs_l0 [i]” or more.

  In step S270, the syntax element “indicating the viewpoint ID of the viewpoint used as the reference for the j-th inter-view prediction in the reference picture list 0 for the non-anchor picture of the i-th viewpoint in the encoding / decoding order in the viewpoint direction. Non_anchor_ref_l0 [i] [j] ”is decoded, and the process proceeds to step S271. Subsequently, in step S271, “1” is added to the value of the index j, and the process proceeds again to step S269.

  In step S272, the syntax element “num_non_anchor_refs_l1 [i] indicating the number of viewpoints that can be used as a reference for inter-view prediction in the reference picture list 1 for the non-anchor picture of the i-th view in the encoding / decoding order in the view direction. ] ". Subsequently, it is determined whether or not the index i is greater than 1 (step S273). When the index i is larger than “1”, the process proceeds to step S276, and when the index i is not larger than “1”, that is, when the index i is “1”, the process proceeds to step S274.

  In step S274, it is determined whether or not the value of “num_non_anchor_refs_l1 [1]” is “1”. Here, “num_non_anchor_refs_l1 [1]” has a value of “0” or “1” as described above. When the value of “num_non_anchor_refs_l1 [1]” is “0”, the process proceeds to step S280, and when the value is “1”, the process proceeds to step S275.

  As described above, when the index i is “1”, the coded bit string is used as a reference for inter-view prediction in the reference picture list 1 for the non-anchor picture of the first view in the coding / decoding order in the view direction. A syntax element indicating the viewpoint ID of the viewpoint is not encoded. Therefore, in step S275, the viewpoint ID of the 0th viewpoint (viewpoint to be encoded / decoded first) in the encoding / decoding order in the viewpoint direction decoded in step S223 of FIG. 17, that is, the value of “view_id [0]” Is set to a value of “non_anchor_ref_l1 [1] [0]”, and the process proceeds to step S280.

  On the other hand, if it is determined in step S273 that the value of index i is greater than “1”, after index j is set to “0” (step S276), whether or not the value of index j is smaller than “num_non_anchor_refs_l1 [i]” Is determined (step S277). If the value of the index j is equal to or greater than the value of “num_non_anchor_refs_l1 [i]”, the process proceeds to step S260. On the other hand, if the value of index j is smaller than the value of “num_non_anchor_refs_l1 [i]”, the process proceeds to step S278, and the processing from step S277 to step S279 is repeated until the value of index j becomes “num_non_anchor_refs_l1 [i]” or more. .

  In step S278, the syntax element indicating the viewpoint ID of the viewpoint used as the j-th inter-view prediction field reference in the reference picture list 1 for the non-anchor picture of the i-th viewpoint in the encoding / decoding order in the viewpoint direction. “Non_anchor_ref_l1 [i] [j]” is decoded, and the process proceeds to step S279. In step S279, “1” is added to the value of index j, and the process proceeds again to step S277. In step S280, “1” is added to the value of index i, and the process proceeds again to step S263.

  In the description of the processing procedures in FIGS. 19 and 20 above, the decoding processes in steps S244 and S252 in FIG. 19 and steps S264 and S272 in FIG. 20 correspond to the decoding operation of the reference viewpoint number information decoding unit 404 in FIG. To do. Further, the processes in steps S247 and S255 in FIG. 19 and steps S267 and S275 in FIG. 20 correspond to the derivation operation of the reference viewpoint information generation unit 405 in FIG. Furthermore, the decoding processes in steps S250 and S258 in FIG. 19 and steps S270 and S278 in FIG. 20 correspond to the decoding operation of the reference viewpoint information decoding unit 406 in FIG.

  Again, referring back to FIG. When the process of step S214 described above with reference to FIGS. 18 to 20 is completed, the sequence information encoding process of FIG. 16 is completed.

  Returning to the flowchart of FIG. When the decoding process of the sequence information in step S205 described above with reference to FIGS. 16 to 20 is completed, the process proceeds to step S208. On the other hand, in step S206, the parameter information related to the coding of the picture is decoded. The processing in step S206 corresponds to the decoding operation in the picture information decoding unit 304 of the multi-viewpoint image decoding apparatus in FIG. When the decoding process in step S206 is completed, the process proceeds to step S208. On the other hand, in step S207, the slice information and the image signal are decoded. The processing in step S207 corresponds to the decoding operation in the image signal decoding unit 305 in the multi-viewpoint image decoding apparatus in FIG. When the process of step S207 is completed, the process proceeds to step S208.

  In step S208, it is determined whether or not all decoding processes for the encoded bit string to be decoded have been completed. If completed, this multi-viewpoint image decoding processing procedure ends. If not completed, the process returns to the first step S201, and the processes from step S201 to step S208 are repeated until all the decoding processes of the encoded bit string to be decoded are completed.

  In the above description, reference of inter-view prediction in the reference picture list 0 and the reference picture list 1 for the anchor picture of the first viewpoint and the non-anchor picture in the encoding / decoding order in the viewpoint direction is referred to. As described above, the syntax element indicating the number of viewpoints that can be used as encoding is encoded on the encoding side and decoded on the decoding side, but only the viewpoint that precedes the encoding / decoding order in the viewpoint direction is referred to in inter-view prediction. If it is possible, only the 0th viewpoint can be referred to in the inter-view prediction of the first viewpoint in the encoding / decoding order in the viewpoint direction.

  Therefore, regarding the first viewpoint, the number of viewpoints that can be used as a reference for inter-view prediction takes a value of “0” or “1”. When the number of viewpoints that can be used as a reference for inter-view prediction is “0”, this represents encoding / decoding without referring to other viewpoints, and the number of viewpoints that can be used as a reference for inter-view prediction is “1”. In this case, it indicates that encoding / decoding can be performed with reference to another viewpoint. Therefore, regarding the first viewpoint, instead of information on the number of viewpoints that can be used as a reference for inter-view prediction, binary information indicating whether or not to reference another viewpoint, or a viewpoint that can be used as a reference for inter-view prediction It is possible to substitute binary information of “0” or “1” by encoding on the encoding side and decoding on the decoding side.

  In the above description, a multi-view image used for encoding and decoding can be encoded and decoded from a multi-view image actually captured from different viewpoints, but it is a virtual image that is not actually captured. It is also possible to encode and decode a viewpoint image that has been converted or generated, such as by interpolating the viewpoint position from surrounding viewpoints, and is included in the present invention.

  For example, a multi-viewpoint image signal including image signals of four viewpoints A, B, C, and D is (1) an image signal obtained by actually photographing all four viewpoint image signals at each viewpoint. In some cases, (2) when all four viewpoint image signals are virtually taken at each viewpoint, (3) A and B viewpoint image signals are actually captured at each viewpoint. The image signal obtained by actually shooting and the image signal obtained by actually shooting, such as the image signal obtained by virtually capturing the image signal of the C and D viewpoints and the image signals of the C and D viewpoints. Are assumed to be mixed with the generated image signal.

  In addition, multi-view images such as computer graphics can be encoded and decoded, and is included in the present invention. Furthermore, the above multi-view image encoding and decoding processes can be realized as a transmission, storage, and reception device using hardware, as well as a ROM (Read Only Memory) and a flash memory. It can also be realized by firmware stored in the computer or software such as a computer. The firmware program and software program can be provided by recording them on a computer-readable recording medium, provided from a server through a wired or wireless network, or provided as a data broadcast of terrestrial or satellite digital broadcasting. Is also possible.

It is a block diagram of an example of the multiview image coding apparatus which produces | generates the encoding bit sequence decoded by this invention. It is a block diagram of an example of the sequence information encoding part 102 which comprises the multiview image encoding apparatus in FIG. It is a flowchart for the multi-view image encoding process description of the encoding bit sequence decoded by this invention. FIG. 4 is a flowchart for explaining an encoding process of sequence information in step S <b> 101 in FIG. 3. FIG. FIG. 5 is a flowchart for explaining viewpoint ID encoding processing according to the encoding / decoding order in the viewpoint direction in step S113 in FIG. 4; FIG. 5 is a flowchart for explaining viewpoint-dependent information encoding processing in step S114 in FIG. 4; FIG. FIG. 7 is a flowchart for explaining processing for encoding viewpoint-dependent information of an anchor picture in step S131 in FIG. 6; FIG. FIG. 7 is a flowchart for explaining encoding processing of viewpoint-dependent information of a non-anchor picture in step S132 in FIG. It is a flowchart for packetization and transmission processing explanation in the case of transmitting via a network. It is a figure which shows an example of the reference dependence relationship between the images at the time of encoding the multiview image which consists of 8 viewpoints. It is a figure which shows an example of the syntax structure of the MVC extension part of SPS of the encoding bit stream decoded by this invention. It is an example of each syntax element and its value of the MVC extension part of SPS at the time of encoding by the reference dependence of prediction shown in FIG. 10 based on the syntax structure shown in FIG. It is a block diagram of one embodiment of a multi-view image decoding device of the present invention. It is a block diagram of one Embodiment of the sequence information decoding part 303 which comprises the multiview image decoding apparatus in FIG. It is a flowchart for multi-view image decoding processing description of this invention. 16 is a flowchart for explaining decoding processing of sequence information in step S205 in FIG. 15. 17 is a flowchart for explaining decoding processing of a viewpoint ID encoded in the encoding / decoding order in the viewpoint direction in step S213 in FIG. FIG. 17 is a flowchart for explaining decoding processing of viewpoint dependent information in step S214 in FIG. 16; FIG. 19 is a flowchart for explaining decoding processing of viewpoint dependent information of an anchor picture in step S231 in FIG. 18; FIG. FIG. 19 is a flowchart for describing decoding processing of viewpoint-dependent information of a non-anchor picture in step S232 in FIG. It is a flowchart for reception processing explanation in the case of receiving via a network. It is a figure which shows an example of the syntax structure of the MVC expansion part of SPS of a prior art example. It is a figure which shows an example of the relationship between the bit sequence and code number which were encoded with the unsigned exponential Golomb code. It is a figure which shows an example of each syntax element and its value of the MVC extension part of SPS at the time of encoding by the reference dependence relationship of prediction shown in FIG. 10 based on the syntax structure of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Coding management part 102 Sequence information coding part 103 Picture information coding part 104 Image signal coding part 105 Multiplexing part 201 Sequence information coding part other than MVC extension part 202 View number information coding part 203 Coding order information Coding section 204 Reference view number information coding section 205 Reference view information coding section 206 View dependent information coding section 207 SPS MVC extended partial coding section 301 Separating section 302 Decoding management section 303 Sequence information decoding section 304 Picture information decoding section 305 Image signal decoding unit 401 Sequence information decoding unit other than MVC extension portion 402 Viewpoint number information decoding unit 403 Decoding order information decoding unit 404 Reference view number information decoding unit 405 Reference viewpoint information generating unit 406 Reference view information decoding unit 407 Switch 408 Viewpoint Dependent information decoding unit 409 S PS MVC extended partial decoding unit

Claims (3)

It is a multi-viewpoint image signal including image signals of each viewpoint obtained respectively at a plurality of set viewpoints, and the image signal of one viewpoint is an image signal obtained by actually photographing from the one viewpoint, or A multi-view image decoding method for decoding encoded data to be decoded in an encoded bit string obtained by encoding a multi-view image signal which is an image signal generated as a virtual image taken from one viewpoint,
The encoded data to be decoded is
First encoded data obtained by encoding information for specifying an encoding / decoding order between the plurality of viewpoints of each viewpoint in encoding of the image signal of each viewpoint;
When the viewpoint that is first encoded in the encoding / decoding order among the plurality of viewpoints is the 0th viewpoint and the viewpoint that is encoded next is the 1st viewpoint, i is the ith (where i is a natural number) Information of the number of zero or more viewpoints that can be referred to in inter-view prediction that refers to decoded image signals of other viewpoints at the time of encoding the image signal of each viewpoint (note that 0 is referred to in inter-view prediction) There is no viewpoint that can be performed, and in this viewpoint, encoding is performed without using inter-view prediction.) Second encoded data obtained by encoding each of the first and subsequent viewpoints;
When the number of the viewpoints that can be referred to in inter-view prediction is one or more in the viewpoint that is encoded jth (where j is a natural number of 2 or more) in the encoding / decoding order, Information that identifies the viewpoint to be referred to in the prediction includes third encoded data obtained by encoding each viewpoint.
A first step of decoding the first encoded data to obtain information for specifying an encoding / decoding order among the plurality of viewpoints;
A second step of decoding the second encoded data to obtain information on the number of viewpoints that can be referred to in the inter-view prediction;
In the first viewpoint, when the number of viewpoints that can be referred to in the inter-view prediction obtained by decoding in the second step is 1, information for identifying the viewpoint to be referred to in the inter-view prediction is A third step of setting information for specifying the 0th viewpoint obtained from information for specifying an encoding / decoding order between the plurality of viewpoints decoded in the first step;
A fourth step of decoding the third encoded data for each of the viewpoints to obtain information for specifying a viewpoint to be referred to in the inter-view prediction. It is a multi-viewpoint image signal including image signals of each viewpoint obtained respectively at a plurality of set viewpoints, and the image signal of one viewpoint is an image signal obtained by actually photographing from the one viewpoint, or A multi-view image decoding device that decodes encoded data to be decoded in an encoded bit string obtained by encoding a multi-view image signal, which is an image signal generated as a virtual image taken from one viewpoint,
The encoded data to be decoded is
First encoded data obtained by encoding information for specifying an encoding / decoding order between the plurality of viewpoints of each viewpoint in encoding of the image signal of each viewpoint;
When the viewpoint that is first encoded in the encoding / decoding order among the plurality of viewpoints is the 0th viewpoint and the viewpoint that is encoded next is the 1st viewpoint, i is the ith (where i is a natural number) Information of the number of zero or more viewpoints that can be referred to in inter-view prediction that refers to decoded image signals of other viewpoints at the time of encoding the image signal of each viewpoint (note that 0 is referred to in inter-view prediction) There is no viewpoint that can be performed, and in this viewpoint, encoding is performed without using inter-view prediction.) Second encoded data obtained by encoding each of the first and subsequent viewpoints;
When the number of the viewpoints that can be referred to in inter-view prediction is one or more in the viewpoint that is encoded jth (where j is a natural number of 2 or more) in the encoding / decoding order, Information that identifies the viewpoint to be referred to in the prediction includes third encoded data obtained by encoding each viewpoint.
First decoding means for decoding the first encoded data to obtain information for specifying an encoding / decoding order between the plurality of viewpoints;
Second decoding means for decoding the second encoded data to obtain information on the number of viewpoints that can be referred to in the inter-view prediction;
In the first viewpoint, when the number of viewpoints that can be referred to in the inter-view prediction obtained by decoding in the second step is 1, information for identifying the viewpoint to be referred to in the inter-view prediction is Third decoding means for specifying the 0th viewpoint obtained from information specifying the encoding / decoding order between the plurality of viewpoints decoded in the first step;
A multi-viewpoint image decoding apparatus comprising: a fourth decoding unit that decodes the third encoded data for each of the viewpoints to obtain information for specifying a viewpoint to be referred to in the inter-viewpoint prediction. It is a multi-viewpoint image signal including image signals of each viewpoint obtained respectively at a plurality of set viewpoints, and the image signal of one viewpoint is an image signal obtained by actually photographing from the one viewpoint, or A multi-viewpoint image decoding program that causes a computer to decode encoded data to be decoded in an encoded bit string obtained by encoding a multi-viewpoint image signal that is an image signal that is virtually captured from one viewpoint. There,
The encoded data to be decoded is
First encoded data obtained by encoding information for specifying an encoding / decoding order between the plurality of viewpoints of each viewpoint in encoding of the image signal of each viewpoint;
When the viewpoint that is first encoded in the encoding / decoding order among the plurality of viewpoints is the 0th viewpoint and the viewpoint that is encoded next is the 1st viewpoint, i is the ith (where i is a natural number) Information of the number of zero or more viewpoints that can be referred to in inter-view prediction that refers to decoded image signals of other viewpoints at the time of encoding the image signal of each viewpoint (note that 0 is referred to in inter-view prediction) There is no viewpoint that can be performed, and in this viewpoint, encoding is performed without using inter-view prediction.) Second encoded data obtained by encoding each of the first and subsequent viewpoints;
When the number of the viewpoints that can be referred to in inter-view prediction is one or more in the viewpoint that is encoded jth (where j is a natural number of 2 or more) in the encoding / decoding order, Information that identifies the viewpoint to be referred to in the prediction includes third encoded data obtained by encoding each viewpoint.
In the computer,
A first step of decoding the first encoded data to obtain information for specifying an encoding / decoding order among the plurality of viewpoints;
A second step of decoding the second encoded data to obtain information on the number of viewpoints that can be referred to in the inter-view prediction;
In the first viewpoint, when the number of viewpoints that can be referred to in the inter-view prediction obtained by decoding in the second step is 1, information for identifying the viewpoint to be referred to in the inter-view prediction is A third step of setting information for specifying the 0th viewpoint obtained from information for specifying an encoding / decoding order between the plurality of viewpoints decoded in the first step;
And a fourth step of decoding the third encoded data for each viewpoint and obtaining information for specifying a viewpoint to be referred to in the inter-view prediction.

Images