JP2015122606A - Image coding apparatus, image coding method, image decoding apparatus, and image decoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve coding efficiency in coding of multi-viewpoint moving images.SOLUTION: An image coding apparatus 10 comprises: a storage unit 11 which stores therein plural moving images 21 and 22 respectively corresponding to viewpoints #1 and #2; and an arithmetic unit 12 which detects, from among images included in the moving image 22, a target image CurrPic having inter-viewpoint motion information mvCol to refer to an image ColPic at the same time included in the moving image 21, and codes the image area of the target image CurrPic by using motion information basemvColand basemvColof the image ColPic at the same time.

Description

  The present invention relates to an image encoding device, an image encoding method, an image decoding device, and an image decoding method.

  For encoding moving images, for example, H.264 recommended by ITU-T (International Telecommunication Union Telecommunication Standardization Sector). An encoding method called H.264 is widely used. In the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), the same technique is called MPEG-4 AVC (MPEG-4 Part 10 Advanced Video Coding). These names are collectively referred to as H.264. H.264 / AVC. H. The H.264 / AVC system achieves higher compression efficiency than the conventional MPEG-2 encoding system.

  H. For encoding of the H.264 / AVC format, a technique called inter-frame prediction (inter prediction) in which a current moving image frame is predicted with reference to different moving image frames is used. A method of predicting a current video frame with reference to a temporally previous video frame is called forward prediction. A method of predicting the current video frame with reference to a temporal video frame is called backward prediction. Furthermore, a method of predicting the current moving image frame with reference to a plurality of moving image frames before or after in time is called bi-directional prediction (Bi-directive Prediction).

  A moving image frame encoded without using inter-frame prediction is called an I (Intra-coded) picture. A moving picture frame encoded using only forward prediction is called a P (Predicted) picture. Furthermore, a moving picture frame that is encoded by selectively using any one of forward prediction, backward prediction, and bidirectional prediction is called a B (Bi-directional Predicted) picture.

  H. In the H.264 / AVC format, a prediction mode called a direct mode is defined as a device for improving compression efficiency. The direct mode can be applied to B pictures. There are two types of direct modes: temporal direct mode and spatial direct mode.

  The temporal direct mode is a prediction mode that uses a motion vector of a pixel block at the same position of an encoded picture when predicting a pixel block in a certain B picture. The spatial direct mode is a prediction mode that uses a motion vector of a pixel block located in the vicinity when predicting a pixel block in a B picture. In any direct mode, the amount of information can be reduced by the amount that it is not necessary to transmit motion vector information to the decoding side for the pixel block to be encoded.

  In recent years, an extension method called MVC (Multi-view Video Coding) (hereinafter referred to as MVC method) has been defined so that the above encoding method can be applied to multi-view video such as stereoscopic video. . In the MVC method, when encoding a pixel block of a moving image frame corresponding to a certain viewpoint, a prediction mode (hereinafter referred to as inter-view prediction) that refers to a pixel block of a moving image frame corresponding to another viewpoint is added. It was. However, in the MVC method, the temporal direct mode is not applied to pixel blocks that use inter-view prediction.

  In addition, regarding the MVC method, when the reference moving image frame is encoded using inter-view prediction, the motion vector set in the moving image frame of the other viewpoint is used to set the temporal direct mode to the reference moving image frame. Has been proposed (hereinafter, proposed method).

JP 2012-182616 A

  The application of the time direct mode described above is an effective method for effectively reducing the amount of information. For this reason, if the temporal direct mode can be applied to a B picture that is encoded using inter-view prediction, it is considered that it contributes to an improvement in encoding efficiency. Here, H. The time direct mode of the H.264 / AVC system has been described as an example. The same applies to the merge mode of the H.265 / HEVC (High Efficiency Video Coding) method.

  Therefore, according to one aspect, an object of the present invention is to provide an image encoding device, an image encoding method, an image decoding device, and an image decoding method capable of improving encoding efficiency in multi-view video encoding. It is to provide.

  According to one aspect of the present disclosure, a storage unit that stores a plurality of moving images respectively corresponding to a plurality of viewpoints, and the same time included in another moving image from among images included in one moving image An image encoding apparatus comprising: a calculation unit that detects a target image having inter-viewpoint motion information that refers to the image of the image and encodes an image region of the target image using the motion information of the image at the same time Provided.

  According to another aspect of the present disclosure, a computer that can acquire a moving image from a storage unit that stores a plurality of moving images respectively corresponding to a plurality of viewpoints is configured to store images of the images included in one moving image. A target image having inter-viewpoint motion information that refers to an image at the same time included in another moving image is detected, and an image area of the target image is encoded using the motion information of the image at the same time An image encoding method is provided.

  Further, according to another aspect of the present disclosure, in a storage unit that stores encoded data obtained by encoding a plurality of moving images respectively corresponding to a plurality of viewpoints, and in a process of decoding the encoded data Detecting a target image having inter-viewpoint motion information referring to an image at the same time included in another moving image from images included in one moving image, and using the motion information included in the image at the same time An image decoding device is provided that includes an arithmetic unit that decodes an image area of a target image.

  According to another aspect of the present disclosure, a computer capable of acquiring encoded data from a storage unit that stores encoded data obtained by encoding a plurality of moving images respectively corresponding to a plurality of viewpoints In the process of decoding the digitized data, a target image having inter-viewpoint motion information that refers to an image at the same time included in another moving image is detected from images included in one moving image, and the same time There is provided an image decoding method for decoding an image area of a target image using motion information of the image.

  According to the present invention, it is possible to improve encoding efficiency in multi-view video encoding.

It is the figure which showed an example of the image coding apparatus which concerns on 1st Embodiment. It is a figure for demonstrating time direct mode. It is a 1st figure for demonstrating the relationship between prediction between viewpoints and temporal direct mode. It is a 2nd figure for demonstrating the relationship between prediction between viewpoints, and the time direct mode. It is a 3rd figure for demonstrating the relationship between prediction between viewpoints, and the time direct mode. It is the figure which showed an example of the system which concerns on 2nd Embodiment. It is the figure which showed an example of the hardware which can implement | achieve the function which the encoding apparatus which concerns on 2nd Embodiment has. It is the 1st block diagram which showed an example of the function which the encoding apparatus which concerns on 2nd Embodiment has. It is the 2nd block diagram which showed an example of the function which the encoding apparatus which concerns on 2nd Embodiment has. It is the 3rd block diagram which showed an example of the function which the encoding apparatus which concerns on 2nd Embodiment has. It is a 1st figure for demonstrating the calculation method of the reference | standard vector which concerns on 2nd Embodiment. It is a 2nd figure for demonstrating the calculation method of the reference | standard vector which concerns on 2nd Embodiment. It is a 1st figure for demonstrating the correction method of the reference | standard vector which concerns on 2nd Embodiment. It is a 2nd figure for demonstrating the correction method of the reference | standard vector which concerns on 2nd Embodiment. It is the 1st block diagram showing an example of the function which the decoding device concerning a 2nd embodiment has. It is the 2nd block diagram showing an example of the function which the decoding device concerning a 2nd embodiment has. It is the 3rd block diagram showing an example of the function which the decoding device concerning a 2nd embodiment has. It is the 1st flowchart which showed the flow of the encoding process which concerns on 2nd Embodiment. It is the 2nd flowchart which showed the flow of the encoding process which concerns on 2nd Embodiment. It is the 3rd flowchart which showed the flow of the encoding process which concerns on 2nd Embodiment. It is a 1st figure for demonstrating the encoding method which concerns on 3rd Embodiment. It is a 2nd figure for demonstrating the encoding method which concerns on 3rd Embodiment.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. In addition, about the element which has the substantially same function in this specification and drawing, duplication description may be abbreviate | omitted by attaching | subjecting the same code | symbol.

<1. First Embodiment>
The first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of an image encoding device according to the first embodiment.

As illustrated in FIG. 1, the image encoding device 10 according to the first embodiment includes a storage unit 11 and a calculation unit 12.
The storage unit 11 is a volatile storage device such as a RAM (Random Access Memory) or a non-volatile storage device such as an HDD (Hard Disk Drive) or a flash memory. The arithmetic unit 12 is a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). However, the arithmetic unit 12 may be an electronic circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). For example, the calculation unit 12 executes a program stored in the storage unit 11 or another memory.

  The storage unit 11 stores a plurality of moving images 21 and 22 respectively corresponding to a plurality of viewpoints # 1 and # 2. In the example of FIG. 1, the case of two viewpoints is shown, but the same applies to the case where the number of viewpoints is three or more. Further, in the example of FIG. 1, the moving image 21 corresponding to the viewpoint # 1 is set to Base-View, and the moving image 22 corresponding to the viewpoint # 2 is set to non-Base-View. Hereinafter, the description will proceed along this example.

Base-View means that an image that performs inter prediction referring to an image of another viewpoint is not included. That is, when the images Pic ₁₀ ,..., Pic ₁₅ of the moving image 21 corresponding to the viewpoint # 1 are encoded and decoded, the images Pic ₂₀ ,..., Pic _{25 of} the moving image 22 corresponding to the viewpoint # 2 are not referred to. . Image Pic _10, ..., since the moving image 21 is encoded and decoded using only information in Pic _15, image Pic ₂₀ of the moving image 22 corresponding to the viewpoint # 2, ..., video without any information about Pic ₂₅ Image 21 can be encoded and decoded.

On the other hand, non-Base-View means that an image that performs inter prediction referring to an image of another viewpoint is included. That is, the image Pic ₂₀ of the moving image 22 corresponding to the viewpoint # 2, ..., in encoding and decoding Pic _25, image Pic ₁₀ of the moving image 21 corresponding to the viewpoint # 1, ..., at least one of Pic ₁₅ One is referenced. That is, information on at least one of the images Pic ₁₀ ,..., Pic ₁₅ of the moving image 21 corresponding to the viewpoint # 1 is used for encoding and decoding of the moving image 22.

The computing unit 12 detects a target image CurrPic having inter-viewpoint motion information mvCol referring to an image ColPic at the same time included in another moving image 21 from images included in one moving image 22. That is, the calculation unit 12 detects the target image CurrPic including an image region in which the prediction mode is set to inter-view prediction. In the example of FIG. 1, the image Pic ₂₄ is detected as the target image CurrPic. The image ColPic referenced by interview motion information mvCol is an image Pic _14.

The calculation unit 12 encodes the image area of the target image CurrPic using the motion information basemVCol ₀ and basemvCol ₁ included in the image ColPic at the same time as the target image CurrPic. The reference image ColPic is encoded before the target image CurrPic. Therefore, when encoding the target image CurrPic, the motion information basemvCol ₀ and basemvCol ₁ of the image ColPic can be used.

For example, the calculation unit 12 specifies one or a plurality of image areas associated with the image area of the target image CurrPic by using the inter-viewpoint movement information mvCol among the image areas included in the image ColPic. Then, the calculation unit 12 encodes the image region of the target image CurrPic based on the motion information basemvCol ₀ and basemvCol ₁ of the specified image region. In the example of FIG. 1, the motion information basemvCol _0, basemvCol motion information mvL _0, mvL ₁ based on ₁ is generated, the target image CurrPic is coded using the motion information mvL _0, mvL _1.

For example, an image Pic _12, time TBL ₀ between Pic _14, image Pic _22, when the time TDL ₀ between Pic ₂₄ are different, computing unit 12, by the ratio (tdL ₀ / _{tbL 0)} motion information BasemvCol ₀ Scale. Then, the calculation unit 12 sets the motion information after scaling (tdL ₀ / tbL ₀ ) * basemvCol ₀ as the motion information mvL ₀ . Similarly, the operation unit 12 scales the motion information basemvCol ₁ by the ratio (tdL ₁ / tbL ₁ ), and sets the scaled motion information (tdL ₁ / tbL ₁ ) * basemvCol ₁ as the motion information mvL ₁ .

As described above, the motion information mvL ₀ and mvL ₁ used for encoding the target image CurrPic is obtained from the motion information basemvCol ₀ and basemvCol _{1 of the} image ColPic pointed to by the inter-viewpoint motion information mvCol. Therefore, even if there is no motion information mvL _0, mvL ₁ of the target image CurrPic, if motion of the image ColPic information basemvCol _0, basemvCol ₁ is able to decode the target picture CurrPic. That is, it is possible to improve the encoding efficiency by the amount that does not include the motion information mvL ₀ and mvL ₁ in the decoding information (encoded data). Note that the above technique can also be applied when decoding a non-Base-View moving image 22, which contributes to an improvement in encoding efficiency.

The first embodiment has been described above.
<2. Second Embodiment>
Next, a second embodiment will be described. Here, for convenience of explanation, the description will be given by taking the MVC method as an example, but the scope of application of the technology according to the second embodiment is not limited to this.

[2-1. Time direct mode]
The time direct mode will be described with reference to FIG. FIG. 2 is a diagram for explaining the time direct mode.

  The temporal direct mode is a method of determining a CurrMB motion vector from a motion vector of a macroblock temporally adjacent to a coding target macroblock (hereinafter, CurrMB), paying attention to a sequence of moving images. When the temporal direct mode is applied, the temporal correlation of motion increases, and the coding efficiency is improved to the extent that motion vector information relating to CurrMB is not transmitted to the decoding side.

  H. The H.264 / AVC system bidirectional prediction is realized by combining List0 (L0) direction prediction and List1 (L1) direction prediction. The L0 direction is often set to the forward direction. Further, the L1 direction is often set to the backward direction. However, H. In the H.264 / AVC system bidirectional prediction, it is allowed to combine a plurality of different forward predictions or to combine a plurality of different backward predictions. Therefore, the prediction directions are expressed as L0 direction and L1 direction. Similar expressions may be used in this paper.

  The reference direction is expressed for each picture using a parameter called ref_idx. For example, ref_idx in the L0 direction is expressed as ref_idx_10. Further, ref_idx in the L1 direction is expressed as ref_idx_l1. In the temporal direct mode, a macroblock located at the same position as CurrMB is specified in the I picture or P picture having the smallest ref_idx in the L1 direction, and the motion vector of CurrMB is determined based on the motion vector pointed to by the macroblock. The A motion vector that is used as a reference when determining the CurrMB motion vector may be referred to as a reference vector.

  In the example of FIG. 2, CurrMB in CurrPic that is a B picture is a macroblock to be encoded in the temporal direct mode. In the temporal direct mode, when predicting CurrMB in a B picture, attention is paid to an already encoded picture (hereinafter referred to as ColPic), and a motion vector of a macroblock (hereinafter referred to as ColMB) located at the same position as CurrMB in ColPic. Use basemvCol. In the example of FIG. 2, the P picture (ref_idx = 0) located in the L1 direction is ColPic. The motion vector basemvCol is a reference vector. In addition, the picture indicated by the reference vector basemvCol may be referred to as RefPicCol.

In the temporal direct mode, a motion vector mvL ₀ from CurrPic to RefPicCol and a motion vector mvL ₁ from CurrPic to ColPic are calculated based on the reference vector basemvCol. For example, mvL ₀ and mvL ₁ are obtained by scaling the reference vector basemvCol based on the ratio of picture intervals (for example, the difference in POC (Picture Order Count)).

In the example of FIG. 2, the time between RefPicCol and ColPic is td, and the time between RefPicCol and CurrPic is tb. In this case, the motion vector (tb / td) * basemvCol obtained by scaling the reference vector basemvCol by the ratio (tb / td) is mvL ₀ . In addition, (mvL ₀ -basemvCol) becomes mvL ₁ . In the temporal direct mode, the motion vectors mvL ₀ and mvL ₁ thus obtained are used for CurrMB prediction. In this example, since the motion vectors mvL ₀ and mvL ₁ are obtained from the reference vector basemvCol, it is not necessary to transmit the motion information mvL ₀ and mvL ₁ to the decoding side.

  It should be noted that how to hold the motion vector used in the direct mode is determined by the value of direct_8 × 8_influence_flag of SPS (Sequence_parameter_set). When direct_8 × 8_influence_flag = 1, it has a motion vector used for the direct mode in units of 8 × 8. Further, when direct_8x8_influence_flag = 0, it has a motion vector used for the direct mode in units of 4x4.

The time direct mode has been described above.
[2-2. Inter-view prediction and temporal direct mode]
Next, the relationship between the inter-view prediction and the temporal direct mode will be described with reference to FIGS. The example of the temporal direct mode shown in FIG. 2 is a method in which the temporal direct mode is applied to encoding of a moving image corresponding to one viewpoint. Therefore, inter-view prediction is not considered in the example of FIG. Here, the relationship between inter-view prediction and temporal direct mode is considered.

  FIG. 3 is a first diagram for explaining the relationship between the inter-view prediction and the temporal direct mode. FIG. 4 is a second diagram for explaining the relationship between the inter-view prediction and the temporal direct mode. FIG. 5 is a third diagram for explaining the relationship between the inter-view prediction and the temporal direct mode.

(2-2-1. Inter-view prediction)
The MVC method relates to encoding of a multi-view video. In the MVC scheme, in addition to prediction modes such as L0 direction prediction and L1 direction prediction, a new prediction mode (inter-view prediction) that refers to a moving picture corresponding to another viewpoint is used. In inter-view prediction, a different view picture at the same time (same POC) as the picture to be encoded is referred to.

  Now consider the example of FIG. FIG. 3 illustrates a moving picture picture corresponding to viewpoint # 1 (Base-View) and a moving picture picture corresponding to viewpoint # 2 (non-Base-View).

  In the example of FIG. 3, the macroblock MB # 2 in the B picture Pic # 2 is a target for encoding based on inter-view prediction. Each picture of the moving picture corresponding to the viewpoint # 1 is encoded without referring to the picture of the moving picture corresponding to the viewpoint # 2. Now, it is assumed that the picture Pic # 1 corresponding to the viewpoint # 1 has already been encoded. In this case, the motion vector used for encoding the macroblock MB # 2 is the inter-viewpoint motion vector (iMV) indicating the macroblock MB # 1 included in the B picture Pic # 1 of the viewpoint # 1 (other viewpoint).

  As described above, in the inter-view prediction, a different view picture at the same time (same POC) as the picture to be encoded is referred to. Therefore, if the B picture Pic # 2 is CurrPic in the temporal direct mode and the B picture Pic # 1 pointed to by the inter-viewpoint motion vector iMV is ColPic, the POC difference between CurrPic and ColPic becomes zero. If the B picture Pic # 2 is ColPic, the B picture Pic # 1 becomes RefPicCol, and the difference in POC between RefPicCol and ColPic becomes zero. Therefore, the CurrPic motion vector cannot be obtained by scaling.

  H. H.264 standard (T-REC-H.264-200903-I! PDF-E.pdf). 7.4.3 describes that the direct mode applicable when there is a block that performs inter-view prediction in ColPic is limited to the spatial direct mode. However, the temporal direct mode that maintains temporal continuity of motion vectors including the preceding and succeeding pictures can be expected to improve coding efficiency more than the spatial direct mode that simply uses the motion vectors of surrounding blocks. . For these reasons, the second embodiment considers the relationship between inter-view prediction and temporal direct mode, and proposes a method that allows temporal direct mode to be applied to multi-view video encoding.

(2-2-2. When the reference block is an inter-view prediction block)
Please refer to FIG. The example of FIG. 4 illustrates a case where a ColPic macroblock at the same position as the CurrPic macroblock corresponding to the viewpoint # 2 refers to the picture corresponding to the viewpoint # 2 by the inter-view motion vector iMV. In this case, even if the inter-view motion vector iMV is scaled, a motion vector used for CurrPic encoding cannot be obtained.

Therefore, the present inventor has proposed a method of setting the motion vector of the block indicated by the inter-viewpoint motion vector iMV as the reference vector basemCol, as shown in FIG. According to this method, by scaling the reference vector basemvCol based on the interval td (POC difference) between the pictures I ₁₂ and P _{15 and} the interval tb (POC difference) between the pictures I ₂₂ and B ₂₄ , the motion vector of CurrPic Is obtained. Therefore, the time direct mode can be applied even in the case of FIG. In the second embodiment, encoding by the method shown in FIG. 4 is also considered.

(2-2-3. Reference block is another viewpoint block)
Please refer to FIG. The example of FIG. 5 shows a case where the CurrPic macroblock corresponding to the viewpoint # 2 refers to the picture corresponding to the viewpoint # 1 by the inter-view motion vector iMV. In this case, the length tb (POC difference) of the motion vector used for CurrPic inter prediction is zero. Therefore, the reference vector basemvCol cannot be scaled by the ratio of the times td and tb corresponding to the length of the reference vector basemvCol.

Therefore, in the second embodiment, the motion vector of the block indicated by the inter-view motion vector iMV is set as the reference vector basemvCol, and the CurrPic motion vector is obtained without using the time tb corresponding to the length of the inter-view motion vector iMV. Suggest a method. For example, in the example of FIG. 5, a method of scaling the reference vector basemvCol by the ratio of the time td corresponding to the length of the reference vector basemvCol and the time corresponding to the interval between the pictures I ₂₂ and B ₂₄ can be considered. According to this method, the time direct mode can be applied even in the case of FIG.

The relationship between the inter-view prediction and the temporal direct mode has been described above.
[2-3. system]
Next, a system according to the second embodiment will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of a system according to the second embodiment.

  As illustrated in FIG. 6, the system according to the second embodiment includes an encoding device 100 and a decoding device 200. The encoding apparatus 100 is an apparatus that encodes a plurality of moving images respectively corresponding to a plurality of viewpoints. Encoded data corresponding to a plurality of moving images encoded by the encoding device 100 is transmitted to the decoding device 200. For example, the encoded data is transmitted to the decoding device 200 via a network or a portable recording medium. The decoding device 200 decodes the encoded data acquired from the encoding device 100 to restore a plurality of moving images.

  In the following description, for convenience of explanation, description will be made on the assumption that a moving image of two viewpoints (Base-View, non-Base-View) is encoded and decoded. However, the scope of application of the technology according to the second embodiment is not limited to this, and the present invention can also be applied when the number of viewpoints is three or more.

The system has been described above.
[2-4. Hardware example]
Next, hardware capable of realizing the functions of the encoding apparatus 100 will be described with reference to FIG. FIG. 7 is a diagram illustrating an example of hardware capable of realizing the functions of the encoding apparatus according to the second embodiment.

  The functions of the encoding apparatus 100 can be realized using, for example, hardware resources of the information processing apparatus illustrated in FIG. That is, the functions of the encoding device 100 are realized by controlling the hardware shown in FIG. 7 using a computer program.

  As shown in FIG. 7, this hardware mainly includes a CPU 902, a ROM (Read Only Memory) 904, a RAM 906, a host bus 908, and a bridge 910. Further, this hardware includes an external bus 912, an interface 914, an input unit 916, an output unit 918, a storage unit 920, a drive 922, a connection port 924, and a communication unit 926.

  The CPU 902 functions as, for example, an arithmetic processing unit or a control unit, and controls the overall operation of each component or a part thereof based on various programs recorded in the ROM 904, the RAM 906, the storage unit 920, or the removable recording medium 928. . The ROM 904 is an example of a storage device that stores a program read by the CPU 902, data used for calculation, and the like. The RAM 906 temporarily or permanently stores, for example, a program read by the CPU 902 and various parameters that change when the program is executed.

  These elements are connected to each other via, for example, a host bus 908 capable of high-speed data transmission. On the other hand, the host bus 908 is connected to an external bus 912 having a relatively low data transmission speed via a bridge 910, for example. As the input unit 916, for example, a mouse, a keyboard, a touch panel, a touch pad, a button, a switch, a lever, or the like is used. Furthermore, as the input unit 916, a remote controller capable of transmitting a control signal using infrared rays or other radio waves may be used.

  As the output unit 918, for example, a display device such as a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), a PDP (Plasma Display Panel), or an ELD (Electro-Luminescence Display) is used. As the output unit 918, an audio output device such as a speaker or headphones, or a printer may be used. In other words, the output unit 918 is a device that can output information visually or audibly.

  The storage unit 920 is a device for storing various data. As the storage unit 920, for example, a magnetic storage device such as an HDD is used. Further, as the storage unit 920, a semiconductor storage device such as an SSD (Solid State Drive) or a RAM disk, an optical storage device, a magneto-optical storage device, or the like may be used.

  The drive 922 is a device that reads information recorded on a removable recording medium 928 that is a removable recording medium or writes information on the removable recording medium 928. As the removable recording medium 928, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is used.

  The connection port 924 is a port for connecting an external connection device 930 such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal. For example, a printer or the like is used as the external connection device 930.

  The communication unit 926 is a communication device for connecting to the network 932. As the communication unit 926, for example, a communication circuit for wired or wireless LAN (Local Area Network), a communication circuit for WUSB (Wireless USB), a communication circuit or router for optical communication, an ADSL (Asymmetric Digital Subscriber Line) Communication circuits, routers, communication circuits for mobile phone networks, and the like are used. A network 932 connected to the communication unit 926 is a wired or wireless network, and includes, for example, the Internet, a LAN, a broadcast network, a satellite communication line, and the like.

  Heretofore, the hardware capable of realizing the functions of the encoding device 100 has been described. Note that the functions of the decoding device 200 can also be realized using the hardware illustrated in FIG. That is, the hardware illustrated in FIG. 7 is an example of hardware that can realize the functions of the decoding device 200. Therefore, detailed description regarding hardware capable of realizing the functions of the decoding device 200 is omitted.

[2-5. Function of encoding apparatus]
Next, functions of the encoding device 100 will be described with reference to FIGS.

  FIG. 8 is a first block diagram illustrating an example of the functions of the encoding device according to the second embodiment. FIG. 9 is a second block diagram showing an example of the functions of the encoding apparatus according to the second embodiment. FIG. 10 is a third block diagram illustrating an example of functions of the encoding apparatus according to the second embodiment.

(2-5-1. Overall)
As illustrated in FIG. 8, the encoding device 100 includes a first viewpoint encoding unit 101, a statistical information acquisition unit 102, and a second viewpoint encoding unit 103. Note that in the encoding device 100, the functions of the first viewpoint encoding unit 101, the statistical information acquisition unit 102, and the second viewpoint encoding unit 103 can be realized using the above-described CPU 902 or the like.

  The first viewpoint encoding unit 101 receives a moving picture corresponding to Base-View (hereinafter referred to as Base-View picture). The first viewpoint encoding unit 101 encodes the input Base-View picture. The function of the first viewpoint encoding unit 101 will be described in detail later.

  The statistical information acquisition unit 102 acquires statistical information obtained when the first viewpoint encoding unit 101 encodes the Base-View picture. The statistical information includes, for example, a quantization scale value (hereinafter referred to as qP value) and the number of difference coefficients. The statistical information acquired by the statistical information acquisition unit 102 is input to the second viewpoint encoding unit 103.

  The second viewpoint encoding unit 103 receives a moving picture corresponding to non-Base-View (hereinafter, non-Base-View picture). The second viewpoint encoding unit 103 encodes the non-Base-View picture using the motion vector and the information about the picture used by the first viewpoint encoding unit 101 and the statistical information. The function of the second viewpoint encoding unit 103 will be described in detail later.

(2-5-2. Details of First View Coding Unit 101)
Here, the function of the first viewpoint encoding unit 101 will be further described with reference to FIG.

  As shown in FIG. 9, the first viewpoint encoding unit 101 includes a subtractor 111, an orthogonal transformation / quantization unit 112, a variable length coding unit 113, an inverse orthogonal transformation / inverse quantization unit 114, and an adder 115. Have. Further, the first viewpoint encoding unit 101 includes a frame memory 116, a motion compensation unit 117, a motion vector detection unit 118, a motion vector memory 119, a reference vector acquisition unit 120, a direct vector calculation unit 121, and a mode determination unit 122. .

  The subtractor 111 receives macroblock data (hereinafter referred to as MB data) obtained by dividing the Base-View picture into blocks (MB) of 16 × 16 pixels (pixels). The subtractor 111 subtracts the MB data of the predicted image output from the motion compensation unit 117 from the input MB data to generate prediction error data. Prediction error data generated by the subtractor 111 is input to the orthogonal transform / quantization unit 112.

  The orthogonal transform / quantization unit 112 orthogonally transforms the input prediction error data in units of 8 × 8 pixel blocks or 4 × 4 pixel blocks. Examples of applicable orthogonal transforms include DCT (Discrete Cosine Transform) transform and Hadamard transform. Through such orthogonal transformation, the prediction error data is converted into data of a horizontal frequency component and a vertical frequency component.

  The orthogonal transform / quantization unit 112 quantizes the frequency component data obtained by the orthogonal transform to generate quantized data. The quantized data generated by the orthogonal transform / quantization unit 112 is input to the variable length coding unit 113 and the inverse orthogonal transform / inverse quantization unit 114.

  The variable length encoding unit 113 performs variable length encoding on the quantized data to generate encoded data. The variable length coding is a coding method that assigns a variable length code according to the appearance frequency of a symbol. For example, the variable length coding unit 113 assigns a short code to a combination of coefficients having a high appearance frequency, and assigns a long code to a combination of coefficients having a low appearance frequency. Examples of applicable variable length coding include CAVLC (Context-Adaptive Variable Length Coding) and CABAC (Context-Adaptive Binary Arithmetic Coding).

  The inverse orthogonal transform / inverse quantization unit 114 dequantizes the quantized data to restore the frequency component data. The inverse orthogonal transform / inverse quantization unit 114 performs inverse orthogonal transform on the frequency component data to restore the prediction error data. The restored prediction error data is input to the adder 115. The adder 115 adds the MB data generated by the motion compensation unit 117 by motion compensation and the prediction error data restored by the inverse orthogonal transform / inverse quantization unit 114 to generate a locally decoded image.

  The locally decoded image generated by the adder 115 is input to the frame memory 116. A deblocking filter may be applied to the locally decoded image. The frame memory 116 stores the locally decoded image input by the adder 115. The locally decoded image stored in the frame memory 116 is referred to by the motion compensation unit 117. In addition, the locally decoded image stored in the frame memory 116 is used by the second viewpoint encoding unit 103 to encode a non-Base-View picture.

  The motion compensation unit 117 performs motion compensation of the locally decoded image stored in the frame memory 116 using the motion vector stored in the motion vector memory 119, and performs MB data after motion compensation (MB of the reference picture). Data). The motion vector detection unit 118 performs motion search using the input MB data of the Base-View picture (MB data of the target picture) and the MB data of the reference picture, and generates a motion vector.

  As a motion search method, for example, there is a block matching method. Note that the magnitude of the sum of absolute differences of pixels is used for motion search. For example, the evaluation value cost used for motion compensation can be expressed as the following equation (1) using the difference absolute value SAD_cost and the evaluation value MV_cost corresponding to the code amount of the motion vector. Note that SAD is an abbreviation for Sum Absolute Difference. MV is an abbreviation for Motion Vector.

cost = SAD_cost + MV_cost
... (1)
The motion vector detection unit 118 detects, for example, a motion vector that minimizes the evaluation value cost. The motion vector detected by the motion vector detection unit 118 is input to the motion vector memory 119 and the mode determination unit 122. The motion vector memory 119 stores a motion vector detected by the motion vector detection unit 118.

  The reference vector acquisition unit 120 determines whether or not the encoding target macroblock (CurrMB) is a macroblock using bi-directional prediction. When the CurrMB is a macroblock using bi-directional prediction, the reference vector acquisition unit 120 specifies a ColPic macroblock (ColMB) that is in the same position as the CurrMB, and acquires a motion vector of the ColMB from the motion vector memory 119 ( (See FIG. 2). The motion vector acquired by the reference vector acquisition unit 120 is input to the direct vector calculation unit 121 as a reference vector basemvCol.

The direct vector calculation unit 121 scales the reference vector basemCol with time distribution to generate a CurrMB motion vector (direct vector). For example, in the example of FIG. 2, the reference vector basemvCol is scaled based on the ratio of the times tb and td, and the direct vectors mvL ₀ and mvL ₁ are generated. The direct vector generated by the direct vector calculation unit 121 in this way is input to the mode determination unit 122.

  The mode determination unit 122 selects a prediction mode with the lowest coding cost among the five prediction modes (intra prediction mode, forward prediction mode, backward prediction mode, bidirectional prediction mode, and direct mode). For example, the mode determination unit 122 calculates the encoding cost shown in the following equations (2) to (6) for each prediction mode.

  cost_direct is the encoding cost of the direct mode. cost_forward is the coding cost of the forward prediction mode. cost_backward is the encoding cost of the backward prediction mode. cost_biddirection is the encoding cost of the bidirectional prediction mode. cost_intra is the coding cost of the intra prediction mode. * Org represents MB data of the target picture. * Ref represents the MB data of the reference picture. * Mv is a motion vector candidate used by CurrMB. * Prevmv represents a prediction vector based on a macroblock located around CurrMB.

cost_direct = SAD (* org, * ref)
... (2)
cost_forward = SAD (* org, * ref) + MV_COST (* mv, * prevmv)
... (3)
cost_backward = SAD (* org, * ref) + MV_COST (* mv, * prevmv)
... (4)
cost_bidirection = SAD (* org, * ref) + MV_COST (* mv, * prevmv)
... (5)
cost_intra = ACT (* org)
(6)
SAD (·) is a function for calculating the sum of pixel difference absolute values as shown in the following equation (7). H. In the H.264 / AVC format, one macro block can be divided into a plurality of sub blocks. For example, when one macroblock is divided into four 8 × 8 sub-blocks, four sums of absolute differences calculated for each sub-block are the calculation results of SAD (•). The same applies when the unit of the sub-block is 8 × 16 pixels, 16 × 8 pixels, 4 × 8 pixels, 8 × 4 pixels, and 4 × 4 pixels.

SAD (* org, * ref) = Σ | * org− * ref |
... (7)
MV_COST (·) is a function for calculating an evaluation value proportional to the code amount of the motion vector. MV_COST (•) is expressed as in the following Expression (8). Note that λ is a weighting constant for adjusting the degree of influence on the coding cost. Table [·] is a table for converting the magnitude of the vector difference into the code amount. These are preset.

MV_COST = λ * (Table [* mv− * prevmv])
... (8)
As shown in the above equation (6), the coding cost of the intra prediction mode is expressed by a function ACT (•) called an activity. In the intra prediction mode, CurrMB itself is orthogonally transformed. Therefore, the encoding cost is obtained using the degree (activity) that the pixel value of each pixel included in CurrMB is away from the average value (AveMB). The function ACT (·) for obtaining the activity is expressed by the following equation (9).

ACT (* org, * ref) = Σ | * org−AveMB |
... (9)
As is clear from the above equations (2) to (6), MV_COST is not included in the coding cost of the direct mode. Therefore, when the prediction is correct and the SAD is equal and relatively small in all prediction modes, the encoding cost of the direct mode is reduced and the direct mode is easily selected.

  Now, the mode determination part 122 which calculated the encoding cost for every prediction mode with the above methods selects the prediction mode with the minimum encoding cost. The mode determination unit 122 also stores a motion vector used for encoding in the selected prediction mode in the motion vector memory 119. Furthermore, the mode determination unit 122 notifies the motion compensation unit 117 of the selected prediction mode.

As described above, the coding of the Base-View picture is performed independently of the coding of the non-Base-View picture.
(2-5-3. Details of Second View Encoding Unit 103)
Here, the function of the second viewpoint encoding unit 103 will be further described with reference to FIG.

  As shown in FIG. 10, the second viewpoint encoding unit 103 includes a subtractor 131, an orthogonal transform / quantization unit 132, a variable length encoding unit 133, an inverse orthogonal transform / inverse quantization unit 134, and an adder 135. Have. The second viewpoint encoding unit 103 includes a frame memory 136, a motion compensation unit 137, a motion vector detection unit 138, and a motion vector memory 139. Furthermore, the second viewpoint encoding unit 103 includes a reference vector acquisition unit 140, a reference vector correction unit 141, a direct vector calculation unit 142, and a mode determination unit 143.

  The subtracter 131 receives MB data of a non-Base-View picture. Similar to the first viewpoint encoding unit 101, the subtractor 131 subtracts the MB data of the prediction image output from the motion compensation unit 137 from the input MB data to generate prediction error data. The prediction error data generated by the subtracter 131 is input to the orthogonal transform / quantization unit 132.

  The orthogonal transform / quantization unit 132 orthogonally transforms the input prediction error data in units of 8 × 8 pixel blocks or 4 × 4 pixel blocks. Examples of applicable orthogonal transform include DCT transform and Hadamard transform. Through such orthogonal transformation, the prediction error data is converted into data of a horizontal frequency component and a vertical frequency component.

  The orthogonal transform / quantization unit 132 quantizes the frequency component data obtained by the orthogonal transform to generate quantized data. The quantized data generated by the orthogonal transform / quantization unit 132 is input to the variable length coding unit 133 and the inverse orthogonal transform / inverse quantization unit 134. The variable length encoding unit 133 performs variable length encoding on the quantized data to generate encoded data. Examples of applicable variable length coding include CAVLC and CABAC.

  The inverse orthogonal transform / inverse quantization unit 134 dequantizes the quantized data to restore the frequency component data. The inverse orthogonal transform / inverse quantization unit 134 performs inverse orthogonal transform on the frequency component data to restore the prediction error data. The restored prediction error data is input to the adder 135. The adder 135 adds the MB data generated by the motion compensation unit 137 by motion compensation and the prediction error data restored by the inverse orthogonal transform / inverse quantization unit 134 to generate a locally decoded image.

  The locally decoded image generated by the adder 135 is input to the frame memory 136 and the motion compensation unit 137. A deblocking filter may be applied to the locally decoded image. The frame memory 136 stores the locally decoded image input by the adder 135. The locally decoded image stored in the frame memory 136 is referred to by the motion compensation unit 137.

  The motion compensation unit 137 performs motion compensation of the locally decoded image stored in the frame memory 136 using the motion vector stored in the motion vector memory 139, and performs MB data after motion compensation (MB of the reference picture). Data). The motion vector detection unit 138 performs motion search using the input MB data of the non-Base-View picture (MB data of the target picture) and the MB data of the reference picture, and generates a motion vector.

  Note that the motion vector detection unit 138 acquires a reference picture from the frame memory 116 of the first viewpoint encoding unit 101 when a reference picture of another viewpoint (Base-View in this example) can be used. Then, the motion vector detection unit 138 performs a motion search using the acquired MB data of the reference picture to generate a motion vector.

  The motion vector detected by the motion vector detection unit 138 is input to the motion vector memory 139 and the mode determination unit 143. The motion vector memory 139 stores a motion vector detected by the motion vector detection unit 138.

  The reference vector acquisition unit 140 determines whether or not the encoding target macroblock (CurrMB) is a macroblock using bidirectional prediction. When CurrMB is a macroblock using bi-directional prediction, reference vector acquisition section 140 determines whether or not the motion vector used for CurrMB prediction is inter-viewpoint motion vector iMV.

  When the motion vector used for CurrMB prediction is the inter-viewpoint motion vector iMV, the reference vector acquisition unit 140 identifies a ColPic macroblock (ColMB) located at the same position as the CurrMB. Then, the reference vector acquisition unit 140 acquires a ColMB motion vector from the motion vector memory 119 of the first viewpoint encoding unit 101 (see FIG. 5). Note that ColPic is a Base-View picture at the same time (same POC) as CurrPic. In this case, the motion vector acquired by the reference vector acquisition unit 140 is input to the reference vector correction unit 141.

  On the other hand, when the motion vector used for CurrMB prediction is not the inter-viewpoint motion vector iMV, the reference vector acquisition unit 140 identifies a ColPic macroblock (ColMB) located at the same position as CurrMB. In this case, ColPic is a non-Base-View picture. In addition, the reference vector acquisition unit 140 determines whether or not the motion vector used for the prediction of ColMB is the inter-viewpoint motion vector iMV.

  When the motion vector used for the prediction of ColMB is the inter-view motion vector iMV, the reference vector acquisition unit 140 specifies the macro block (RefMB) of the Base-View picture indicated by the inter-view motion vector iMV used for the prediction of ColMB. Then, the reference vector acquisition unit 140 acquires the RefMB motion vector from the motion vector memory 119 of the first viewpoint encoding unit 101 (see FIG. 4). In this case, the motion vector acquired by the reference vector acquisition unit 140 is input to the reference vector correction unit 141.

  On the other hand, when the motion vector used for the prediction of ColMB is not the inter-viewpoint motion vector iMV, the reference vector acquisition unit 140 identifies a ColPic macroblock (ColMB) at the same position as CurrMB, and stores the motion vector of ColMB in the motion vector memory. 139 (see FIG. 2). In this case, the motion vector acquired by the reference vector acquisition unit 140 is input to the direct vector calculation unit 141 as the reference vector basemvCol.

  The reference vector correction unit 141 corrects the motion vector input from the reference vector acquisition unit 140 based on the statistical information input from the statistical information acquisition unit 102 to generate a reference vector basemvCol. A correction method performed when generating the reference vector basemvCol will be described later. The reference vector basemCol generated by the reference vector correction unit 141 is input to the direct vector calculation unit 142.

  The direct vector calculation unit 142 scales the reference vector basemCol with time distribution, and generates a CurrMB motion vector (direct vector). The direct vector generated by the direct vector calculation unit 142 is input to the mode determination unit 143.

  The mode determination unit 143 selects a prediction mode with the lowest coding cost among the five prediction modes (intra prediction mode, forward prediction mode, backward prediction mode, bidirectional prediction mode, and direct mode). Further, the mode determination unit 143 stores a motion vector used for encoding in the selected prediction mode in the motion vector memory 139. Furthermore, the mode determination unit 143 notifies the motion compensation unit 137 of the selected prediction mode.

As described above, the encoding of the non-Base-View picture is performed by appropriately using the information used for encoding the Base-View picture.
(2-5-4. Supplementary explanation on calculation of reference vector)
Here, with reference to FIGS. 11 to 14, a supplementary explanation will be given for the calculation of the reference vector basemvCol when the motion vector of CurrMB is the inter-viewpoint motion vector iMV. Further, the explanation will be supplemented for the calculation of the direct vector.

  FIG. 11 is a first diagram for explaining a reference vector calculation method according to the second embodiment. FIG. 12 is a second diagram for explaining the reference vector calculation method according to the second embodiment. FIG. 13 is a first diagram for explaining a reference vector correction method according to the second embodiment. FIG. 14 is a second diagram for explaining the reference vector correction method according to the second embodiment.

(Calculation of reference vector and direct vector)
Please refer to FIG. In the example of FIG. 11, the picture B24 of viewpoint # 2 (non-Base-View) is CurrPic including a macroblock (CurrMB) to be encoded. The motion vector used for CurrMB prediction is the inter-viewpoint motion vector iMV. In this case, the picture B14 at the viewpoint # 1 (Base-View) is ColPic.

  CurrPic and ColPic are pictures with high similarity. However, since the shooting position and angle are different, CurrPic and ColPic are not completely the same picture. Therefore, the inter-viewpoint motion vector iMV has a finite length. The block iMB referred to by the inter-viewpoint motion vector iMV is ColMB.

  When CurrPic is a B picture, ColPic at the same time (same POC) as CurrPic is a B picture. Therefore, as shown in FIG. 11, motion vectors in the L0 direction and the L1 direction are used for prediction of ColMB.

The reference vector correction unit 141 sets the motion vector in the L0 direction acquired by the reference vector acquisition unit 140 as the reference vector basemvCol ₀ and sets the motion vector in the L1 direction as the reference vector basemvCol ₁ . At this time, the reference vector correction unit 141 corrects the reference vectors basemvCol ₀ and basemvCol ₁ using the inter-viewpoint motion vector iMV. This correction method will be described later.

When the reference vector basemvCol _0, basemvCol ₁ is obtained, the direct vector calculation unit 142 calculates a direct vector mvL _0, mvL ₁ by scaling the reference vector basemvCol _0, basemvCol _1.

In the example of FIG. 11, the reference vector basemvCol ₀ is scaled using the ratio of the interval tbL ₀ (POC difference) between the pictures I ₁₂ and B _{14 and} the interval tdL ₀ (POC difference) between the pictures I ₂₂ and B _24. . In addition, the reference vector basemvCol ₁ is scaled using the ratio of the interval tbL ₁ (POC difference) between the pictures B ₁₄ and P _{15 and} the interval tdL ₁ (POC difference) between the pictures B ₂₄ and P ₂₅ .

That is, the spacing TBL ₀ of ref_idx_l0 and minimum picture and ColPic in intervals tdL _0, Base-View of ref_idx_l0 in non Base-View is the smallest picture and CurrPic are used for scaling. Further, the interval TBL ₁ of ref_idx_l1 is ref_idx_l1 is the interval tdL _1, Base-View of the minimum picture and CurrPic the smallest picture and ColPic in non Base-View is used for scaling. In this case, the direct vectors mvL ₀ and mvL ₁ are obtained by the following equations (10) and (11).

mvL ₀ = basemvCol ₀ · (tbL ₀ / tdL ₀ )
(10)
mvL ₁ = basemvCol ₁ (tbL ₁ / tdL ₁ )
... (11)
As described above, by scaling with _{tbL 0, tdL 0, tbL 1} , tdL 1, it is possible to obtain a direct vector from the reference vector even if other viewpoint picture when predicting the CurrMB is referred to. That is, the temporal direct mode can be applied even when CurrMB uses inter-viewpoint reference. Various modification examples such as a method of selectively using one reference vector and a method of selecting and using one or a plurality of direct vectors are conceivable.

Example of FIG. 12, the direct vector mvL _0, mvL ₁ from the reference vector basemvCol _0, basemvCol ₁ calculates, shows how to select and use one of the direct vector. The direct vector mvL ₀ is obtained by scaling the reference vector basemvCol ₀ using the picture intervals tbL ₀ and td. On the other hand, the direct vector mvL ₁ is obtained by inverting the reference vector basemvCol ₁ in the L0 direction (mvL ₁ = −basemvCol ₁ ). The selection of the direct vector is performed based on the encoding cost, for example.

In the example of FIG. 12, the direct vector indicating the picture B ₂₃ is calculated. However, the information of the picture indicated by the direct vector can be specified by SEI (Supplemental Enhancement Information), for example. SEI is an example of header information. In the example of FIG. 12, a direct vector indicating a picture of ref_idx_I0 = 2 is specified.

By applying the above method, a direct vector can be obtained from a reference vector. The reference vector correction method will be described below.
(Reference vector correction # 1: Maximum area)
A first correction method will be described.

  Please refer to FIG. As already described, pictures with different viewpoints are not completely the same because of differences in shooting position and angle. Therefore, the inter-viewpoint motion vector iMV has a finite length. That is, there is a high possibility that the block iMB pointed to by the inter-viewpoint motion vector iMV does not completely match any of the macroblocks included in the ColPic. For example, consider the case where the ColPic macroblocks MB # 1,..., # 4 and the block iMB pointed to by the inter-viewpoint motion vector iMV are in the positional relationship illustrated in FIG. Macro blocks MB # 1,..., # 4 are examples of peripheral blocks.

Assume that the sizes of the macroblocks MB # 1,..., # 4, and the block iMB are N × M. In the example of FIG. 13, the area where the macro blocks MB # 1, # 2 and the block iMB overlap is zero. Furthermore, the overlapping area and the macro-block MB # 3 and the block iMB is _{num y × (N-num x} ). Then, the overlapping area and the macro-block MB # 4 and the block iMB is num _y × num _x. In the example of FIG. 13, num _x is larger than (N-num _x ). Therefore, the overlapping area between the macro block MB # 4 and the block iMB is maximized.

  The first correction method is a method of obtaining the macro block MB # 4 having the maximum overlap area and setting a motion vector used for prediction of the macro block MB # 4 having the maximum overlap area as the reference vector basemvCol. That is, when the first correction method is employed, the reference vector correction unit 141 calculates the overlap area between the block iMB and the peripheral blocks, and uses the motion vector used for prediction of the macroblock with the maximum overlap area as the reference vector basemvCol. Set to. According to the first correction method, the reference vector basemvCol can be set in consideration of the inter-viewpoint motion vector iMV while suppressing an increase in calculation load.

(Reference vector correction # 2: weighted average)
A second correction method will be described. Here, consider the case where the ColPic macroblocks MB # 1,..., # 4 and the block iMB pointed to by the inter-viewpoint motion vector iMV are in the positional relationship illustrated in FIG. Also, it is assumed that the sizes of the macro blocks MB # 1,... In the second correction method, the motion vectors of the macroblocks MB # 1,..., # 4 located around the block iMB are weighted and averaged, and the vector obtained by the weighted average is set as the reference vector basemvCol. Hereinafter, two examples (Example 1 and Example 2) of the calculation method of the weighted average are shown.

(Example 1: Calculation of weighted average based on overlapping area)
In the example of FIG. 14, the area S ₁ where the macro block MB # 1 and the block iMB overlap is dx _A × dy _A. The area S ₂ where the macro block MB # 2 and the block iMB overlap is dx _B × dy _A. The area S ₃ where the macro block MB # 3 and the block iMB overlap is dx _A × dy _B. The area S ₄ where the macro block MB # 4 and the block iMB overlap is dx _B × dy _B. Note that dx _A , dx _B , dy _A , and dy _B are expressed in units of the number of pixels.

In Example 1, the evaluation value Costxy used for calculating the weighted average is calculated based on the overlapping areas S ₁ ,..., S ₄ . The evaluation value Costxy is given by the following equation (12). Here, K is a weighting factor set in advance. Further, the reference vector basemvCol is calculated by calculating a weighted average using the evaluation value Costxy as a weight. For example, the reference vector basemvCol is calculated according to the following equation (13). Here, basemvCol _x and basemvCol _y are the x component and the y component of the reference vector basemvCol, respectively. Further, MVx _i and MVy _i are the x component and y component of the motion vector used for prediction of the macroblock MB # i, respectively.

Costxy _i = K × S _i (i = 1,..., 4)
(12)

(Example 2: Calculation of weighted average based on overlapping area and quantized value)
In Example 2, the evaluation value Costxy used for the calculation of the weighted average is the above-described overlapping area S ₁ ,..., S ₄ , the quantization value (quantization scale value: qP value) used for encoding, and the number of effective coefficients. Calculated based on. The qP value and the number of effective coefficients are acquired by the statistical information acquisition unit 102. If the number of effective coefficients used for encoding and the absolute values of the components are small, it can be determined that the reliability of the prediction result is high. Moreover, if the effective coefficient is comparable, it can be judged that the reliability of a prediction result is so high that the used qP value is small. Therefore, in Example 2, these values are used.

The evaluation value Costxy is given by the following equation (14). Here, K is a weighting factor set in advance. qP _i is a qP value used for encoding the macroblock MB # i. numCoef _i is the number of effective coefficients used for encoding the macroblock MB # i. As is clear from the following formula (14), the evaluation value Costxy increases as the overlapping area increases. The evaluation value Costxy increases as the qP value increases. Furthermore, the evaluation value Costxy increases as the number of effective coefficients decreases.

The reference vector basemvCol is calculated by calculating a weighted average using the evaluation value Costxy given in this way as a weight. For example, the reference vector basemvCol is calculated according to the above equation (13). Here, basemvCol _x and basemvCol _y are the x component and the y component of the reference vector basemvCol, respectively. Further, MVx _i and MVy _i are the x component and y component of the motion vector used for prediction of the macroblock MB # i, respectively.

The reference vector basemCol is corrected by the above method, and a direct vector that can be used for the prediction of CurrMB is obtained.
In the above, the function which the encoding apparatus 100 has was demonstrated.

[2-6. Function of Decoding Device]
Next, functions of the decoding device 200 will be described with reference to FIGS.

  FIG. 15 is a first block diagram illustrating an example of functions of the decoding device according to the second embodiment. FIG. 16 is a second block diagram illustrating an example of the functions of the decoding device according to the second embodiment. FIG. 17 is a third block diagram illustrating an example of functions of the decoding device according to the second embodiment.

(2-6-1. Overall)
As illustrated in FIG. 15, the decoding device 200 includes a first viewpoint decoding unit 201, a statistical information acquisition unit 202, and a second viewpoint decoding unit 203. Note that in the decoding device 200, the functions of the first viewpoint decoding unit 201, the statistical information acquisition unit 202, and the second viewpoint decoding unit 203 can be realized using the above-described CPU 902 or the like.

  The first viewpoint decoding unit 201 receives encoded data obtained by encoding a Base-View picture (hereinafter referred to as Base-View encoded data). The first viewpoint decoding unit 201 decodes the input Base-View encoded data. The function of the first viewpoint decoding unit 201 will be described in detail later.

  The statistical information acquisition unit 202 acquires statistical information obtained when the first viewpoint decoding unit 201 decodes Base-View encoded data. The statistical information includes, for example, the qP value and the number of differential effective coefficients. The statistical information acquired by the statistical information acquisition unit 202 is input to the second viewpoint decoding unit 203.

  The second viewpoint decoding unit 203 receives encoded data obtained by encoding a non-Base-View picture (hereinafter, non-Base-View encoded data). The second viewpoint decoding unit 203 decodes the non-Base-View encoded data using the motion vector and the information related to the picture used by the first viewpoint decoding unit 201 and the statistical information. The function of the second viewpoint decoding unit 203 will be described in detail later.

(2-6-2. Details of First View Decoding Unit 201)
Here, the function of the first viewpoint decoding unit 201 will be further described with reference to FIG.

  As illustrated in FIG. 16, the first viewpoint decoding unit 201 includes a variable length decoding unit 211, an inverse orthogonal transform / inverse quantization unit 212, a prediction mode acquisition unit 213, a switch 214, and an intra prediction unit 215. The first viewpoint decoding unit 201 includes a motion vector acquisition unit 216, a frame memory 217, a motion compensation unit 218, a switch 219, and an adder 220. Furthermore, the first viewpoint decoding unit 201 includes a motion vector memory 221, a reference vector acquisition unit 222, and a direct vector calculation unit 223.

  The variable-length decoding unit 211 receives Base-View encoded data (Base-View side bit stream). The variable-length decoding unit 211 to which the Base-View encoded data is input performs variable-length decoding on the Base-View encoded data in a scheme corresponding to the variable-length encoding used by the encoding device 100, and the quantized data Restore information such as. At this time, the variable length decoding unit 211 also restores header information such as SPS (Sequence Parameter Set) and PPS (Picture Parameter Set), information such as prediction mode and motion vector for each macroblock, and difference coefficient.

  Information such as the prediction mode restored by the variable length decoding unit 211 is input to the prediction mode acquisition unit 213. Information such as header information and difference coefficients restored by the variable length decoding unit 211 is input to the statistical information acquisition unit 202. The quantized data restored by the variable length decoding unit 211 is input to the inverse orthogonal transform / inverse quantization unit 212.

  The inverse orthogonal transform / inverse quantization unit 212 dequantizes the quantized data input from the variable length decoding unit 211 to restore the frequency component data. The inverse orthogonal transform / inverse quantization unit 212 performs inverse orthogonal transform on the frequency component data to restore the prediction error data. The restored prediction error data is input to the adder 220. The adder 220 adds the MB data input via the switch 219 and the prediction error data restored by the inverse orthogonal transform / inverse quantization unit 212 to generate a decoded image.

  The decoded image generated by the adder 220 is input to the frame memory 217 and output as a decoding result. Note that a deblocking filter may be applied to the decoded image. The frame memory 217 stores the decoded image input by the adder 220. The decoded image stored in the frame memory 217 is referred to by the motion compensation unit 218.

  The prediction mode acquisition unit 213 determines a prediction mode (intra prediction mode, forward prediction mode, backward prediction mode, bidirectional prediction mode, direct mode) for each macroblock. Note that the block division size may be added to the prediction mode. The switch 214 is switched according to the determination result of the prediction mode. When the prediction mode is the intra prediction mode, the intra prediction unit 215 decodes the target macroblock using the intra prediction. When the prediction mode is any one of the forward prediction mode, the backward prediction mode, and the bidirectional prediction mode, the motion vector acquisition unit 216 acquires information on the motion vector restored by the variable length decoding unit 211.

  When the prediction mode is the direct mode, the reference vector acquisition unit 222 acquires the motion vector used at the time of decoding ColPic from the motion vector memory 221. In addition, the reference vector acquisition unit 222 sets the acquired motion vector as the reference vector basemvCol. The reference vector basemCol set by the reference vector acquisition unit 222 is input to the direct vector calculation unit 223. The direct vector calculation unit 223 calculates a direct vector by scaling the reference vector basemCol.

  The direct vector calculated by the direct vector calculation unit 223 is stored in the motion vector memory 221. The direct vector stored in the motion vector memory 221 is referred to by the motion compensation unit 218. The motion compensation unit 218 performs motion compensation of the decoded image (reference picture) stored in the frame memory 217 using the direct vector or the motion vector acquired by the motion vector acquisition unit 216, and after motion compensation MB data is generated. The MB data after motion compensation is input to the adder 220 via the switch 219.

As described above, decoding of Base-View encoded data is performed independently of decoding of non-Base-View encoded data.
(2-6-3. Details of second viewpoint decoding unit 203)
Here, the function of the second viewpoint decoding unit 203 will be further described with reference to FIG.

  As illustrated in FIG. 17, the second viewpoint decoding unit 203 includes a variable length decoding unit 231, an inverse orthogonal transform / inverse quantization unit 232, a prediction mode acquisition unit 233, a switch 234, and an intra prediction unit 235. The second viewpoint decoding unit 203 includes a motion vector acquisition unit 236, a motion compensation unit 237, a frame memory 238, a switch 239, and an adder 240. Further, the second viewpoint decoding unit 203 includes a motion vector memory 241, a reference vector acquisition unit 242, a reference vector correction unit 243, and a direct vector calculation unit 244.

  Non-Base-View encoded data (non-Base-View side bit stream) is input to the variable length decoding unit 231. The variable length decoding unit 231 to which the non-Base-View encoded data is input performs variable length decoding on the non-Base-View encoded data in a scheme corresponding to the variable length encoding used by the encoding apparatus 100, Information such as data. At this time, the variable length decoding unit 231 also restores information such as header information such as SPS and PPS, prediction modes and motion vectors for each macroblock, and difference coefficients.

  Information such as the prediction mode restored by the variable length decoding unit 231 is input to the prediction mode acquisition unit 233. The quantized data restored by the variable length decoding unit 231 is input to the inverse orthogonal transform / inverse quantization unit 232.

  The inverse orthogonal transform / inverse quantization unit 232 dequantizes the quantized data input from the variable length decoding unit 231 to restore the frequency component data. Further, the inverse orthogonal transform / inverse quantization unit 232 performs inverse orthogonal transform on the frequency component data to restore the prediction error data. The restored prediction error data is input to the adder 240. The adder 240 adds the MB data input via the switch 239 and the prediction error data restored by the inverse orthogonal transform / inverse quantization unit 232 to generate a decoded image.

  The decoded image generated by the adder 240 is input to the frame memory 238 and output as a decoding result. Note that a deblocking filter may be applied to the decoded image. The frame memory 238 stores the decoded image input by the adder 240. The decoded image stored in the frame memory 238 is referred to by the motion compensation unit 237.

  The prediction mode acquisition unit 233 determines a prediction mode (intra prediction mode, forward prediction mode, backward prediction mode, bidirectional prediction mode, direct mode) for each macroblock. Note that the block division size may be added to the prediction mode. The switch 234 is switched according to the determination result of the prediction mode. When the prediction mode is the intra prediction mode, the intra prediction unit 235 decodes the target macroblock using the intra prediction. When the prediction mode is any one of the forward prediction mode, the backward prediction mode, and the bidirectional prediction mode, the motion vector acquisition unit 236 acquires information on the motion vector restored by the variable length decoding unit 231.

  When the prediction mode is the direct mode, the reference vector acquisition unit 242 acquires a motion vector used for CurrMB prediction from the motion vector memory 241. The reference vector acquisition unit 242 determines whether the motion vector used for CurrMB prediction is the inter-viewpoint motion vector iMV.

  When the motion vector used for CurrMB prediction is the inter-viewpoint motion vector iMV, the reference vector acquisition unit 242 identifies a ColPic macroblock (ColMB) located at the same position as CurrMB. Then, the reference vector acquisition unit 242 acquires a ColMB motion vector from the motion vector memory 221 of the first viewpoint decoding unit 201 (see FIG. 5). Note that ColPic is a Base-View picture at the same time (same POC) as CurrPic. In this case, the motion vector acquired by the reference vector acquisition unit 242 is input to the reference vector correction unit 243.

  On the other hand, when the motion vector used for CurrMB prediction is not the inter-viewpoint motion vector iMV, the reference vector acquisition unit 242 identifies a ColPic macroblock (ColMB) that is in the same position as CurrMB. In this case, ColPic is a non-Base-View picture. Further, the reference vector acquisition unit 242 determines whether or not the motion vector used for the prediction of ColMB is the inter-viewpoint motion vector iMV.

  When the motion vector used for the prediction of ColMB is the inter-view motion vector iMV, the reference vector acquisition unit 242 specifies the macro block (RefMB) of the Base-View picture pointed to by the inter-view motion vector iMV used for the prediction of ColMB. Then, the reference vector acquisition unit 242 acquires the RefMB motion vector from the motion vector memory 221 of the first viewpoint decoding unit 201 (see FIG. 4). In this case, the motion vector acquired by the reference vector acquisition unit 242 is input to the reference vector correction unit 243.

  On the other hand, when the motion vector used for the prediction of ColMB is not the inter-viewpoint motion vector iMV, the reference vector acquisition unit 242 specifies a ColPic macroblock (ColMB) at the same position as CurrMB, and stores the motion vector of ColMB in the motion vector memory. 241 (see FIG. 2). In this case, the motion vector acquired by the reference vector acquisition unit 242 is input to the direct vector calculation unit 244 as the reference vector basemvCol.

  The reference vector correction unit 243 corrects the motion vector input from the reference vector acquisition unit 242 based on the statistical information input from the statistical information acquisition unit 202 and generates a reference vector basemvCol. The correction method performed when generating the reference vector basemvCol is the same as the correction method performed by the encoding device 100 when encoding. The reference vector basemCol generated by the reference vector correction unit 243 is input to the direct vector calculation unit 244.

  The direct vector calculation unit 244 scales the reference vector basemvCol with time distribution to generate a CurrMB motion vector (direct vector). The scaling method is the same as the scaling method performed by the encoding device 100 during encoding. The direct vector generated by the direct vector calculation unit 244 is stored in the motion vector memory 241. The direct vector stored in the motion vector memory 241 is referred to by the motion compensation unit 237.

  The motion compensation unit 237 performs motion compensation of the decoded image (reference picture) stored in the frame memory 238 using the direct vector or the motion vector acquired by the motion vector acquisition unit 236, and after motion compensation MB data is generated. The MB data after motion compensation is input to the adder 240 via the switch 239.

As described above, the decoding of the non-Base-View encoded data is performed using the decoding result of the Base-View encoded data.
In the above, the function which the decoding apparatus 200 has was demonstrated.

[2-7. Processing flow]
Next, the flow of the encoding process executed by the encoding device 100 will be described with reference to FIGS.

  FIG. 18 is a first flowchart showing the flow of the encoding process according to the second embodiment. FIG. 19 is a second flowchart showing the flow of the encoding process according to the second embodiment. FIG. 20 is a third flowchart showing the flow of the encoding process according to the second embodiment.

(2-7-1. Overall)
The overall flow of the encoding process will be described with reference to FIG.
The picture to be encoded is referred to as CurrPic, the macroblock to be encoded is referred to as CurrMB, and the picture pointed to by the motion vector mvCol used for CurrMB prediction is referred to as ColPic. Also, the inter-viewpoint motion vector is represented as iMV, the reference vector is represented as basemvCol, and the direct vector is represented as mvL ₀ and mvL ₁ . Also, a macroblock in ColPic located at the same position as CurrMB is denoted as ColMB, a motion vector used for prediction of ColMB is denoted as mvRef, a picture pointed to by mvRef is denoted as RefPicCol, and a block pointed to by iMV is denoted as iMB.

  (S101) The encoding apparatus 100 determines whether or not the encoding method is the MVC method and CurrPic is a picture (B picture) encoded using bi-directional prediction. If the encoding method is the MVC method and CurrPic is a B picture, the process proceeds to S102. On the other hand, if the encoding method is not the MVC method or CurrPic is not the B picture, the process proceeds to S108.

  (S102) The encoding apparatus 100 determines whether ColPic is a picture of another viewpoint. When ColPic is a picture of another viewpoint (that is, when mvCol is the inter-viewpoint motion vector iMV), the process proceeds to S103. On the other hand, if ColPic is not a picture of another viewpoint, the process proceeds to S105.

  (S103) The encoding apparatus 100 calculates the reference vector basemvCol using the inter-viewpoint motion vector iMV (mvCol). The calculation process of the reference vector basemvCol will be described in detail later.

(S104) The encoding apparatus 100 calculates the direct vectors mvL ₀ and mvL ₁ by scaling the reference vector basemvCol. For example, the encoding apparatus 100 scales the reference vector basemvCol by a method as illustrated in FIGS. When the process of S104 is completed, the process proceeds to S108.

  (S105) The encoding apparatus 100 determines whether RefPicCol is a picture of another viewpoint. When RefPicCol is a picture of another viewpoint (that is, when mvRef is an inter-viewpoint motion vector iMV), the process proceeds to S106. On the other hand, if RefPicCol is not a picture of another viewpoint, the process proceeds to S108.

  (S106) The encoding apparatus 100 calculates the reference vector basemvCol using the inter-view motion vector iMV (mvRef). The calculation process of the reference vector basemvCol will be described in detail later.

(S107) The encoding apparatus 100 calculates the direct vectors mvL ₀ and mvL ₁ by scaling the reference vector basemvCol. For example, the encoding apparatus 100 scales the reference vector basemVCol by a method as illustrated in FIG.

  (S108) The encoding apparatus 100 determines a suitable prediction mode used for CurrMB prediction. For example, the encoding apparatus 100 selects a prediction mode with the lowest encoding cost among five prediction modes (intra prediction mode, forward prediction mode, backward prediction mode, bidirectional prediction mode, and direct mode). To determine the result.

(S109) The encoding apparatus 100 encodes CurrPic using the prediction mode selected in the process of S108.
(S110) The encoding apparatus 100 determines whether or not the encoding process has been completed for all macroblocks (for one picture) included in the CurrPic. When the encoding process is completed for all the macroblocks included in CurrPic, the series of processes illustrated in FIG. 18 ends. On the other hand, if the encoding process has not been completed for all the macroblocks included in CurrPic, the process proceeds to S102.

(2-7-2. Calculation of reference vector)
Here, the processing of S103 and S106 will be further described with reference to FIG.

(S121) The encoding apparatus 100 calculates an overlap area between the block iMB indicated by the inter-viewpoint motion vector iMV and the peripheral blocks located around the block iMB.
(S122) The encoding apparatus 100 determines whether to select a peripheral block having the largest overlap area calculated in the process of S121. That is, the encoding apparatus 100 determines whether or not to adopt the correction method for the reference vector basemvCol illustrated in FIG. Note that the correction method to be adopted is set in advance. When the peripheral block with the largest overlap area is selected, the process proceeds to S123. On the other hand, when the peripheral block having the largest overlap area is not selected, the process proceeds to S124.

  (S123) The encoding apparatus 100 specifies a peripheral block including the center coordinates of the block iMB. Then, the encoding apparatus 100 sets the motion vector of the identified peripheral block as the reference vector basemvCol. In the description related to FIG. 13, the method of calculating the overlapping area for each peripheral block and selecting the peripheral block corresponding to the maximum value of the overlapping area has been introduced. But you can get similar results. When the process of S123 is completed, the series of processes shown in FIG.

  (S124) The encoding apparatus 100 performs weighted averaging of the motion vectors of the peripheral blocks based on the overlapping area between the iMB and the peripheral blocks. For example, the encoding apparatus 100 calculates a weighted average based on the above equations (12) to (14). The weighted average calculation process will be described in detail later.

  (S125) The encoding apparatus 100 sets a vector obtained by weighted averaging of the motion vectors of the neighboring blocks as the reference vector basemvCol. When the process of S125 is completed, the series of processes shown in FIG.

(2-7-3. Calculation of weighted average)
Here, the processing of S124 will be further described with reference to FIG.
(S131) The encoding apparatus 100 selects one of the peripheral blocks overlapping the block iMB.

(S132) The encoding apparatus 100 acquires an overlapping area related to the peripheral block selected in the process of S131 and the block iMB.
(S133) The encoding apparatus 100 calculates an evaluation value based on the overlapping area acquired in the process of S132. For example, the encoding apparatus 100 calculates the evaluation value Costxy based on the above formula (12) or formula (14).

  (S134) The encoding apparatus 100 selects all the peripheral blocks that overlap the block iMB, and determines whether or not the evaluation value Costxy for each peripheral block has been calculated. If all of the neighboring blocks that overlap the block iMB have been selected, the process proceeds to S135. On the other hand, if all the peripheral blocks overlapping the block iMB have not been selected, the process proceeds to S131.

  (S135) The encoding apparatus 100 calculates a weighted average of motion vectors used for prediction of each neighboring block using the evaluation value Costxy of each neighboring block calculated in the process of S133. For example, the encoding apparatus 100 calculates a weighted average based on the above equation (13). When the process of S135 is completed, the series of processes shown in FIG.

The flow of the encoding process executed by the encoding device 100 has been described above.
As described above, if the technique according to the second embodiment is applied, the temporal direct mode can be used even when inter-view prediction is applied to CurrPic and ColPic. As a result, this contributes to further improvement in encoding efficiency in multi-view video encoding.

The second embodiment has been described above.
<3. Third Embodiment>
Next, a third embodiment will be described with reference to FIGS. 21 and 22. FIG. 21 is a first diagram for describing an encoding method according to the third embodiment. FIG. 22 is a second diagram for describing the encoding method according to the third embodiment. The third embodiment is described in H.264. This corresponds to an application example to the H.265 / HEVC scheme, and provides a mechanism for combining inter-view prediction and merge mode. H. In the H.265 / HEVC scheme, a PU (Prediction Unit) is used as a predictive coding unit.

  In merge mode, a PU having a motion parameter that can be used as it is is selected from PUs located near the time and near the space of the encoding target PU (hereinafter referred to as CurrPU), and the motion parameters of the selected PU are shared. It is a method to convert. When the merge mode is applied, only the index indicating the position of the selected PU needs to be encoded, which contributes to improvement in encoding efficiency. Hereinafter, H.C. The spatial neighborhood prediction, temporal neighborhood prediction employed in the H.265 / HEVC scheme, and the relationship between these prediction methods and inter-viewpoint prediction will be described.

  H. In the H.265 / HEVC scheme, a motion vector prediction technique called AMVP (Adaptive Motion Vector Prediction) is employed. In AMVP, not only a spatial motion vector on the same picture but also a temporal motion vector on an encoded reference picture is considered as a prediction value candidate.

For example, the motion vector (mvL _A0 , mvL _A1 , mvL _B0 , mvL _B1 , mvL _B2 ) of the PU including each of the pixels A ₀ , A ₁ , B ₀ , B ₁ , B ₂ shown in FIG. Are candidates for predicted values near the space. Further, the motion vector (mvL _C0 , mvL _C1 ) of the PU including each of the pixels C ₀ and C ₁ shown in (B) of FIG. 21 is a motion vector candidate in the vicinity of time. Furthermore, mvL _A0, mvL _A selected from mvL _A1, the mvL _B0, mvL _B1, mvL selected from mvL _{B2 B,} mvL _C0, mvL selected from mvL _{C1 C,} and the predicted value from the zero vector Candidates are narrowed down.

  CurrPU prediction is performed using prediction value candidates narrowed down in this way. Here, attention is focused on the method for deriving the temporal neighborhood prediction value. H. In the H.265 / HEVC method, the H.264 / HEVC method is used when the temporal neighborhood prediction value (CurrPU motion vector) is derived. The same scaling process as in the H.264 / AVC time direct mode is performed.

For example, the process of scaling a motion vector mvCol predicting the PU in time near the picture in CurrPU the same position (position including the pixel C ₁₎ (ColPic) is performed. At this time, colPic the target PU (e.g., PU including the pixel C ₁₎ the distance between the reference picture and colPic which refers td, the distance tb between the picture (CurrPic) including a reference picture and CurrPU which CurrPU references are utilized The The scaling method is the same as in the time direct mode illustrated in FIG.

When applying inter-view prediction to CurrPU, the situation shown in FIG. 22 is considered to occur. That is, referring to non-Base-View (Dependent View) CurrPU is by interview motion vector iMV of Base-View picture side ColPU (e.g., PU including the pixel D _1). In this case, the above scaling process cannot be performed. However, since the scaling process can be performed by applying the method illustrated in FIG. 11, the CurrPU motion vector can be predicted using the temporal neighborhood prediction value. As a result, H.C. In the H.265 / HEVC system, an improvement in encoding efficiency can be expected.

The third embodiment has been described above.
As mentioned above, although preferred embodiment was described referring an accompanying drawing, this invention is not limited to the example which concerns. It is obvious for a person skilled in the art that various variations and modifications can be conceived within the scope of the claims, and such variations and modifications are naturally understood by the technical scope of the present invention. It goes without saying that it belongs to a range.

10 image encoding device 11 storage unit 12 operation unit 21 moving picture _{Pic 10, Pic 11, Pic 12} , Pic 13, Pic 14, Pic 15, Pic 20, Pic 21, Pic 22, Pic 23, Pic 24, Pic ₂₅ , ColPic image CurrPic target image basemvCol ₀ , basemvCol ₁ , mvL ₀ , mvL ₁ motion information mvCol inter-viewpoint motion information tbL ₀ , tbL ₁ , tdL ₀ , tdL ₁ time

Claims (8)

A storage unit storing a plurality of moving images respectively corresponding to a plurality of viewpoints;
A target image having inter-viewpoint motion information that refers to an image at the same time included in another moving image is detected from images included in the one moving image, and the motion information included in the image at the same time is detected. A calculation unit that encodes an image region of the target image using,
An image encoding device. The calculation unit specifies one or a plurality of image areas associated with the image area of the target image based on the inter-viewpoint movement information among the image areas included in the image at the same time, and the specified image area The image encoding device according to claim 1, wherein an image region of the target image is encoded based on the motion information of the target image. When there are a plurality of image regions associated with the image region of the target image based on the inter-viewpoint movement information, the calculation unit selects the image region having the maximum size among the plurality of identified image regions, The image encoding device according to claim 2, wherein the image region of the target image is encoded based on the motion information of the selected image region. When there are a plurality of image areas associated with the image area of the target image based on the inter-viewpoint motion information, the calculation unit sets a weight on the motion information of the image area based on the specified size of the image area, The image encoding device according to claim 2, wherein statistical information of the motion information is generated in consideration of the set weight, and an image region of the target image is encoded based on the statistical information. When there are a plurality of image areas associated with the image area of the target image based on the inter-viewpoint motion information, the arithmetic unit weights the motion information of the image area based on the specified size and quantization information of the image area. The image encoding device according to claim 2, wherein statistical information of the motion information is generated in consideration of the set weight, and an image region of the target image is encoded based on the statistical information. A computer capable of acquiring the moving image from a storage unit storing a plurality of moving images respectively corresponding to a plurality of viewpoints,
A target image having inter-viewpoint motion information that refers to an image at the same time included in another moving image is detected from images included in the one moving image, and the motion information included in the image at the same time is detected. An image encoding method for encoding an image region of the target image using the image encoding method. A storage unit that stores encoded data obtained by encoding a plurality of moving images respectively corresponding to a plurality of viewpoints;
In the process of decoding the encoded data, a target image having inter-viewpoint motion information that refers to an image at the same time included in another moving image is detected from images included in one moving image. A calculation unit that decodes the image area of the target image using motion information of the image at the same time;
An image decoding apparatus. A computer capable of acquiring the encoded data from a storage unit in which encoded data obtained by encoding a plurality of moving images respectively corresponding to a plurality of viewpoints is stored.
In the process of decoding the encoded data, a target image having inter-viewpoint motion information that refers to an image at the same time included in another moving image is detected from images included in one moving image. An image decoding method for decoding an image area of the target image using motion information included in the image at the same time.

Images