JP2009505604A - Method and apparatus for encoding multi-view video - Google Patents

Abstract

A method and apparatus for encoding a multi-view video so as to minimize the information amount of the multi-view video. A multi-view video encoding method according to the present invention includes a step of grouping a plurality of B frames into at least two groups according to a predetermined criterion, and a step of sequentially encoding the grouped B frames. In addition, it is possible to provide a multi-view video that minimizes the amount of information and provides a large number of people with a stereoscopic effect and a sense of reality.

Description

  The present invention relates to a method and apparatus for encoding a multi-view video sequence, and more particularly, encodes a multi-view video to minimize the amount of information of a multi-view video represented by a multi-view camera. The present invention relates to a method and an apparatus.

  One of the most ideal characteristics for realizing high quality information and communication services is reality. This can be achieved by image communication based on 3D video. 3D video systems can be used in many potential applications such as education, entertainment, medical surgery, and image conferencing. In order to convey a greater sense of reality and accurate information about remote scenes to multiple viewers, three or more cameras are placed in slightly different views to generate a multi-view sequence.

  Many research groups have reported on 3D video processing and display systems due to their interest in the field of 3D video. In Europe, research on 3DTV has been disclosed by various projects, such as DISTIMA, for the purpose of developing a system for capturing, encoding, and transmitting digital stereo video sequences for display. These projects led PANORAMA, another project aimed at improving video information in 3D telepresence communications. Currently, these projects led ATTEST, another project, in which technologies in various fields of 3D content acquisition, 3D compression and transmission, and 3D display systems were studied. In this project, MPEG-2 and DVB (Digital Video Broadcasting) systems, especially for using a base layer for 2D content transmission and an advanced layer for transmission of 3D depth data, It has been applied to transmit 3D content with Temporal Scalibility (TS).

  In MPEG-2, the Multiview Profile (MVP) was defined in 1996 with a modification to the MPEG-2 standard. An important new component of MVP is the definition of the use of time extensibility (TS) mode for multi-camera sequences and the definition of acquired camera parameters in MPEG-2 syntax.

  Encodes a base layer stream that represents the signal at a reduced frame rate and, if the two streams are valid, used to insert additional frames in the middle to allow playback at the full frame rate Defined enhancement layers. An efficient way to encode the enhancement layer is that for each macroblock in the enhancement layer frame, a decision regarding the best motion compensated prediction is made from the base layer frame or the recently reconstructed enhancement layer frame. Is acceptable.

  It is easy to perform stereo and multi-view channel coding for such signals using time-expandable syntax. For this purpose, a frame from one camera viewpoint (usually a left eye frame) is defined as a base layer, and the remaining frames are defined as enhancement layers. The base layer shows a simultaneous monoscopic sequence. For the enhancement layer, disparity compensated prediction may fail in the occluded region, but continues to maintain the video quality reconstructed by motion compensated prediction within the same channel. Since MPEG-2MVP is mainly defined as a stereo sequence, it does not support multi-view sequences and is difficult to extend from the original to multi-view sequences.

  FIG. 1 is a diagram illustrating an MPEG-2 multi-view profile encoder and decoder.

  The extensibility provided by MPEG-2 is for decoding video having other resolutions and formats at the same time using one video equipment. Of the extensibility supported by MPEG-2, time extension Is a technique for improving visual image quality by increasing the frame rate. The multi-view profile is applied to stereo moving images in consideration of such time extensibility.

  Substantially, the structure of the encoder and decoder having the concept of stereo video has the same structure as the time extensibility of FIG. 1, and the left video of the stereo video is base viewpoint encoding. The right video of the stereo moving image is input to a temporal auxiliary view encoder (temporal auxiliary view encoder).

  The auxiliary viewpoint encoder is an interlayer encoder that is for temporal extensibility and interleaves video between base layer videos temporally.

  Accordingly, if the left video is separately encoded and decoded, a two-dimensional video can be obtained. If the left video and the right video are encoded and decoded simultaneously, a stereoscopic video can be realized. Here, a system multiplexer and a system demultiplexer that match or separate the sequences of two images for moving image transmission and storage are required.

  FIG. 2 is a diagram illustrating a stereo video encoder and decoder using an MPEG-2 multi-view profile.

  The base layer is encoded using motion compensation and discrete cosine transform (DCT), and is decoded through an inverse process. The temporal auxiliary viewpoint encoder performs a role of a temporal inter encoder that is predicted based on the decoded base layer image.

  That is, two disparity compensated predictions, or one disparity prediction and motion compensated prediction each, are used here, and the temporal auxiliary viewpoint encoder as well as the base layer encoder and decoder Compensating DCT encoder and decoder.

  In addition, a disparity predictor and a compensator are required in the disparity compensation encoding process, as a motion predictor (predictor) and a compensator are required in the motion prediction / compensation encoding process. In addition to block-based motion / disparity prediction and compensation, the encoding process includes DCT such as the difference video between the predicted result video and the original video, quantization of DCT coefficients, variable length coding, and the like. Conversely, the decoding process includes processes such as variable length decoding, inverse quantization, and inverse DCT.

  MPEG-2 coding is a very efficient compression method with bi-directional motion prediction for B-pictures, and the time extensibility is also quite efficient. Highly efficient compression is obtained when used for video encoding.

  FIG. 3 is a diagram illustrating prediction coding using only two disparity predictions for bi-directional prediction and considering only disparity.

  The left video is encoded using a non-scalable MPEG-2 encoder, and the right video is MPEG-2 temporally located based on the decoded left video. Is encoded using.

  That is, the right video is encoded with a B picture using predictions obtained from two reference videos, eg, the left video. At this time, one of the two reference images is a left image displayed at the same time, and the other is a left image that appears next in time.

  The two predictions have three prediction modes, forward, backward, and bidirectional, as in motion estimation / compensation. Here, the forward mode means the parallax predicted from the left video at the same time, and the reverse mode means the parallax predicted from the next left video immediately. In the case of such a method, since the prediction of the right image is performed through the disparity vectors of the two left images, this type of prediction method is referred to as predictive encoding considering only the disparity vector. Therefore, the encoder estimates two disparity vectors for each frame of the right moving image, and the decoder uses the two disparity vectors to decode the right moving image from the left moving image.

  FIG. 4 is a diagram illustrating predictive coding using a disparity vector and a motion vector for bidirectional prediction. FIG. 4 uses the B picture through bi-directional prediction shown in FIG. 3, but bi-directional prediction uses one disparity estimate and one motion estimate. That is, the parallax prediction from the left video of the same time zone and the motion prediction from the right video of the immediately preceding time are used.

  Similar to predictive coding considering only disparity, bi-directional prediction includes three kinds of prediction modes called forward, reverse, and interpolated modes. Here, the forward mode means motion prediction from the decoded right video, and the reverse mode means parallax prediction from the decoded left video.

  As described above, the MPEG-2 multi-view profile standard itself is designed to be incompatible with actual stereo video without considering an encoder for multi-view video, so that a large number of people can experience stereoscopic effects simultaneously. There is also a need for an encoder that can efficiently provide multi-view video for providing a sense of reality.

  The technical problem to be solved by the present invention is to provide a method and apparatus for encoding multi-view video that efficiently provides multi-view video by providing stereoscopic feeling and realism to a large number of people simultaneously. It is.

  Another technical problem to be solved by the present invention is to provide a method and apparatus for encoding a multi-view video using a prediction structure that minimizes the amount of information about the multi-view video. .

  The technical problem includes a step of grouping a plurality of B frames according to the present invention into at least two groups according to a predetermined criterion, and a step of sequentially encoding the grouped B frames. This is achieved by the viewpoint video encoding method.

  Desirably, the predetermined standard may be the number of frames referred to by each B frame, or the predetermined standard may be the number of frames referred to by each B frame and the position of the referenced frame.

  Preferably, the grouped B frames are predicted with reference to two horizontally adjacent frames, two vertically adjacent frames, or one horizontally adjacent frame and one vertically adjacent frame. A group and a second group predicted with reference to two horizontally adjacent frames and one vertically adjacent frame, or one horizontally adjacent frame and two vertically adjacent frames, and horizontally Including a third group predicted with reference to two adjacent frames and two vertically adjacent frames, wherein the one or two horizontally adjacent frames are multi-viewpoints at the same time as B frames A frame obtained from a moving image, and the one or two vertically adjacent frames are frames obtained from a multi-view moving image at the same viewpoint, such as a B frame. Multi-view video, wherein the at.

  Preferably, the step of sequentially encoding the grouped B frames includes the step of sequentially encoding the grouped B frames in the order of the first group, the second group, and the third group.

  Desirably, the moving picture coding structure including the B frame is horizontally and vertically extended as a structure for predicting disparity between frames from a plurality of viewpoints horizontally and vertically predicting motion between frames over time. Can be done.

  Preferably, the moving picture coding structure includes B frames, and the moving picture coding structure for n viewpoints does not operate the n-1th frame sequence, thereby moving the moving picture coding for n-1 viewpoints. It is configured as a structure, where n is an odd number of natural numbers.

  According to another aspect of the present invention, there is provided a prediction unit that predicts a disparity vector and a motion vector for an input multi-view video, and compensates the image using the predicted disparity vector and motion vector. A parallax motion compensation unit, a difference video encoding unit that receives the reconstructed video provided from the original video and the parallax motion compensation unit, and performs differential video encoding; a parallax vector, a motion vector, and a differential video encoding An entropy encoding unit that generates a bitstream for multi-view video using the performed data, and the prediction unit groups a plurality of B frames into at least two groups according to a predetermined criterion, and performs grouping This is achieved by a multi-view video encoding apparatus that sequentially predicts the B frames.

  The technical problem is achieved by a computer-readable recording medium storing a program for implementing the multi-view video encoding method of the present invention.

  According to the present invention, it is possible to provide a method and apparatus for encoding a multi-view video that efficiently provides a multi-view video by providing a stereoscopic effect and a sense of presence to a large number of people at the same time.

  Also, a method and apparatus for encoding a multi-view video that minimizes the amount of information about the multi-view video by adopting a B-frame prediction structure according to the present invention may be provided.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 5 is a block diagram showing an internal configuration of a multi-view video encoding apparatus according to an embodiment of the present invention.

  The multi-view video encoding apparatus according to an embodiment of the present invention includes a multi-view video buffer 510, a prediction unit 520, a parallax / motion compensation unit 530, a difference video encoding unit 540, and an entropy encoding unit 550.

  In FIG. 5, the proposed multi-view video encoder receives a multi-view video source that is typically acquired from multiple camera systems or other possible methods. The input multi-view video is stored in the multi-view video buffer 510. The multi-view video buffer 510 provides the stored multi-view video source data to the prediction unit 520 and the difference video encoding unit 540.

  The prediction unit 520 includes a parallax estimation unit 522 and a motion estimation unit 524, and performs motion prediction and parallax prediction on the stored multi-view video source. The prediction unit 520 estimates the parallax vector and the motion vector in the direction of the arrows in FIGS. 6 to 11 and provides them to the parallax / motion compensation unit 530.

  When the prediction unit 520 performs prediction about motion and parallax, it occurs when the multi-view parallax vector and this are expanded to a moving picture as in the multi-view moving picture coding structure shown in FIGS. The estimated direction can be set by efficiently using the motion vector. That is, the spatial / temporal correlation can be used by extending the MPEG-2 coding structure to the viewpoint axis.

  In the parallax / motion compensation unit 530, parallax and motion compensation are executed using the motion vector and the parallax vector estimated by the parallax estimation unit 522 and the motion estimation unit 524. The parallax / motion compensation unit 530 provides the video restored using the estimated motion and the parallax vector to the difference video coding unit 540.

  The difference video encoding unit 540 provides difference information between the original video provided from the multi-view video buffer 510 and the restored video compensated by the parallax / motion compensation unit 530 to provide better image quality and stereoscopic effect. Difference video coding is performed and provided to the entropy coding unit 550.

  The entropy encoding unit 550 receives the information about the disparity vector and the motion vector generated by the prediction unit 520 and the residual video from the difference video encoding unit 540, and receives a bitstream for the multi-view video source data Is generated.

  FIG. 6 is a diagram illustrating a unit coding structure of a multi-view video according to an embodiment of the present invention. FIG. 6 shows a core-predictive structure or a unit-predictive structure when the number of viewpoints is assumed to be three. A square block indicates a video frame in a multi-view video source. The horizontal arrows indicate the sequence of frames by viewpoint or camera position, and the vertical arrows indicate the sequence of frames over time. In the frame, the I picture is MPEG-2 / 4 or H.264. The same “intra picture” as the I frame in H.264 is shown. P picture and B picture are MPEG-2 / 4 or H.264, respectively. Similarly to H.264, “prediction picture” and “bidirectional prediction picture” are shown.

  However, the P picture and the B picture are both predicted by motion prediction and disparity prediction in multi-view video coding. In FIG. 6, an arrow between picture frames means a prediction direction. An arrow indicating the horizontal direction indicates parallax measurement, and an arrow indicating the vertical direction indicates motion prediction. According to an embodiment of the present invention, there are three types of B pictures, which will be described with reference to FIG.

  FIG. 7 is a diagram illustrating three types of B pictures used for encoding a multi-view video according to an embodiment of the present invention.

  A B picture according to an embodiment of the present invention is composed of three types of pictures. In FIG. 7, B-, B1-, and B2-pictures mean picture frames predicted by two or more frames that are horizontally or vertically adjacent.

  B-pictures can be two horizontally adjacent frames as shown in Figure 7A, two vertically adjacent frames as shown in Figure 7B, or one horizontally adjacent frame and one vertically adjacent as shown in Figure 7C. Predicted by one frame.

  A B1-picture is predicted by two horizontally adjacent frames as shown in Figure 7D, or one frame vertically adjacent, or two horizontally adjacent frames as shown in Figure 7E. The A B2-picture is predicted by four adjacent frames horizontally or vertically as shown in FIG. 7F.

  With reference to FIG. 6 again, a unit coding structure indicating a prediction sequence of a multi-view video is described. In FIG. 6, the basic prediction sequence order is as follows.

I → P → B → B1 → B2
First, the I frame 601 is intra-predicted. A P frame 603 is predicted horizontally in the I frame 601, and a P frame 610 is predicted vertically.

  Next, B frame 602 is predicted by performing bi-directional prediction horizontally with reference to I frame 601 and P frame 603, and B frame 604 is performed by performing bi-directional prediction vertically with reference to I frame 601 and P frame 610. And B frame 607 is predicted, and B frame 612 is predicted by referring to P frame 610 horizontally and P frame 603 vertically.

  A B1 frame is then predicted. B1 frame 606 is predicted with reference to B frame 604 horizontally, with reference to P frame 603 and B frame 612 with vertical reference to B frame 607 with reference to P frame 603 and B frame 612 vertically Thus, the B1 frame 609 is predicted, the P1 frame 610 and the B frame 612 are referenced horizontally, and the B1 frame 611 is predicted with reference to the B frame 602 vertically.

  Finally, a B2 frame is predicted. B2 frame 605 is predicted with reference to B frame 604 and B1 frame 606 horizontally, B frame 602 and B1 frame 611 with reference to B frame 607 and B1 frame 609 with reference to B frame 607 and B1 frame 611 with vertical B frame With reference to 602 and B1 frame 611, a B2 frame 608 is predicted.

  As described with reference to FIGS. 6 and 7, according to the present invention, when bidirectional prediction is performed, the number of B frames can be increased using not only B frames but also B1 frames and B2 frames. The amount of information can be minimized when the viewpoint video is encoded. Therefore, according to a preferred embodiment of the present invention, in order to efficiently encode a multi-view video, the B frames are grouped according to the type shown in FIG. 7, and the grouped B frames are B according to the prediction order. Encoding is performed in the order of frame, B1 frame, and B2 frame.

  FIG. 8 is a diagram illustrating a horizontally expanded structure of the multi-view video unit coding structure of FIG. 6 according to an embodiment of the present invention. FIG. 8 shows the structure of a prediction block having five viewpoints of input video sources.

  FIG. 9 is a diagram illustrating a prediction order of the multi-view video in FIG. In FIG. 8, frames in the same column mean that they are predicted at the same time. Therefore, referring to FIG. 9, first, intra prediction is performed on the I frame 801. Next, P frame 803 and P frame 816 in the second column are predicted, and B frames 802, 806, 811 and 818 and P frame 805 are predicted in the third column. Next, B1 frames 817, 808, 813 and B frames 804, 820 are predicted. In the fifth column, B2 frames 807 and 821 and B1 frames 810, 819 and 815 are predicted. Finally, B2 frames 809 and 814 are predicted.

  Therefore, the prediction order according to the present invention is as follows.

I → P → B → B1 → B2 → P → B → B1 → B2
FIG. 10 is a diagram illustrating a moving picture coding structure for motion prediction and disparity prediction having an odd number of viewpoints according to an embodiment of the present invention.

  FIG. 11 is a diagram illustrating a moving picture coding structure for motion prediction and disparity prediction having an even number of viewpoints according to an embodiment of the present invention. The moving picture coding structure in FIG. 11 is obtained by not operating the fourth predicted frame sequence in the moving picture coding structure for the five viewpoints in FIG. Such a moving picture coding structure according to the present invention can be extended horizontally and vertically.

  Therefore, according to an embodiment of the present invention, the moving picture coding structure for n (where n is an odd number among natural numbers) viewpoints does not operate the sequence of the (n-1) th predicted frame. -It can be configured as a moving picture coding structure for one viewpoint.

  FIG. 12 is a flowchart illustrating a multi-view video encoding process according to an embodiment of the present invention.

  In the multi-view video encoding process according to the embodiment of the present invention described with reference to FIGS. 6 to 11, the B frame is encoded as follows.

  A plurality of B frames are grouped into at least two groups according to a predetermined standard (S1210). The predetermined standard is the number of frames referred to by each B frame, or the number of frames referred to by each B frame and the position of the referenced frame.

  Grouped B frames can be either two horizontally adjacent frames, two vertically adjacent frames, or a first group predicted with reference to one horizontally adjacent frame and one vertically adjacent frame. A second group predicted with reference to two horizontally adjacent frames and one vertically adjacent frame, or one horizontally adjacent frame and two vertically adjacent frames, and two horizontally adjacent A third group predicted with reference to one frame and two vertically adjacent frames.

  In this way, the grouped B frames are sequentially encoded (S1220). At this time, the grouped B frames can be sequentially encoded in the order of the first group, the second group, and the third group.

  The multi-view video encoding method according to the present invention can be embodied as a computer-readable code on a computer-readable recording medium. Codes and code segments embodying the program can be easily inferred by computer programmers in the field. A computer readable recording medium includes any kind of recording device in which data read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, etc., and in the form of carrier waves (eg, transmission over the Internet). Including what is embodied. The computer-readable recording medium can be distributed in a computer system connected to a network and stored and executed as computer-readable code as a distributed system.

  The above description is only one embodiment of the present invention, and those skilled in the art will be able to implement the present invention in a modified form without departing from the essential characteristics of the present invention. Therefore, the scope of the present invention is not limited to the above-described embodiments, but should be construed to include various embodiments within the scope equivalent to the contents described in the claims.

It is a figure which shows the encoder and decoder of an MPEG-2 multi view profile (MVP: Multi-view profile). It is a figure which shows the stereo moving image encoder and decoder using an MPEG-2 multi view profile. It is a figure which shows the prediction encoding which considered only the parallax using two parallax predictions for bidirectional | two-way prediction. It is a figure which shows the prediction encoding which uses a disparity vector and a motion vector for bi-directional prediction. It is a block diagram which shows the internal structure of the multi-view video encoding device by one Embodiment of this invention. FIG. 3 is a diagram illustrating a unit coding structure of a multi-view video according to an embodiment of the present invention. It is a figure which shows 3 types of B picture types used for the encoding of the multi-view moving image by one Embodiment of this invention. It is a figure which shows 3 types of B picture types used for the encoding of the multi-view moving image by one Embodiment of this invention. It is a figure which shows 3 types of B picture types used for the encoding of the multi-view moving image by one Embodiment of this invention. It is a figure which shows 3 types of B picture types used for the encoding of the multi-view moving image by one Embodiment of this invention. It is a figure which shows 3 types of B picture types used for the encoding of the multi-view moving image by one Embodiment of this invention. It is a figure which shows 3 types of B picture types used for the encoding of the multi-view moving image by one Embodiment of this invention. FIG. 7 is a diagram illustrating a structure in which the unit coding structure of the multi-view video of FIG. 6 is horizontally expanded according to an embodiment of the present invention. FIG. 9 is a diagram illustrating a prediction order of the multi-view video in FIG. 8 according to an embodiment of the present invention. FIG. 6 is a diagram illustrating a moving picture coding structure for motion prediction and disparity prediction having an odd number of viewpoints according to an embodiment of the present invention. FIG. 3 is a diagram illustrating a moving picture coding structure for motion prediction and disparity prediction having an even number of viewpoints according to an embodiment of the present invention. 5 is a flowchart illustrating a multi-view video encoding process according to an embodiment of the present invention.

Claims (27)

Grouping a plurality of B frames into at least two groups according to a predetermined criterion;
And sequentially encoding the plurality of grouped B frames. 5. A multi-view video encoding method comprising: The predetermined criterion is:
2. The multi-view video encoding method according to claim 1, wherein each of the plurality of B frames is the number of frames referred to. The predetermined criterion is:
The multi-view video encoding method according to claim 1, wherein each of the plurality of B frames is the number of frames to be referenced and the position of the frame to be referenced. The plurality of grouped B frames are:
A first group predicted with reference to two horizontally adjacent frames, two vertically adjacent frames, or one horizontally adjacent frame and one vertically adjacent frame;
A second group predicted with reference to two horizontally adjacent frames and one vertically adjacent frame, or one horizontally adjacent frame and two vertically adjacent frames;
A third group predicted with reference to two horizontally adjacent frames and two vertically adjacent frames, and
The one or two horizontally adjacent frames are frames obtained from multi-view video at the same time, such as the B frame of the first group, the second group or the third group, The one or two vertically adjacent frames are frames obtained from a multi-view video from the same viewpoint, such as the B frame of the first group, the second group, or the third group. The multi-view video encoding method according to claim 1, wherein: The step of sequentially encoding the plurality of grouped B frames includes:
5. The multi-view video encoding method according to claim 4, further comprising a step of sequentially encoding the grouped B frames in order of a first group, a second group, and a third group. The sequential encoding is performed based on a moving image encoding structure including the plurality of B frames,
The sequential encoding includes performing disparity prediction between horizontally arranged frames by a plurality of viewpoints, and performing motion prediction between vertically arranged frames over time,
The moving picture coding structure is extended in at least one of horizontal and vertical directions,
The multi-view video according to claim 1, wherein the horizontal frame is obtained from a multi-view video of the same time, and the vertical frame is obtained from a multi-view video of the same view. Video encoding method.   The multi-view video encoding method according to claim 6, wherein the plurality of viewpoints include n viewpoints, and n is an odd number among natural numbers.   The multi-view video encoding method according to claim 7, wherein a frame obtained from the (n-1) -th viewpoint is not used for the parallax prediction and the motion prediction.   In the moving picture coding structure, a frame obtained from viewpoints excluding the first viewpoint among the plurality of viewpoints does not include an I frame, and a frame obtained from the kth includes only a B frame, 8. The multi-view video encoding method according to claim 7, wherein k is an even natural number smaller than n. A prediction unit that predicts a disparity vector and a motion vector for an input multi-view video,
A parallax motion compensation unit that compensates an image using the predicted parallax vector and motion vector;
A difference video encoding unit that receives the input multi-view video and the compensated video provided from the parallax motion compensation unit, and performs difference video encoding;
An entropy encoding unit that generates a bitstream for the multi-view video using the disparity vector, the motion vector, and the data on which the difference video encoding has been performed, and
The predicting unit groups each of a plurality of B frames into at least two groups according to a predetermined criterion, and sequentially predicts the plurality of grouped B frames. Encoding device. The predetermined criterion is:
The multi-view video encoding apparatus according to claim 10, wherein each of the plurality of B frames is the number of frames referred to. The predetermined criterion is:
The multi-view video encoding apparatus according to claim 10, wherein each of the plurality of B frames is a number of frames to be referred to and a position of the frame to be referred to. The plurality of grouped B frames are:
A first group predicted with reference to two horizontally adjacent frames, two vertically adjacent frames, or one horizontally adjacent frame and one vertically adjacent frame;
A second group predicted with reference to two horizontally adjacent frames and one vertically adjacent frame, or one horizontally adjacent frame and two vertically adjacent frames;
A third group predicted with reference to two horizontally adjacent frames and two vertically adjacent frames, and
The one or two horizontally adjacent frames are frames obtained from multi-view video at the same time, such as the B frame of the first group, the second group or the third group, The one or two vertically adjacent frames are frames obtained from a multi-view video from the same viewpoint, such as the B frame of the first group, the second group, or the third group. The multi-view video encoding apparatus according to claim 10. The prediction unit
The multi-view video encoding apparatus according to claim 13, wherein prediction is sequentially performed on each of the grouped B frames in order of a first group, a second group, and a third group. The prediction unit predicts the disparity vector and the motion vector of the input multi-view video based on a video encoding structure including the plurality of B frames;
The prediction unit performs a parallax prediction between frames arranged horizontally by a plurality of viewpoints, performs a motion prediction between frames arranged vertically over time,
The moving picture coding structure can be extended in at least one of horizontal and vertical directions,
The multi-view video according to claim 10, wherein the horizontal frame is obtained from a multi-view video of the same time, and the vertical frame is obtained from a multi-view video of the same view. Video encoding device.   The multi-view video encoding apparatus according to claim 15, wherein the plurality of viewpoints include n viewpoints, and n is an odd number among natural numbers.   The multi-view video encoding apparatus according to claim 16, wherein a frame obtained from the (n-1) -th viewpoint is not used for the parallax prediction and the motion prediction.   In the moving picture coding structure, a frame obtained from viewpoints excluding the first viewpoint among the plurality of viewpoints does not include an I frame, and a frame obtained from the kth includes only a B frame, 17. The multi-view video encoding apparatus according to claim 16, wherein k is an even natural number smaller than n. Grouping a plurality of B frames into at least two groups according to a predetermined criterion;
A step of sequentially encoding the plurality of grouped B frames, and a computer-readable program recording a multi-view video encoding method recoding media. The predetermined criterion is:
The recording medium according to claim 19, wherein each of the plurality of B frames is the number of frames referred to. The predetermined criterion is:
The recording medium according to claim 19, wherein each of the plurality of B frames is a number of frames to be referred to and a position of the frame to be referred to. The plurality of grouped B frames are:
A first group predicted with reference to two horizontally adjacent frames, two vertically adjacent frames, or one horizontally adjacent frame and one vertically adjacent frame;
A second group predicted with reference to two horizontally adjacent frames and one vertically adjacent frame, or one horizontally adjacent frame and two vertically adjacent frames;
A third group predicted with reference to two horizontally adjacent frames and two vertically adjacent frames, and
The one or two horizontally adjacent frames are frames obtained from multi-view video at the same time, such as the B frame of the first group, the second group or the third group, The one or two vertically adjacent frames are frames obtained from a multi-view video from the same viewpoint, such as the B frame of the first group, the second group, or the third group. The recording medium according to claim 19. The step of sequentially encoding the plurality of grouped B frames includes:
The recording medium according to claim 22, further comprising a step of sequentially encoding the grouped B frames in the order of a first group, a second group, and a third group. The sequential encoding is performed based on a moving image encoding structure including the plurality of B frames,
The sequential encoding includes performing disparity prediction between horizontally arranged frames by a plurality of viewpoints, and performing motion prediction between vertically arranged frames over time,
The moving picture coding structure is extended in at least one of horizontal and vertical directions,
The recording medium according to claim 19, wherein the horizontal frame is obtained from a multi-view video at the same time, and the vertical frame is obtained from a multi-view video at the same view.   The recording medium according to claim 24, wherein the plurality of viewpoints include n viewpoints, and n is an odd number among natural numbers.   The recording medium according to claim 25, wherein a frame obtained from the (n-1) th viewpoint is not used for the parallax prediction and the motion prediction.   In the moving picture coding structure, a frame obtained from viewpoints excluding the first viewpoint among the plurality of viewpoints does not include an I frame, and a frame obtained from the kth includes only a B frame, 26. The recording medium according to claim 25, wherein k is an even natural number smaller than n.

Images