JPH09261653A - Multi-view-point picture encoder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide view point number scalability while keeping high encoding efficiency by limiting a reference screen to the same view point picture in the encoding of a reference view point picture and limiting the reference screen to the same view point picture and an adjacent view point picture in the picture encoding of a view point other than a reference view point. SOLUTION: At the time of encoding the picture of the reference view point V2, only motion compensation is performed and the pictures of the other view points are not referred to. On the other hand, at the time of encoding the view point V1 on a left side, an already encoded screen at the view point V1 or V2 is turned to the reference screen and the encoding is performed. It is similar for the view point V3 on a right side as well. In such a manner, by using only the picture strings of the view points V1 and V2 or the view points V2 and V3, reproduction for only the two view points can be performed. That is, no matter which of 1, 2 and 3 the view point number of the pictures to be reproduced is, by using the picture strings for the same number as the view point number, the reproduction is made possible and the view point number scalability is realized.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a high-efficiency encoding apparatus for multi-view images, which uses motion compensation or parallax compensation prediction in high-efficiency image encoding.

[0002]

2. Description of the Related Art As a conventional method of motion compensation and parallax compensation (hereinafter referred to as compensation) of a binocular stereoscopic image, there is a method shown in FIG. In FIG. 20, V1 represents a moving image sequence for the left eye (hereinafter referred to as viewpoint V1), V2 represents a moving image sequence for the right eye (hereinafter referred to as viewpoint V2), and the arrow in the drawing indicates the direction of prediction. Show. In this case, the reference screen when encoding the viewpoint V2 is fixed to the image of the viewpoint V1.

If this is directly applied to the multi-view image coding, it cannot handle the case where the prediction error becomes smaller when another screen (for example, the screen one frame before the coded screen) is used. A conventional example for solving the problem is disclosed in JP-A-6-98312. FIG. 21 shows a compensating method in the conventional example, and the coding efficiency is improved by selecting the one having the smallest prediction error from the plurality of reference screens.

[0004]

However, FIG.
With the compensation method as shown in (4), since which image is used as the reference screen is determined only by the magnitude of the prediction error, the number-of-views scalability cannot be realized. Here, the viewpoint number scalability means that it is possible to reproduce only an arbitrary number of viewpoints from a multi-viewpoint image. Although FIG. 21 is an example of a three-view stereoscopic image, even if it is desired to reproduce only the image of the viewpoint V2, all the image sequences from the viewpoint V1 to the viewpoint V3 must be decoded.

Further, in the parallax compensation, the parallax vector search range (especially in the horizontal direction) needs to be wider than the general motion vector search range because the parallax changes greatly depending on the positional relationship between the camera and the subject. is there.

Therefore, it is an object of the present invention to provide a coding apparatus which realizes the number-of-views scalability while having high coding efficiency in multi-view image coding.
Another object of the present invention is to provide an encoding device capable of efficiently searching for a compensation vector in compensation prediction.

[0007]

In order to achieve the above object, the present invention provides three pattern matching units for performing pattern matching between an encoded screen and a reference screen to obtain a prediction error and a compensation vector, and the prediction error. , A minimum value selection unit that selects the compensation vector of the reference screen that minimizes, an encoder that encodes the prediction error between the prediction value from the reference screen and the encoded screen, and a plurality of units for holding the reference screen. And a standard viewpoint image is defined, the reference screen is limited to the same viewpoint image when encoding the standard viewpoint image, and the reference screen is the same viewpoint when encoding the viewpoint other than the standard viewpoint. It is characterized in that it is limited to the image and the adjacent viewpoint image close to the reference viewpoint.

Further, according to the present invention, in the encoding of the standard viewpoint image, the reference screen is only the frame one frame before the encoded screen, and in the image encoding of the viewpoint other than the standard viewpoint, the reference screen is encoded. There may be three screens one frame before the screen, the same frame screen adjacent to the reference viewpoint of the encoded screen, and the screen one frame before the adjacent viewpoint closer to the reference viewpoint of the encoded screen.

Further, in the present invention, the coded screen is divided into a plurality of blocks, the pattern matching unit performs pattern matching in block units, and the minimum value selecting unit selects a reference screen in block units. Good.

In the present invention, the minimum value selection unit may select the reference screen in frame units.

In the present invention, the number M of the frame memories is M = n + 2 (n = 2, 3, 4) M = n + 3 (n ≧ 5) when the number of viewpoints is n (n: natural number). be able to.

In the present invention, the reference viewpoint is the nth viewpoint when the number of viewpoints is 2n + 1 (n: natural number), and the nth or n + 1 when the number of viewpoints is 2n (n: natural number).
It may be the second viewpoint.

Further, according to the present invention, the order of performing compensation prediction is as follows: a screen one frame before the same viewpoint as the coded screen (motion compensation prediction), a screen of the same frame between the coded screen and an adjacent viewpoint (parallax compensation prediction), When the parallax compensation prediction and the motion parallax compensation prediction are performed in the order of the coding screen and the screen one frame before the adjacent viewpoint (motion parallax compensation prediction), the motion vector obtained in the motion compensation prediction and the parallax vector buffer are stored. You may make it move the search range of pattern matching using the parallax vector of.

Further, in the present invention, when performing the parallax compensation prediction, the motion vector obtained in the motion compensation prediction and the parallax vector in the parallax vector buffer are used to move the search range of the pattern matching to perform the motion parallax compensation prediction. When performing, the search range for pattern matching may be moved using the motion vector obtained by motion compensation prediction and the disparity vector obtained by disparity compensation prediction.

In the present invention, when performing the parallax compensation prediction and the motion parallax compensation prediction, the search range of pattern matching may be narrower than the search range of the motion compensation prediction.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described. FIG. 1 shows an example of a compensation method according to the present embodiment, and shows a case of a three-lens system. V1, V2, V
3 represents moving image sequences of different viewpoints, V2 is a central viewpoint (hereinafter referred to as viewpoint V2), V1 is a left viewpoint (hereinafter referred to as viewpoint V1), and V3 is a right viewpoint (hereinafter referred to as viewpoint V1). It is called viewpoint V3). The arrow indicates the direction of prediction. In the present invention, the reference viewpoint (viewpoint V2 in the case of FIG. 1)
It is characterized in that only the motion compensation is performed when the image is encoded, and images from other viewpoints are not referred to. Therefore, when reproducing only the reference viewpoint image, an image of another viewpoint is not required. On the other hand, when the viewpoint V1 is encoded, the screen already encoded at the viewpoint V1 or V2 is encoded as the reference screen. The same applies to the viewpoint V3. By doing so, it becomes possible to reproduce only two viewpoints by using only the image sequences of the viewpoints V1 and V2 or the viewpoints V2 and V3.

That is, regardless of whether the number of viewpoints of the image to be reproduced is 1, 2, or 3, reproduction can be performed by using only the same number of image sequences as the number of viewpoints, and the viewpoint number scalability is realized. .

FIG. 2 shows a block diagram of the coding apparatus according to the present embodiment. FIG. 2 shows an example in the case of the trinocular type. The coded screen P0 and the reference screens P1, P2, and P3 are input to the three pattern matching units 101 to 103. In the present embodiment, the encoded screen is divided into a plurality of blocks (one block is composed of 8 × 8 pixels or 16 × 16 pixels), and pattern matching units 101 to 103 perform matching in block units. The prediction error that is the matching result is output to the minimum value detection unit 104. The minimum value detection unit 104 selects the minimum output from the three pattern matching units and outputs a selection flag indicating which reference screen is selected. The selection unit 105 outputs the compensation vector corresponding to the selection flag from the minimum value detection unit 104.

As the encoder, a widely known one can be used, and the orthogonal transformation unit 106 and the quantization unit 10 can be used.
7, inverse quantizer 108, inverse orthogonal transformer 109, selector 1
15, 116, frame memories 110 to 114, a variable length coding unit 117, and a compensation unit 118. In an encoder used for normal moving image encoding, in the case of only forward prediction, the required frame memories are two frames for a reference screen and for writing a decoded image, but an encoder for a three-eye stereoscopic image. In this embodiment, a frame memory for 5 frames is required. This will be described later.

The selection unit 115 displays the reference screens P1, P2, P3.
The frame memory in which the image to be used as is stored is selected, and the screen used for compensation prediction is passed to the compensation unit 118 in block units based on the selection flag from the minimum value detection unit 104. The method of selecting the reference screen in the selection unit 115 is the main part of the present invention, and will be described later in detail together with the encoding procedure. In the compensation unit 118, the selection unit 10
Compensation prediction is performed based on the compensation vector output from 5. The selection unit 116 also selects, for each frame, a frame memory into which the locally decoded image generated by the inverse quantization unit 108 and the inverse orthogonal transform unit 109 is written. The variable length coding unit 117 also codes the prediction error output from the quantization unit 107, the selection flag output from the minimum value detection unit 104, and the compensation vector output from the selection unit 105. To be done.

Next, the encoding procedure will be described in detail with reference to FIG. FIG. 3 shows an example of the relationship between the encoding order and the reference screen when encoding the three-eye stereoscopic image as shown in FIG. First, the first screen 21 of the reference viewpoint V2 is intra-frame coded (step 1). Screen 21
At the same time as being encoded, it is decoded in the encoding device and stored in the frame memory. Next, the screen 11, the screen 31, and the screen 22 are encoded using the decoded image of the screen 21 as a reference screen (steps 2, 3, 4). As described above, according to the present invention, what is used as a reference screen is a decoded image of a screen encoded before the time of reference, and thereafter, the reference screen is a decoded image (not an original image). I do not particularly refuse.

Next, the screen 12 is encoded with reference to the screens 11, 21 and 22 (step 5). At this time, screen 12
Pattern matching is performed on a block-by-block basis between the screens 11, 21 and 22 and the screen with the smallest prediction error is set as the reference screen. That is, the reference screen is selected in block units. After that, the same processing may be performed. When the viewpoint V2 is encoded, there is one reference screen, and when the viewpoints V1 and V3 are encoded, there are three reference screens. Also, considering only the viewpoint V2, the motion compensation predictive coding is the same as the normal motion compensation predictive coding, and since the intra-frame coded screen appears at an appropriate cycle, the processing may be repeated by returning to the beginning of FIG. 2 each time. The selection of the reference screen is not limited to the block unit, and may be performed in the frame unit. When selecting in frame units, the reference screen is the one in which the sum of squares of prediction errors (prediction error in the entire screen) obtained by pattern matching in all block units is the smallest.

Here, the number of required frame memories will be described. Taking the case of step 5 in FIG. 3 as an example, the images stored in the frame memory at the time when step 4 ends are the screens 11, 21, 22, 31.
4 frames. Since the screen 12 is encoded in step 5, one frame memory for writing the decoded image of the screen 12 is required in addition to the above four frames during the process of step 5, and a total of five frame memory is required.

Since the screen 11 is no longer necessary after the end of step 5, the contents of the frame memory after the end of step 5 are the screens 12, 21, 22, 31. The screen 31 is unnecessary in step 5, but it is necessary to retain it because it is necessary in step 6. The encoding itself should have a frame memory for 4 frames in total, 3 frames for reference screen and 1 frame for writing decoded image.
As described above, a frame memory that is not used at that time but is required to hold a screen used in a later step is separately required. Therefore, considering this point, the number of viewpoints n and the number of frame memories required. The relationship with M is: M = n + 2 (n = 2, 3, 4) M = n + 3 (n ≧ 5; n is a natural number)

Next, an example of the compensation method in the case of the five-lens type is shown in FIG.
Shown in In this case, V3 is the reference viewpoint.

The block diagram of the encoder is almost the same as that shown in FIG. 2, except that the number of required frame memories is eight frames. In the present invention, since the number of reference screens is three or less regardless of the number of viewpoints, the number of pattern matching units may be three regardless of the number of viewpoints.

In FIG. 4, the portions of V2, V3 and V4 are the same as in FIG. 1, and it can be considered that V1 and V5 are added to the outside thereof. The viewpoint V1 selects the reference screen only from V1 itself or V2, and the viewpoint V5 selects the reference screen only from V5 itself or V4. That is, the present invention is characterized in that, when encoding the image sequence Vx of the viewpoint other than the standard viewpoint, Vx itself or an image sequence closer to the standard viewpoint than Vx is selected as the reference screen. In general, the reference screen for encoding the y-th image Pxy of the viewpoint Vx in the present invention is as shown in FIG. 5 when the standard viewpoint is Vx0. The multi-viewpoint image encoded by selecting the reference screen as shown in FIG. 4 can be reproduced with an arbitrary number of viewpoints by selecting the decoded image sequence as shown in FIG. That is, the number-of-views scalability can be realized.

FIG. 7 shows the coding procedure in the case of the 5-lens system.
The screen 31 of step 1 is intraframe coding. The reference viewpoint V3 and the first screen of each viewpoint have one reference screen, and the other screens have three reference screens. In FIG. 7, for example, the locally decoded image 11 generated in step 4 is not referred to in steps 5 to 8 but is referred to for the first time in step 9, so it is necessary to hold it during this period.
As can be seen from FIG. 7, the maximum number of images that need to be held in the frame memory is 7, and a total of 8 frames, which is one frame for writing, is
This is the number of frame memories required in the case of the 5-lens system.

Similarly, an example of the compensating method and the coding procedure in the case of the two-lens type is shown in FIGS. 8 and 9, and an example of the compensating method and the coding procedure in the case of the four-lens type is shown in FIGS. . The reference viewpoint is selected as viewpoint Vi when the number of viewpoints is 2i + 1 and as viewpoint Vi or Vi + 1 when the number of viewpoints is 2i. Here, i is a natural number. In the examples of FIGS. 8 to 11, the reference viewpoint is Vi.

Although the case where only the forward prediction is used has been described so far, the coding apparatus can be configured in the same way even when the bidirectional prediction is used.

Next, a second embodiment of the present invention will be described. FIG. 12 is a diagram for explaining the coding of the screen 32 when the three-eye stereoscopic image shown in FIG. 1 is coded.
When the screen 32 is encoded, pattern matching is performed between the screens 21, 22 and 31, and the screen having the smallest prediction error is used as the reference screen. In this pattern matching, the compensation vector is obtained, but it is necessary to change the search range depending on the screen to be referenced. This will be described in detail.

In the present invention, there are three types of compensation prediction methods depending on the relationship between the coding screen and the reference screen.
If the encoded screen in 2 is screen 32, screen 3
1 (motion-compensated prediction) from 1 (a screen one frame before the same viewpoint as the encoded screen), parallax-compensated prediction from screen 22 (a screen of the same frame from the viewpoint next to the encoded screen), screen 21 ( Prediction from the screen one frame before the viewpoint next to the encoded screen will be referred to as motion parallax compensation prediction. Also, the compensation vector obtained in each compensation prediction is referred to as a motion vector, a parallax vector, and a moving parallax vector, respectively. The pattern matching is performed in the order of screen 32 and screen 31 (motion compensation prediction), screen 32 and screen 22 (parallax compensation prediction), and screen 32 and screen 21 (moving parallax compensation prediction) (block unit or frame). unit).

In the case of motion vector search (for example, pattern matching between the screen 32 and the screen 31 is performed), since the amount of motion between frames is not so large, for example, ± 15 pixels in both the horizontal and vertical directions. It suffices to search a relatively narrow square area. In FIG. 12, in order to perform matching of the block located at the position d on the screen 32, a square area having an appropriate size centered on the position d is searched on the screen 31. However, in the case of parallax vector search (for example, pattern matching between the screen 32 and the screen 22), since the horizontal component of the parallax vector is generally larger than the motion vector, a search range similar to that of the motion vector is set. However, correct pattern matching cannot be performed.

In FIG. 12, the position d on the screen 32 is
In the matching of the blocks in the position c (screen 22
Although the upper) should be the optimum solution, the position c is outside the search range set by the motion vector search. In order to deal with this, it is conceivable to expand the search range of the parallax vector in the horizontal direction. Also in the present embodiment, when the top screen of the image sequence (screen 31 in FIG. 12) is encoded, the parallax vector is obtained using the rectangular search range expanded in the horizontal direction. Although parallax vectors are obtained for all blocks on the first screen, the parallax vectors for one screen are stored in the parallax vector buffer in this embodiment. Since the parallax vector generally has a small vertical component (ideally 0), the vertical search range may be smaller than the vertical search range of the motion vector, for example, ± 5 pixels.

As described above, when pattern matching (parallax compensation prediction) between the screen 32 and the screen 22 is performed, searching for a square area centered on a block to be matched does not work, as described above. The parallax vector stored in the parallax vector buffer is used in. Considering a certain block d on the screen 32, the block most matching with this is required in the motion compensation prediction (pattern matching between the screen 32 and the screen 31) performed previously (the one showing this correspondence is the motion vector). Is). If it is found from the obtained motion vector that the block d corresponds to the block b on the screen 31, the parallax vector corresponding to the block b on the screen 31 is stored in the parallax vector buffer. Screen 3 for the search center of the target block
Move to the position indicated by the parallax vector on 1 and search for a square region centered there. In this way, after moving the search center by the disparity vector, it can be handled in the same manner as a normal motion vector search. In the case of performing pattern matching (motion disparity compensation prediction) between the screen 32 and the screen 21, similarly, the search center is moved by the disparity vector, and a square area centered there is searched.

Of the three types of compensation vectors (motion vector, parallax vector, and motion parallax vector) thus obtained, the one with the smallest prediction error is output as the final compensation vector for the block. Also, the screen 32
Since a new disparity vector is obtained for all macroblocks by pattern matching on the screen 22 and the contents of the disparity vector buffer is updated using this.

FIG. 13 shows a block diagram of the coding apparatus according to the present embodiment. The coding apparatus of FIG. 13 has two parallax vector buffers 120 in addition to the coding apparatus of FIG.
121 and selectors 122 and 123 are added.

The pattern matching units 101 to 103 in FIG. 2 are the motion compensation pattern matching unit 131 and the motion parallax compensation pattern matching unit 13 in FIG.
2. It is replaced by the parallax compensation pattern matching unit 133. Since the encoding apparatus of FIG. 13 is a trinocular system, two parallax vector buffers are required. Generally, when the number of viewpoints is i, the required number of parallax vector buffers is i−1.
The selection unit 122 selects the parallax vector buffer in which the newly obtained parallax vector should be written. FIG. 14 shows an example of the coding order and the contents of the disparity vector buffer according to the present embodiment. In FIG. 14, the buffer A is for the viewpoint V1, and the buffer B is for the viewpoint V3, and the contents of the buffer are updated each time an image belonging to each viewpoint is encoded. Note that the notation 11-21 in FIG. 14 means a disparity vector from the screen 11 to the screen 21.

When encoding one screen, first, the motion compensation pattern matching unit 131 performs motion compensation,
Based on the result, the selection unit 123 obtains the corresponding parallax vector from the parallax vector buffer by the parallax compensation pattern matching unit 133 and the moving parallax compensation pattern matching unit 132.
Output to In the parallax compensation pattern matching unit 133 and the moving parallax compensation pattern matching unit 132, the search center is moved by the transmitted parallax vector to perform pattern matching.

In the present embodiment, the search centers in the parallax compensation prediction and the motion parallax compensation prediction are the same, but the sizes of the search ranges do not have to be the same. FIG. 15 shows an example in which the size of the search range is changed in both compensation predictions. FIG.
Corresponds to the portion of the screen 32 in FIG. In the parallax compensation prediction, the vertical component of the parallax vector is small, so the vertical search range can be narrowed, and since the approximate position can be estimated by using the parallax vector of the previous frame, the horizontal direction is not so much. There is no need to search a wide range. This means that the search time can be shortened or the search results can be highly accurate.
Of course, the search area need not be square. In addition,
In the motion parallax compensation prediction in this case, even if the parallax vector of the previous frame is used, a search range similar to that of normal motion compensation is required.

Next, a third embodiment of the present invention will be described. FIG. 16 is a diagram for explaining the coding of the screen 32 when the three-eye stereoscopic image shown in FIG. 1 is coded.
16 corresponds to the part of the screen 32 in FIG.
There are three types of compensation methods and compensation vectors as shown in FIG. 2, and the pattern matching is performed in the order of motion compensation prediction, parallax compensation prediction, and motion parallax compensation prediction (block unit or frame unit). ).

Also in FIG. 16, motion compensation prediction is first performed, and in the subsequent parallax compensation prediction, pattern matching is performed by moving the search center by the parallax vector using the parallax vector of the previous frame, as described with reference to FIG. Is the same as. However, in the present embodiment, the method of setting the search range in the motion disparity compensation prediction is as shown in FIG.
Different from the method explained in. In performing the motion parallax compensation prediction in FIG. 16, pattern matching is performed with the search center in the parallax compensation prediction moved by the parallax vector of the previous frame as the search center and the point further moved by the motion vector already obtained. To do. By moving the search center in this way, the approximate position of the optimum solution can be estimated even in the motion parallax compensation prediction, so the search range in the motion parallax compensation prediction is smaller than the search range used in normal motion compensation. can do. That is, it is possible to shorten the search time or improve the accuracy of the search result.

FIG. 17 shows a block diagram of an encoding apparatus for realizing the compensation method as shown in FIG. The difference from the encoding device shown in FIG. 13 is that the output from the motion compensation pattern matching unit 31 (before the output from the selection unit 123 (selected parallax vector) is input to the motion parallax compensation pattern matching unit 132 ( Motion vector) is added.

Further, in FIG. 16, the search center in the motion parallax compensation prediction is a point moved by "the parallax vector of the previous frame + the motion vector", but the parallax compensation prediction of the current frame is used instead of the parallax vector of the previous frame. It is also possible to use the parallax vector newly obtained in. FIG.
FIG. 8 is a diagram for explaining this method, in which the search center in the motion parallax compensation prediction is set as a point moved by the “newly obtained parallax vector + motion vector”. By doing so, the search center in the motion parallax compensation prediction can be brought to a position closer to the optimal solution, and the search range can be further reduced. That is, it is possible to more effectively reduce the search time or improve the accuracy of the search result.

FIG. 19 shows a block diagram of an encoding apparatus for realizing the compensation method as shown in FIG. The difference from the encoding device shown in FIG. 17 is that the output of the parallax compensation pattern matching unit 133 (parallax vector) is the output of the motion compensation pattern matching unit 131 (motion vector) as an input to the motion parallax compensation pattern matching unit 132. Is added. In the compensation method of FIG. 18, the parallax vector of the previous frame stored in the parallax vector buffer is used only for determining the search center in the parallax compensation prediction, and is not used in the moving parallax compensation prediction.

[0046]

According to the multi-view image coding device of the present invention, three pattern matching units for performing pattern matching between a coded screen and a reference screen to obtain a prediction error and a compensation vector, and the prediction error are minimized. A minimum value selection unit that selects a compensation vector of the reference screen, an encoder that encodes a prediction error between the prediction value from the reference screen and the encoded screen, and a plurality of frames for holding the reference screen In addition to the memory, the standard viewpoint image is defined, in the standard viewpoint image encoding, the reference screen is limited to the same viewpoint image, and in the viewpoint image encoding other than the standard viewpoint,
Since the reference screen is limited to the same-viewpoint image and the adjacent-viewpoint image close to the standard viewpoint, the number-of-views scalability can be realized while maintaining high coding efficiency.

Further, since three reference screens are selected in block units, the prediction error can be further reduced and high coding efficiency can be realized.

Further, a parallax vector buffer is provided, and the order of performing compensation prediction is motion compensation prediction, parallax compensation prediction, and motion parallax compensation prediction. When performing parallax compensation prediction and motion parallax compensation prediction, motion compensation prediction is performed. Since the search range of the pattern matching is moved using the motion vector obtained by the compensation prediction and the disparity vector in the disparity vector buffer, the disparity vector and the moving disparity vector can be efficiently searched.

Further, since the motion vector obtained by the motion compensation prediction and the parallax vector obtained by the parallax compensation prediction are used to move the search range of the pattern matching when the motion parallax compensation prediction is performed, You can search more efficiently.

[Brief description of drawings]

FIG. 1 is a diagram showing a method of compensating and predicting a three-eye stereoscopic image according to the present invention.

FIG. 2 is a block diagram of an encoding device that implements the compensation prediction method of FIG.

FIG. 3 is a diagram illustrating an encoding procedure and a reference screen of the compensation prediction method of FIG.

FIG. 4 is a diagram showing a method of compensating and predicting a five-eye stereoscopic image according to the present invention.

FIG. 5 is a diagram showing a relationship between a viewpoint and a reference screen according to the present invention.

FIG. 6 is a diagram showing a relationship between the number of reproduction viewpoints and a decoded image according to the present invention.

7 is a diagram illustrating an encoding procedure and a reference screen of the compensation prediction method of FIG.

FIG. 8 is a diagram illustrating a method of compensating and predicting a twin-lens stereoscopic image according to the present invention.

9 is a diagram illustrating an encoding procedure and a reference screen of the compensation prediction method of FIG.

FIG. 10 is a diagram showing a compensation prediction method for a four-eye stereoscopic image according to the present invention.

11 is a diagram illustrating an encoding procedure and a reference screen of the compensation prediction method of FIG.

FIG. 12 is a diagram illustrating three types of compensation prediction methods according to the present invention.

13 is a block diagram of an encoding device that implements the compensation prediction method of FIG.

[Fig. 14] Fig. 14 is a diagram for describing the encoding procedure and disparity vector buffer contents of the compensation prediction method in Fig. 12.

15 is a diagram illustrating a search range in the compensation prediction method of FIG.

[Fig. 16] Fig. 16 is a diagram for describing a method of setting a search range in motion parallax compensation prediction.

17 is a block diagram of an encoding device that realizes the compensation prediction method of FIG.

[Fig. 18] Fig. 18 is a diagram for describing another method of setting a search range in motion parallax compensation prediction.

19 is a block diagram of an encoding device that implements the compensation prediction method of FIG.

[Fig. 20] Fig. 20 is a diagram illustrating a conventional method for compensating and predicting a two-eye stereoscopic image.

FIG. 21 is a diagram illustrating a conventional multi-view stereoscopic image compensation prediction method.

[Explanation of symbols]

 P0 coded screen P1, P2, P3 reference screen V1, V2, V3, V4, V5 viewpoint 11 to 53 frame screen 104 minimum value detection unit 101, 102, 103 pattern matching unit 105, 115, 116, 122, 123 selection unit 106 Orthogonal Transform Unit 107 Quantizer 108 Inverse Quantizer 109 Inverse Orthogonal Transform Unit 110-114 Frame Memory 117 Variable Length Encoder 118 Compensator 120, 121 Parallax Vector Buffer 131 Motion Compensation Pattern Matching Unit 132 Motion Parallax Compensation Pattern Matching Part 133 Parallax compensation pattern matching unit

Claims (10)

[Claims] 1. A prediction error and a compensation vector are obtained by performing pattern matching between an encoded screen and a reference screen.
One pattern matching unit, and a minimum value selection unit that selects a compensation vector of the reference screen that minimizes the prediction error,
An encoder that encodes the prediction error between the predicted value from the reference screen and the encoded screen, and a plurality of frame memories for holding the reference screen are provided. In the encoding, the reference screen is limited to the same viewpoint image, and in the image encoding of the viewpoints other than the standard viewpoint, the reference screen is limited to the same viewpoint image and the adjacent viewpoint image close to the standard viewpoint. View image encoding device. 2. In the encoding of the reference viewpoint image,
The reference screen is only the screen one frame before the encoded screen,
In image encoding of a viewpoint other than the standard viewpoint, the reference screen is a frame one frame before the encoded screen, the same frame screen is adjacent to the standard viewpoint of the encoded screen, and the adjacent frame is closer to the standard viewpoint of the encoded screen. The multi-view image encoding device according to claim 1, wherein there are three screens one frame before the viewpoint. 3. The pattern matching unit divides an encoded screen into a plurality of blocks and performs pattern matching in block units, and the minimum value selection unit selects a reference screen in block units. The multi-view image encoding device according to claim 1. 4. The pattern matching unit divides an encoded screen into a plurality of blocks and performs pattern matching in block units, and the minimum value selecting unit selects a reference screen in frame units. The multi-view image encoding device according to claim 1. 5. The number M of the frame memories is the number of viewpoints n
The multi-view image encoding device according to claim 1, wherein, in the case of (n: natural number), M = n + 2 (n = 2, 3, 4) M = n + 3 (n ≧ 5). 6. The number of viewpoints is 2n + 1 (n:
The multi-view image encoding device according to claim 1, wherein the viewpoint is the n-th viewpoint in the case of a natural number) and the n-th or n + 1-th viewpoint in the case of the viewpoint number 2n (n: natural number). 7. A prediction error and a compensation vector are obtained by performing pattern matching between an encoded screen and a reference screen.
One pattern matching unit, and a minimum value selection unit that selects a compensation vector of the reference screen that minimizes the prediction error,
An encoder that encodes the prediction error between the predicted value from the reference screen and the encoded screen, a plurality of frame memories for holding the reference screen, and a disparity vector obtained in the previous frame of the encoded screen are stored And a parallax vector buffer for defining a standard viewpoint image, and in encoding the standard viewpoint image, the reference screen is only the screen one frame before the encoded screen, and the image code of the viewpoint other than the standard viewpoint is used. In conversion,
There are three reference screens, a screen one frame before the encoded screen, a same frame screen adjacent to the reference viewpoint of the encoded screen, and a frame one frame before the adjacent viewpoint closer to the reference viewpoint of the encoded screen. The order in which the prediction is performed is one frame before the encoded screen and one frame before the same viewpoint (motion compensation prediction), the same frame between the encoded screen and the adjacent viewpoint (parallax compensation prediction), and one frame between the encoded screen and the adjacent viewpoint. When performing the parallax compensation prediction and the motion parallax compensation prediction in the order of the previous screen (motion parallax compensation prediction), the motion vector obtained by the motion compensation prediction and the parallax vector in the parallax vector buffer are used to perform pattern matching. A multi-view image coding device characterized by moving a search range. 8. The multi-view image code according to claim 7, wherein, when performing the parallax compensation prediction, a search range for pattern matching is set to an area narrower than a search range for motion compensation prediction. Device. 9. A prediction error and a compensation vector are obtained by performing pattern matching between an encoded screen and a reference screen.
One pattern matching unit, and a minimum value selection unit that selects a compensation vector of the reference screen that minimizes the prediction error,
An encoder that encodes the prediction error between the predicted value from the reference screen and the encoded screen, a plurality of frame memories for holding the reference screen, and a disparity vector obtained in the previous frame of the encoded screen are stored And a parallax vector buffer for defining a standard viewpoint image, and in encoding the standard viewpoint image, the reference screen is only the screen one frame before the encoded screen, and the image code of the viewpoint other than the standard viewpoint is used. In conversion,
There are three reference screens, a screen one frame before the encoded screen, a same frame screen adjacent to the reference viewpoint of the encoded screen, and a frame one frame before the adjacent viewpoint closer to the reference viewpoint of the encoded screen. The order in which the prediction is performed is one frame before the encoded screen and one frame before the same viewpoint (motion compensation prediction), the same frame between the encoded screen and the adjacent viewpoint (parallax compensation prediction), and one frame between the encoded screen and the adjacent viewpoint. In the order of the previous screen (moving parallax compensation prediction), when performing parallax compensation prediction, the search range for pattern matching is moved using the motion vector obtained by motion compensation prediction and the parallax vector in the parallax vector buffer. When moving-parallax-compensated prediction is performed, the search range for pattern matching is moved using the motion vector obtained by motion-compensated prediction and the parallax vector obtained by parallax-compensated prediction. Multi-view image encoding apparatus. 10. The pattern matching search range is set to an area narrower than the search range used in the motion compensation prediction when performing the parallax compensation prediction and the motion parallax compensation prediction. Multi-view image coding device.