JP3080487B2 - Motion compensation prediction method for multi-view stereoscopic video - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

(Contents) Industrial Application Field Conventional Technology (FIGS. 19 to 25) Problems to be Solved by the Invention Means for Solving the Problems (FIG. 1) Function (FIG. 1) Embodiment First Embodiment Description of Examples (FIGS. 2 to 12)-Description of Second Embodiment (FIGS. 13 to 18)-Other Effects of the Invention

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a motion compensation prediction method for a multi-view stereoscopic video, which performs motion compensation on a multi-view stereoscopic video of two or more eyes.

[0003]

2. Description of the Related Art FIG. 19 shows a conventional multi-view stereoscopic video system. In FIG. 19, reference numeral 102 denotes a camera, and a plurality of cameras 102 are provided.
Each camera 102 captures an image of the subject 101, and the positions of the cameras 102 are slightly shifted vertically and horizontally.

The reason why the position of the camera 102 is slightly shifted vertically and horizontally is that an output from one camera is used as an input to one eye to form binocular parallax so that stereoscopic vision can be obtained. That's why. Therefore, by using a large number of cameras 102 in this way, a person viewing the display 107 in the output system can obtain a natural stereoscopic view even if the position at which the display 107 is viewed is changed.

[0005] Reference numeral 103 denotes an encoder.
03 encodes image data input from each camera 102. 104 is multiplexing (multiplex)
The multiplexer 104 multiplexes the encoded image data in order to transmit the image data encoded by the encoder 103 on a transmission path.
Reference numeral 105 denotes a demultiplexer. The demultiplexer 105 separates multiplexed image data input via a transmission path. Reference numeral 106 denotes a decoder, which decodes the image data separated by the separator 105 in order to display the image data on a display 107.

As the display 107, a lenticular lens (when there is parallax only in the horizontal direction) or a fly's eye lens (when there is parallax in the vertical and horizontal directions) is used as an example. With such a configuration, the above-described multi-view stereoscopic video system has the following operation.

First, a still or moving subject 101 is photographed by a plurality of cameras 102 whose positions are slightly shifted vertically and horizontally. Next, the image data obtained from the plurality of cameras 102 is encoded with high efficiency and multiplexed (multiplexed) by the multiplexer 104. After that, the signal is transmitted through a transmission path or the like, and is separated on the receiving side by a separator 10.
5 after demultiplexing (demultiplexing) by the decoder 10
The decoding is performed by 6 and displayed on the display 107.

[0008] As an example of an image projected on the display 107, FIG.
The output from each of a camera with five eyes vertically and five eyes horizontally is shown.
In this example, it can be seen that there is also a parallax in the vertical direction. Further, as another example, FIG. 21 shows an example of a system for forming a holographic stereogram. 21 is the same as the multi-view stereoscopic video system shown in FIG. 19 up to the point where decoding is performed by the decoder 106.

Reference numeral 108 denotes an image size conversion unit. The image size conversion unit 108 converts the image size as a pre-process of the hologram phase calculation by the phase calculation unit 109. The phase calculation unit 109 performs hologram phase calculation for outputting to the holographic output system 110 such as a display after image size conversion. With such a configuration, the holographic stereogram has the same operation as that of the multi-view stereoscopic video system shown in FIG. 19 until decoding by the decoder 106 is performed.

However, after decoding is performed by the decoder 106, the image size is converted by the image size conversion unit 108 in order to calculate the hologram phase by the phase calculation unit 109. After that, the phase calculation section 109 calculates the phase, and the holographic output system 110 outputs an image. By the way, in realizing a multi-view stereoscopic image, it is necessary to compensate for screen motion.
There is a method of performing block matching prediction in the time direction and block matching prediction (parallax compensation) with an image of a camera at another position at the same time.

FIG. 22 is a diagram showing the motion compensation method according to the present invention.
In this case, block matching prediction in the time direction, which is an example of the former, is performed. The block matching prediction in the time direction is a block matching between the past image and the input image, and the difference between the original screen and the reference screen is only time. Therefore, the method is the same from the monocular system to the multi-view system. In FIG. 22, reference numeral 201 denotes a block matching and vector detection unit. The block matching and vector detection unit 201 converts each block of the past image and the input image, which is divided into a block size of, for example, 16 × 16 pixels. By using this, a motion vector is detected by a method described later.

FIG. 23 is a diagram showing the above-mentioned block matching and vector detecting section 201 in detail. In FIG. 23, reference numeral 208 denotes a memory;
08 corresponds to the memory 208 in FIG. 22 and stores encoded image data relating to the reference screen. Reference numeral 209 denotes a square error calculator. The square error calculator 209 calculates a square error between the input screen and the reference screen.
And a square calculator 209-2, an adder 209-3, and a delay element (D1) 209-4.

Reference numeral 210 denotes a vector conversion unit, which converts the encoded image data of the reference screen into a vector. Reference numeral 211 denotes a vector detection unit.
The square error calculating unit 209 detects a vector that minimizes the value calculated by accumulating the square error between the input screen and the reference screen. The comparison unit 211-1 and the delay element (D2) 211- 2, 211-5 and selector 211-
3, 211-4.

By the way, reference numeral 202 in FIG. 22 denotes a variable delay section, and this variable delay section 202 variably delays a reference screen. Reference numeral 203 denotes a first calculation unit. The first calculation unit 203 includes an input screen and a variable delay unit 202.
Is used to calculate the difference from the reference screen that has been variably delayed. Reference numeral 204 denotes an encoding unit. The encoding unit 204 encodes input image data. 205
Is a variable-length coder (VLC). The variable-length coder 205 allocates a small number of bits to a code word having a high frequency and a large number of bits to a code word having a low frequency. The encoding is performed by reducing the average number of bits per codeword to reduce the encoding bit rate.

Reference numeral 206 denotes a local decoding unit. The local decoding unit 206 performs variable-length encoding on the image data encoded by the encoding unit 204 by a variable-length encoding unit 205, and also serves as a reference screen. Is also used for local decoding. 207 is the second
The second arithmetic unit 207 performs a predetermined arithmetic operation on the image data from the local decode unit 206 and the image data from the variable delay unit 202,
And to the variable delay unit 202.

Reference numeral 208 denotes a memory.
Stores the encoded image data relating to the reference screen. With such a configuration, in the conventional motion compensation method, block matching in a block having an input screen and a reference screen is performed by the block matching and vector detection unit 201.

That is, the block matching and vector detection unit 201 calculates the square error between the reference screen and the input screen stored in the memory 208 in FIG.
Calculated by the squared error calculator 209. The vector detector 211 detects and outputs a vector that minimizes the square error. Here, the vector input to the selector 211-4 in the vector detection unit 211 is obtained by converting the image data from the reference screen into a vector in the vector conversion unit 210.

After the block matching, the motion vector is detected by the block matching and vector detecting section 201, and the reference screen is variably delayed by the variable delay section 202.
The difference from the input screen is obtained by the first calculation unit 203, and this is used as a prediction screen. This prediction screen is encoded by the encoding unit 204, and then the variable length encoder (VLC) 2
05 and is transmitted after being variable-length coded.

In order to make a reference screen, the encoding unit 2
The local decoding unit 206 performs local decoding on the image data encoded in step 04. In addition,
There is also a concept of global motion compensation using the above-described motion compensation. Global motion compensation is camera pan,
It compensates for the overall movement of the screen due to zooming or the like. For example, motion compensation is locally performed for each block, and thereafter, three parameters of vertical and horizontal panning and zooming are extracted by a matrix operation.
When actually used, processing and transmission are performed by combining the parameters of global motion compensation and the parameters of local motion compensation.

FIG. 24 is a block diagram showing an example of the latter parallax compensation in the above-described motion compensation method. In FIG. 24, reference numeral 212 denotes a block matching unit. Using the input image on the right eye side (original image or encoded reproduced image) as a reference screen, block matching with the input image on the left eye side is performed.

Reference numeral 213 denotes a parallax compensation vector detection unit.
The disparity compensation vector detection unit 213 detects a disparity compensation vector after performing block matching in the block matching unit 212. Therefore, the block matching unit 212 and the parallax compensation vector detection unit 213 have the same functions as those of the block matching and vector detection unit 101 in FIG.

Reference numeral 214 denotes a variable delay unit which, like the variable delay unit 102 shown in FIG. 22, variably delays an input image on the right eye side as a reference screen. Reference numeral 215 denotes a subtraction unit.
Reference numeral 15 denotes a variable delay unit 2 that, after encoding the input image by the encoding unit 218a or the encoding unit 218b on the right eye side, encodes the corresponding coded block on the right eye side according to the disparity compensation vector.
14 to obtain a difference signal from the input image on the left eye side. Reference numeral 216 denotes an encoding unit. The encoding unit 216 encodes the image data obtained by subtracting the difference in the subtraction unit 215.

Reference numerals 217 and 220 denote variable-length coding units. The variable-length coding units 217 and 220 perform the same variable-length coding as the variable-length coding unit 105 in FIG. Here, the variable length coding unit 217 performs variable length coding on left eye image data, and the variable length coding unit 220 performs variable length coding on right eye image data.

Reference numerals 218a and 218b denote encoding sections. The encoding sections 218a and 218b encode the image data in the same manner as the encoding section 216. Position varies according to That is, when the right-eye input image to be input to the block matching unit 212 as the reference screen is a coded image, the coding unit 2
18a, and before encoding, that is, when an original image is used, an encoding unit 218b is included.

In such a configuration, the block matching and the detection of the disparity compensation vector in the conventional disparity compensation have operations as shown in a flowchart of FIG. That is, step S1 in FIG.
In the right eye input image FR (X _OR, Y _OR) and referring to, in which the left eye input image FL (X _OL, Y _OL) the parallax compensation. The search range is ± Sx in the x direction and ± Sx in the y direction.
Sy. The size of the block is defined as Bx,
Let it be By in the y direction.

Then, steps S2 and S3
Then, the initial value of the coordinate displacement that varies per block in the right-eye input image FR ( _XOR , _YOR ) and the left-eye input image FL ( _XOL , _YOL ) is set. That is, J = 1 is set at the Y coordinate, and I = 1 at the X coordinate. Thereafter, a vector that minimizes the accumulation of the square error is selected over steps S4 to S12. That is, in steps S4 to S6, the cumulative value of the square error of the vector at the initial coordinate value is set, and
From step 7 to step S21, the minimum value of the vector is determined.

Then, in step S12, the minimum value of the determined vector is output as a disparity compensation vector.
The process ends. When the block matching and the disparity compensation vector are detected in this manner, after the right-eye-side input image is encoded by the encoding unit 218a or 218b, the right-eye-side encoded corresponding block is subjected to the variable delay unit 214 according to the disparity compensation vector. cut.

Thereafter, a subtraction unit 215 obtains a difference signal from the input image on the left eye side. The obtained difference signal is encoded by the encoding unit 216 and then variable-length encoded by the variable-length encoding unit 217 and transmitted. For the right eye side as well, variable length coding is performed by the variable length coding unit 220 before transmission.

[0029]

However, in such conventional motion compensation methods and disparity compensation methods, the pan,
In a scene in which zooming or the like occurs violently, an uncovered background (Uncovered Ba) is used.
ckround) occurs, and accurate motion compensation cannot be performed, which causes a problem of lowering the coding efficiency.

Here, the uncovered background means that an area hidden on the reference screen, that is, an area which is not the area of the screen appears on the input screen. The present invention has been made in view of such a problem, and even when an uncovered background occurs, encoding is performed by reducing the prediction error by making the reference screen and the input screen in motion compensation close to each other. An object of the present invention is to provide a motion compensation prediction method for a multi-view stereoscopic video, which can improve efficiency.

[0031]

FIG. 1 is a block diagram showing the principle of the present invention. In FIG. 1, reference numeral 2 denotes a switch. Switching is performed so as to select one image from images (past ones), and the selected image is output to the motion compensation prediction unit 3.

Reference numeral 3 denotes a motion compensation prediction unit, which performs motion compensation for multi-view stereoscopic video of two or more eyes, and outputs a current image output and a past image output. Has been entered. Reference numeral 4 denotes a block matching unit. The block matching unit 4 determines a motion compensation vector by performing block matching using a current image output as an input screen and a past image output as a reference screen. Further, a local motion compensation vector obtained by removing the global motion compensation vector parameter from the motion compensation vector is obtained.

Reference numeral 5 denotes a global vector detector, which calculates global motion compensation vector parameters based on the motion vectors detected by the block matching unit 4. Reference numeral 6 denotes an uncovered background prediction unit. The uncovered background prediction unit 6 detects an uncovered background area based on the global vector parameters detected by the global vector detection unit 5.

More specifically, the uncovered background prediction unit 6 uses the zoom parameter of the global motion compensation vector parameter detected by the global vector detection unit 5 when reducing the screen from the enlarged state. The periphery of the screen is detected as an uncovered background area. When moving the screen, the uncovered background prediction unit 6 uses the pan parameters of the global motion compensation vector parameters detected by the global vector detection unit 5 to unwrap the peripheral part of the moving screen. Detected as a covered background area.

Further, the uncovered background prediction unit 6 uses the local motion compensation vector obtained by the block matching unit 4, and calculates the local motion compensation vector of the block currently being processed by the local motion compensation vector of the surrounding block. Compared with a typical motion compensation vector,
For example, when the number of the differences exceeding the first threshold exceeds the second threshold, it is possible to detect the current block to be processed as an uncovered background area.

Reference numeral 7 denotes a reference screen determination unit. The reference screen determination unit 7 performs processing when using the past output from another camera for the uncovered background area detected by the uncovered background prediction unit 6. By referring to the position of the block to be tried, a camera from which the past output is used is determined. Also, the reference screen determination unit 7 performs global motion compensation on a block to be processed when using a past output from another camera for the uncovered background area detected by the uncovered background prediction unit 6. It is also possible to configure so as to determine which camera from which past output is to be used by referring to the vector (claims 1 to 7 above).

[0037]

In the motion compensation prediction method for a multi-view stereoscopic video of the present invention described above, when performing motion compensation prediction for a multi-view stereoscopic video of two or more eyes, first, the current output from the corresponding camera and the past output are output. , The block matching unit 4 and the global motion compensation vector detection unit 5 calculate a motion compensation vector and a global motion compensation vector parameter.

Further, a local motion compensation vector obtained by removing the global motion compensation vector parameter from the motion compensation vector by the block matching unit 4 is obtained.
The uncovered background prediction unit 6 detects an uncovered background area using the global motion compensation vector parameter. Then, the reference screen determination unit 7 and the motion compensation prediction unit 3 perform the motion compensation prediction on the uncovered background area using the past output from another camera (claim 1).

At this time, when detecting the uncovered background area using the global motion compensation vector parameter, the uncovered background prediction unit 6 uses the global motion compensation A peripheral portion of the screen is detected as an uncovered background area using the zoom parameter of the vector parameter (claim 2).

The uncovered background prediction unit 6 uses the global motion compensation vector parameter to detect the uncovered background area and, when moving the screen, sets the pan parameter of the global motion compensation vector parameter. Then, the peripheral part of the moving screen is detected as an uncovered background area (claim 3).

The uncovered background prediction unit 6 uses the local motion compensation vector obtained by the block matching unit 4, and calculates the local motion compensation vector of the block currently being processed by the local motion compensation vector of the surrounding block. If the difference exceeds the first threshold, for example, if the difference exceeds the second threshold, the current block to be processed is uncovered back. It is detected as a ground area (claims 4 and 5).

The reference screen determination unit 7 refers to the position of the block to be processed when using the past output from another camera for the uncovered background area detected by the uncovered background prediction unit 6. Then, it is determined which camera uses the past output (claim 6). Further, the reference screen determination unit 7
When the past output from another camera is used for the uncovered background area detected by the uncovered background prediction unit 6, reference is made to the global motion compensation vector of the block to be processed. It is also possible to determine whether to use the past output from.

[0043]

Embodiments of the present invention will be described below with reference to the drawings. (A) Description of First Embodiment FIG. 2 is a block diagram showing a first embodiment of the present invention. In FIG. 2, reference numeral 12 denotes a switch.
Receives a signal from the reference screen determination unit 17 and outputs n camera outputs (or a local decode screen; hereinafter, when simply referred to as a camera output, includes the case of this local decode screen) 1a-1 to 1a-. n, camera output 1a
The camera output of −k is selected, switched, and output to the motion compensation prediction unit 30.

The motion compensation predicting section 30 performs motion compensation for a multi-view stereoscopic image of two or more eyes, and the camera outputs 1a-1 to 1a-n are input by switching by the switch 12. ing. The motion compensation prediction unit 30 has the same function as that of the above-described block diagram showing the conventional motion compensation method shown in FIG. 22, and includes a block matching unit 31, a variable delay unit 32, and a first
Operation unit 33, encoding unit 34, and variable length encoding unit (VLC)
35, a local decoding unit 36, a second arithmetic unit 37, and a memory 38.
Block matching unit and vector detection unit 10
1, variable delay unit 102, first operation unit 103, encoding unit 1
04, which has similar functions corresponding to the variable length coding unit (VLC) 105, the local decoding unit 106, the second arithmetic unit 107, and the memory 108.

In FIG. 2, reference numeral 14 denotes a block matching unit. The block matching unit 14 uses the camera outputs 1a-k as input screens, and outputs the camera outputs 1a-1 to 1a-1.
By performing block matching using 1a-n as a reference screen, a motion vector is obtained for each block, and a motion compensation vector is determined from the motion vectors. Further, a local motion compensation vector obtained by removing the global motion compensation vector parameter from the motion compensation vector is also obtained.

For example, when the prediction is to be performed on the current screen k (i, j) using the reference screen k '(i + a, j + b), the motion vector V is V (a, b). The block matching section 14 has the same function as the conventional block matching section shown in FIG. By the way, this block matching unit 1
The motion compensation vector and the local motion compensation vector output in 4 are output to the global vector detection unit 15.

Reference numeral 15 denotes a global vector detection unit.
The global vector detection unit 15 calculates a global motion compensation vector parameter based on the motion vector detected by the block matching unit 14,
This is to detect a global vector of one entire screen. This global motion compensation vector parameter is composed of three parameters of pan and zoom in the horizontal and vertical directions. For example, assuming that the horizontal direction Ph and the vertical direction Pv of the zoom parameter and the pan parameter are:
The motion vector V is a global vector only, and V (i
z + Ph, jz + Pv). Therefore,
When zooming in, z takes a negative value, and when zooming out, z takes a positive value.

Uncovered background predictor 16
Is to detect an uncovered background area based on three global motion compensation vector parameters detected by the global vector detection unit 15. Here, the uncovered background area refers to an area that is hidden on the reference screen, that is, an area that is not the area of the screen but appears on the input screen.

When the screen is reduced from the enlarged state, the uncovered background prediction unit 16 uses the zoom parameter of the global motion compensation vector parameter detected by the global vector detection unit 15 to determine the peripheral portion of the screen. As an uncovered background area. When the screen is moved, the uncovered background prediction unit 16 uses the pan parameter of the global motion compensation vector parameter detected by the global vector detection unit 15 to uncover the peripheral portion of the moving screen. It is designed to be detected as a ground area.

The uncovered background prediction unit 16 uses the local motion compensation vector obtained by the block matching unit 14 together with the global motion compensation vector parameter in order to cope with local motion. Then, when the local motion compensation vector of the block currently being processed is different from the local motion compensation vector of the surrounding block, for example,
When the number of the differences exceeding the first threshold exceeds the second threshold, the current block to be processed is detected as an uncovered background area.

The reference screen determination unit 17 inputs a signal directly from the block matching unit 14 when using the past output from another camera for the uncovered background area detected by the uncovered background prediction unit 16. The position of the block to be processed is referred to to determine which camera uses the past output.

With the above configuration, when performing motion compensation prediction for a multi-view stereoscopic image of two or more eyes, first, the block matching unit 14 and the global motion compensation vector detection are performed based on the current output and the past output from the corresponding camera. The unit 15 calculates a motion compensation vector and a global motion compensation vector parameter. Further, the local motion compensation vector obtained by removing the global motion compensation vector parameter from the motion compensation vector by the block matching unit 14 is obtained, and the uncovered background prediction unit 16 uses the global motion compensation vector parameter to perform Detect covered background areas.

Then, the reference screen determination unit 17 and the motion compensation prediction unit 30 perform motion compensation prediction for the uncovered background area using the past output from another camera. At this time, when detecting the uncovered background area using the global motion compensation vector parameter, the uncovered background prediction unit 16 uses the global motion compensation vector parameter Using the zoom parameter, the periphery of the screen is detected as an uncovered background area.

Here, the uncovered background prediction method by the uncovered background prediction unit 16 paying attention to the zoom parameter is performed according to the flowchart shown in FIG. That is, first,
In step A1, the zoom parameter is z as a global motion compensation vector parameter, and the maximum value A in the x-axis direction when the center point of the image is coordinates (0, 0), and the maximum value in the y-axis direction is The maximum value is B.

Then, in step A2, the sign of the zoom parameter z is checked. Here, if the zoom parameter z is negative, zoom-in is performed, so in step A3,
It is determined that there is no uncovered background. If the zoom parameter z is positive, step A
In step 4, from the outer frame of the image, Az in the x-axis direction and Bz in the y-axis direction
The area inside by z is the uncovered background.

Therefore, the uncovered background area is determined in step A5 as a whole by the following equations (1-1) to (1).
-4) The area shown in FIG. Au ≦ x ≦ A, −B ≦ y ≦ B (1-
1) -A≤x≤-A + u, -B≤y≤B (1-2) -A≤x≤A, -B≤y≤B + V (1-3) -A≤x≤A, BV≤ y ≦ B (1-4) The uncovered background prediction unit 16 uses a global motion compensation vector parameter to detect the uncovered background area, and if the screen is moved, the global motion compensation vector Using the pan parameter, the peripheral part of the moving screen is detected as an uncovered background area.

Now, for example, the motion vector V can be expressed by a global vector only, where the zoom parameter is z, the horizontal component of the pan parameter is Ph, and the vertical component is Pv.
(Iz + Ph, jz + Pv). Therefore, when Ph is a positive value, the camera moves to the right, when the value is negative, the camera moves to the left, when Pv is a positive value, the camera moves up, and when Pv is a negative value, the camera moves down. To indicate that

Next, the prediction of the uncovered background area by the uncovered background prediction unit 16 paying attention to the pan parameter is performed according to the process shown in the flowchart of FIG. That is, in FIG. 4, in step B1, the horizontal component of the pan parameter is Ph, the vertical component is Pv, and the maximum value in the x-axis direction when the center point of the image is coordinates (0, 0) is A. , The maximum value in the y-axis direction is set as B.

Then, in step B2, the sign of the vertical component Pv of the pan parameter is checked. Here, when the vertical component Pv of the pan parameter takes a positive value, it means that the camera is moving upward, and in step B3, the uncovered background appears with the width of Pv above the image. That is, the uncovered background area is as shown in Expression (2-1).

−A ≦ x ≦ A, B−Pv ≦ y ≦ B (2-1) Also, when the vertical component Pv of the pan parameter takes a negative value, it means that the camera is moving downward. In step B4, the uncovered background appears with a width of Pv below the image. That is, the uncovered background area is as shown in Expression (2-2).

-A≤x≤A, -B≤y≤B + Pv (2-2) Next, in step B5, the horizontal component Ph of the pan parameter
Check the sign of. Here, the horizontal component P of the pan parameter
If h is a positive value, it means that the camera is moving to the right, and in step B6, the uncovered background appears on the right side of the image with a width of Ph. That is, the uncovered background area is as shown in Expression (2-3).

A−Ph ≦ x ≦ A, −B ≦ y ≦ B (2-3) When the horizontal component Ph of the pan parameter takes a negative value, it means that the camera is moving to the left. B
7, the uncovered background is P on the left side of the image
Appears with a width of h. That is, the uncovered background area is as shown in Expression (2-4).

-A≤x≤-A + Ph, -B≤y≤B (2-4) Further, the uncovered background prediction unit 16
Uses the local motion compensation vector obtained by the block matching unit 14, and if the local motion compensation vector of the block currently being processed is different from the local motion compensation vector of the surrounding block, For example,
When the number of the differences exceeding the first threshold exceeds the second threshold, the current block to be processed is detected as an uncovered background area.

In this case, the uncovered background prediction section 16 predicts the uncovered background area by using the local motion compensation vector together with the global vector. Is a case where a stationary person is waving only his hand, for example. The prediction of the uncovered background area is performed according to the process shown in the flowchart of FIG.

That is, first, in step C1, as an initial setting, a local motion compensation vector V (vx, vy) to be processed and its surrounding vectors are represented by V1 (vx
1, vy1) to V8 (vx8, vy8),
The thresholds are set as TH1 (first threshold) and TH2 (second threshold). Then, step C2
Then, for Vn (n = 1 to 8), the absolute value differences between V and Vn in the x and y directions are calculated, and
xn, Uyn.

Next, in steps C3 and C4,
Initial settings of the count number and the value of n shown below are respectively performed. Thereafter, in steps C5 to C8, n = 1
The number that satisfies Uxn <TH1 and Uyn <TH1 in ８8 is counted. Then, in step C9, a magnitude relationship between the count number and the threshold value TH2 is checked.

If the counted number is smaller than the threshold value TH2, it is determined in step C10 that the block is an uncovered background area. If not, the block is uncovered in step C11. Judge as not the background area. However, even when a local motion is indicated, if a certain motion compensation vector is extremely different from the leading vector, it is conceivable that there is a high possibility of a mismatch due to the uncovered background. For example, if the threshold value TH1 is 2
3, assuming that TH2 is 1, if the vector direction is shifted within 2 to 3 pixels, it is regarded as a similar vector. If there is no vector, the possibility of mismatch due to the uncovered background is large. Is determined.

Then, in the above, the union of the uncovered background area and the predicted area is obtained, and all blocks including the local uncovered background are predicted as the uncovered background area. That is, in this case, the prediction of the uncovered background area is performed according to the flowchart shown in FIG.

First, in step D1 in FIG. 6, as an initial setting, arrays B _{1 to} B ₃ and B _{0 in which} the values of all elements are all 0 are set.
Set. These arrays are 1-bit arrays,
Each element corresponds to the number of pixels on the screen. Here, in step D2, the sequence in which the pixels which are determined to the uncovered background area to "1" and B ₁ using a zoom parameter. Then, at step D3, the sequence in which the pixels which are determined to the uncovered background area to "1" and B ₂ with the pan parameter. further,
In step D4, the sequence in which the pixels which are determined to the uncovered background area focuses on local motion to "1" and B _3.

Thereafter, in step D5, the arrays B _{1 to} B ₃
Is obtained by performing an OR operation on each pixel, and the array B ₀ is obtained. In step D6, the array B ₀ is partitioned into blocks. If the pixel “1” in this block is equal to or larger than the threshold value TH, it is predicted as an uncovered block. FIGS. 7 and 8 show specific examples of the above operation. First, in FIG. 7, (an array of 8 × 8 for simplicity) 1 the sequence of bits and B ₀ having the number of pixels the screen. The initial state of each pixel of B ₀ is all 0 (see (1) in FIG. 7).

The array in which the pixel determined to be uncovered by paying attention to the zoom parameter is set to “1” is represented by B
₁ [Refer to (2) in FIG. 7], an array in which the pixel determined to be uncovered by paying attention to the pan parameter is set to “1” is represented by B ₂
[See (3) in FIG. 7], local interest and a pixel determined to uncovered by the motion and B ₃ of the sequences as the "1" [(1) in FIG. 8 refer to Fig.

[0072] In addition, the sequence B _1, B _2, SEQ B ₀ ORed pixels relative to B ₃ [see (2) in FIG. 8], then separate the sequence B ₀ for each block, the block When the number of uncovered pixels in the set exceeds the threshold value TH, the block is determined to be an uncovered block [(3) in FIG.
reference〕. Note that (1) to (3) in FIG.
Correspond to steps D1 to D3 in FIG.
This is a process corresponding to (1) to (1) in FIG.
(3) is a process corresponding to step D4 to step D6 in FIG.

When the uncovered background area is predicted as described above, the reference screen determination unit 1
In FIG. 7, an area where the reference screen did not exist in the past screen appears so as to surround the periphery of the image. Thus, the switch 12, the motion compensation prediction unit 30, the block matching unit 14, the global vector detection unit 15, the uncovered background prediction unit 16, and the reference screen determination unit 1
By providing the value of 7, the uncovered background area can be reliably predicted.

Thereafter, in the reference screen determination section 17,
When using the past output from another camera for the uncovered background area detected by the uncovered background prediction unit 16, refer to the position of the block to be processed and use the past output from any camera. A determination is made. in this way,
The reference screen determination unit 17 in this embodiment determines the reference screen only at the position of the uncovered block without using the detected global motion compensation vector or the like. Therefore, the reference screen determination unit 17 As shown in FIG. 9, the configuration includes comparison units 51-1 to 51-6, a ROM 52, a third calculation unit 54, a multiplication unit 55, and a sign inversion unit 56.

Here, the comparison units 51-1 to 51-6 determine whether or not the block is uncovered based on the uncovered determination result, and determine the upper left coordinate position of the block currently being processed. The result is input to the ROM 52 as 1-bit information. The ROM 52 stores data from the comparing units 51-1 to 51-6.
The upper left coordinate discrimination information and the information indicating the camera position of the block to be processed are input, and the switching information is output to the switch 12 in response to the input.

The third arithmetic unit 54 calculates the sum of the absolute values of “i” and “j” at the upper left coordinates (i, j) of the block, and outputs the sum to the comparator 51-2. Is compared with “0” to determine whether the coordinates are located at the origin. The multiplication unit 55 calculates a multiplication value of the x coordinate “i” of the coordinate (i, j) and “B / A”, and determines the magnitude of the multiplication value and the y coordinate “j”. -3 for comparison.

The sign inverting section 56 inverts the sign of the output of the multiplying section 55 so that the comparator 51-4 compares the magnitude of this value with the y coordinate "j". Therefore, the reference screen determination by the reference screen determination unit 17 is performed according to the flowcharts shown in FIGS. First, in step E1 in FIG. 10, a coordinate system in which the coordinates of the center of the screen is the origin (0, 0) is considered, and the upper left coordinate of the block to be processed is (i, j). The maximum value of the screen in the x-axis direction is A, and the maximum value of the screen in the y-axis direction is B.

Next, at step E2, the comparator 51-1 determines whether or not the block is an uncovered block.
It is determined whether the upper left coordinates (i, j) of the block using the process of -2 are located at the origin. Here, if it is not an uncovered block, or if (i, j) is (0, 0), in step E3,
The reference screen performs block matching centering on the same position in the past screen of the same camera.

If the above case does not apply, it is checked in step E4 whether or not the camera is at the right end. If it is the right end, in step E3, the reference screen is the same position as the previous screen of the same camera. Block matching is performed around. If the camera is not at the right end in step E4, in step E5, i is positive and j is B ×
It is determined whether or not j is equal to or less than i / A and j is equal to or more than -B × i / A. That is, it is determined whether or not the coordinates (i, j) are in the area (a) in FIG.

If i and j satisfy these conditions, in step E6, the reference screen is centered on the same position in the past screen of the camera located on the right side (or right end) of the camera to be processed. Perform block matching. If i and j do not meet these conditions,
In step E7, it is checked whether the camera is at the left end.
Here, when the camera is at the left end, in step E3,
The reference screen performs block matching centering on the same position of the past screen of the same camera.

If the camera is not at the left end in step E7, in step E8 in FIG. 11, i is negative and j
Is equal to or greater than B × i / A and j is equal to or less than −B × i / A. That is, it is determined whether or not the coordinates (i, j) are in the area (b) in FIG. If i and j correspond to these conditions, in step E9, the reference screen is centered on the same position of the past screen of the camera located on the right side (or the left end) of the camera to be processed, and the block matching is performed. Perform

If i and j do not satisfy these conditions in step E8, it is checked in step E10 whether or not the camera is at the upper end. Here, if the camera is at the upper end, in step E3, the reference screen performs block matching centering on the same position in the past screen of the same camera. If the camera is not at the upper end, step E1
1, j is positive, j is B × i / A or more, and j is −
It is determined whether it becomes B × i / A or more. That is, it is determined whether or not the coordinates (i, j) are in the area (c) in FIG.

If i and j satisfy these conditions, in step E12, the reference screen is centered on the same position in the past screen of the camera located directly above (or directly above) the camera to be processed. Perform block matching. If i and j do not satisfy these conditions, it is checked in step E13 whether or not the camera is at the lower end. Here, if the camera is on the lower side, step E
In step 3, the reference screen performs block matching centering on the same position in the past screen of the same camera.

If the camera is not on the lower side, the coordinates (i, j) at the upper left of the block are at the position shown in FIG. 12D, and the reference screen is processed by the camera to be processed in step E14. Block matching is performed centering on the same position in the past screen of the camera located immediately below (immediately lower end). The above-described block matching is performed by the motion compensation prediction unit 30, and the block matching in the motion compensation prediction unit 30 is performed in the same manner as the conventional one.

As described above, by using only the predicted uncovered background area (position information), it is possible to determine a reference screen for performing motion compensation on a past screen of another camera. The reference screen is closer to the input screen. Therefore, the prediction error can be reduced and the coding efficiency can be improved.
In addition, since all use past images, there is an advantage that delays between the images are less likely to occur than using the current images.

(B) Description of the Second Embodiment FIG. 13 is a block diagram showing a second embodiment of the present invention. The device shown in FIG. 13 includes a switch 12, a block matching unit 14, a global vector detector. It comprises a unit 15, an uncovered background prediction unit 16, a reference screen determination unit 17 ', and a motion compensation prediction unit 30.

Here, the switch 12, the block matching unit 14, the global vector detection unit 15, the uncovered background prediction unit 16, and the motion compensation prediction unit 30
Is similar to that of the first embodiment described above, and a detailed description thereof will be omitted. In the second embodiment, the difference from the first embodiment is a reference screen determination unit 17 ′, which is detected by the uncovered background prediction unit 16. When using the past output from another camera for the uncovered background area, refer to the global motion compensation vector etc. of the block to be processed and the position of the uncovered block to determine the past output from any camera. Decide whether to use.

For this reason, the motion compensation vector and the local motion compensation vector output from the block matching unit 14 are output to the global vector detection unit 15,
The global motion compensation vector parameters are also output to the reference screen determination unit 17 ′.
7 'is also input.

In other words, the reference screen determining method in the reference screen determining unit 17 'is such that when the output of the camera in the direction to which the global motion compensation vector is directed is used, the area where the camera wraps around and becomes uncovered is used. It can be said that it is based on the fact that it is often reflected. by the way,
As shown in FIG. 14, the reference screen determination unit 17 'performs the functions as described above.
5, ROM 62, global motion compensation vector calculator 6
4-1, the first operation unit 64-2 and the sign inversion unit 65-1,
65-2.

Here, the comparing sections 61-1 to 61-5 determine whether or not the data is uncovered based on the uncovered determination result and determine the direction of the global motion compensation vector and the like. This is input to the ROM 62 as 1-bit information. ROM 62 is
The determination information and the camera position information relating to the global motion compensation vector from the comparison units 61-1 to 61-5 are input, and the switching information is output to the switch 12 according to these inputs.

Global motion compensation vector calculator 64-
1 uses the upper left coordinates (i, j) and the global motion compensation vector parameters (Ph, Pv, z) of the block to be processed to obtain the global motion compensation vector (Ux, U
y). The first multiplying unit 64-2 calculates a constant “1/1” for the x component Ux of the global motion compensation vector.
4 ". Also, the second multiplying unit 64-3
Is obtained by multiplying the output of the first multiplier 64-2 by a constant “3” and outputting the result.

Sign inverting section 65-1 and sign inverting section 65-
Numeral 2 indicates that the sign of the output of the second multiplier 64-3 and the sign of the output of the first multiplier 64-2 are inverted and output. With the above configuration, in the motion compensation prediction method for a multi-view stereoscopic video according to the second embodiment of the present invention, when performing motion compensation prediction for a multi-view stereoscopic video of two or more eyes, first, the current And the past output, the block matching unit 14 and the global motion compensation vector detection unit 15 calculate a motion compensation vector and a global motion compensation vector parameter.

Further, a local motion compensation vector obtained by removing the global motion compensation vector parameter from the motion compensation vector is obtained by the block matching unit 14, and the uncovered background prediction unit 16 uses the global motion compensation vector parameter. hand,
Detect uncovered background areas. Then, the reference screen determination unit 17 'and the motion compensation prediction unit 13 perform motion compensation prediction on the uncovered background area using the past output from another camera.

At this time, when detecting the uncovered background area using the global motion compensation vector parameter, the uncovered background prediction unit 16 uses global motion compensation to reduce the screen size from the enlarged state. The peripheral part of the screen is detected as an uncovered background area using the zoom parameter of the vector parameter. The detailed flowchart at this time is the same as that shown in FIG.

When detecting the uncovered background area using the global motion compensation vector parameter, the uncovered background prediction unit 16 sets the pan parameter of the global motion compensation vector parameter to move the screen. make use of,
The peripheral part of the moving screen is detected as an uncovered background area. The detailed flowchart at this time is the same as that shown in FIG.

The uncovered background prediction unit 16 uses the local motion compensation vector obtained by the block matching unit 14 and calculates the local motion compensation vector of the block currently being processed by the local motion compensation vector of the surrounding block. If the difference exceeds the first threshold value, for example,
When the threshold value is exceeded, the current block to be processed is detected as an uncovered background area. The detailed flowchart at this time is shown in FIG.
Are the same as those shown in FIG.

Thereafter, the uncovered background prediction section 16 calculates the uncovered background area using the zoom parameter, the area using the pan parameter, and the area detected using the local motion compensation vector. The logical sum is calculated and detected as an uncovered background area. The detailed flowchart at this time is the same as that shown in FIG.

Further, the reference screen determination unit 17 'uses the past output from another camera for the uncovered background area detected by the uncovered background prediction unit 16 when processing the block to be processed. With reference to the global motion compensation vector and the position of the vector currently being processed, it is determined which camera to use the past output from.

In the reference screen determination method performed by the reference screen determination unit 17 ', the reference screen determination method shown in FIGS.
It operates according to the flowchart shown in FIG. First, in step F1 in FIG. 15, the coordinates of the center of the screen are set to the origin (0, 0), the maximum value of the screen in the x-axis direction is A, and the maximum value of the screen in the y-axis direction is B as an initial setting. Set the system.

[0100] The upper left coordinate of the block to be processed is (i, j). Then, the original motion compensation vector is (Vecx, Vecy), the global motion compensation parameter is (Ph, Pv, Z), and the local motion compensation vector (Vx , Vy). Here, FIG.
When the reference screen is determined by the reference screen determination unit 17 ′ in 3, the prediction of the uncovered background by the uncovered background prediction unit 16 has already been completed.

Therefore, in step F2, the comparison unit 61-1 in FIG. 14 determines whether the block to be processed is an uncovered background. If the background is not the uncovered background, in step F3, the reference screen performs block matching centering on the same position in the past screen of the same camera.

If the block has an uncovered background, the global motion compensation vector (Ux, Uy) in the block is calculated in step F4. The calculation is performed using the global motion compensation parameters (Ph, Pv, Z) and the coordinates (i, j) of the block. As a method of calculating the global motion compensation vector (Ux, Uy), there is also a method of subtracting the local motion compensation vector from the original motion compensation vector to calculate (step F4 ').

Next, in step F5, it is checked whether or not the camera is not at the right end. If it is the right end, in step F3, the reference screen performs block matching centering on the same position in the past screen of the same camera. If it is not the right end, it is determined in step F6 whether Uy is equal to or more than -Ux / 4 and equal to or less than Ux / 4. That is, it is determined whether or not the y component Uy of the motion compensation vector is in the area (a) in FIG.

If Uy is in the area, in step F7, the reference screen is centered on the same position on the past screen of the camera located on the right side (or right end) of the camera to be processed. Perform block matching. If none of the above cases applies, it is checked in step F8 whether the camera is not at the upper right end. Here, if it is the upper right end, in step F3, the reference screen performs block matching centering on the same position of the past screen of the same camera.

If the camera is not located at the upper right end, it is determined in step F9 whether Uy is not less than Ux / 4 and not more than Ux × 3/4. That is, y of the motion compensation vector
It is determined whether or not the component Uy is in the area (b) in FIG. Here, when Uy is in the area,
In step F10, the reference screen performs block matching centering on the same position of the past screen of the camera located on the upper right side (or upper right end) of the camera to be processed.

If the above case does not apply, it is checked in step F11 in FIG. 16 whether the camera is not at the upper end. Here, if it is the upper end, in step F3,
The reference screen performs block matching centering on the same position of the past screen of the same camera. If the camera is not at the upper end, in step F12, Uy is Ux × 3 /
It is determined whether the value falls within the range of 4 or more and −Ux × 3/4 or more. That is, the y component Uy of the motion compensation vector
Is located in the area (c) in FIG.

Here, if Uy is in the area, in step F13, the reference screen is centered on the same position of the past screen of the camera located above (or at the upper end of) the camera to be processed, and is subjected to block matching. Perform
If Uy does not fall within the above range, it is checked in step F14 whether the camera is not at the upper left corner. Here, if the camera is at the upper left corner, in step F3, the reference screen performs block matching centering on the same position of the past screen of the same camera.

However, if the camera is not at the upper left corner, Uy is equal to or more than -Ux / 4 and Ux × 3 /
It is determined whether it is within the range of four or more. That is, it is determined whether or not the y component Uy of the motion compensation vector is in the area (d) in FIG. Here, if Uy is in the area, in step F16, the reference screen is centered on the same position on the past screen of the camera located on the upper left (or upper left) of the camera to be processed, and block matching is performed. Do.

If Uy does not fall within the above range, it is checked in step F17 whether the camera is not at the left end.
Here, if the camera is at the left end, in step F3,
The reference screen performs block matching centering on the same position in the past screen of the same camera. However, when the camera is not at the left end, in step F18 in FIG.
It is determined whether or not y is in the range from Ux / 4 to -Ux / 4. That is, the y component Uy of the motion compensation vector
Is in the area (e) in FIG.

Here, when Uy is in the above range,
In step F19, the reference screen performs block matching centering on the same position in the past screen of the camera located on the left side (or left end) of the camera to be processed. If Uy does not fall within the above range, step F
At 20, it is determined whether the camera is at the lower left corner. Here, if the camera is at the lower left corner, in step F3,
The reference screen performs block matching centering on the same position in the past screen of the same camera.

However, if the camera is not at the lower left corner, it is determined in step F21 whether or not Uy is in the range from Ux × 3/4 to Ux / 4. That is, it is determined whether or not the y component Uy of the motion compensation vector is in the area (f) in FIG. Here, if Uy is within the above range, in step F22, the reference screen is centered on the same position of the past screen of the camera located on the lower left side (or lower left end) of the camera to be processed, and block matching is performed. Perform

If Uy does not fall within the above range, it is determined in step F23 whether or not the camera is at the lower end. Here, if the camera is at the lower end, step F3
Then, the reference screen performs block matching centering on the same position of the past screen of the same camera. However, if the camera is not at the lower end, in step F24, Uy is −U
It is determined whether it is within the range of xx 3/4 or less and Ux 3/4 or less. That is, the y component U of the motion compensation vector
It is determined whether or not y is in the area (g) in FIG.

When Uy is within the above range,
In step F25, the reference screen performs block matching centering on the same position of the past screen of the camera positioned below (or at the lower end of) the camera to be processed. If Uy is not within the above range, step F26
To determine whether the camera is at the lower right corner. If the camera is located at the lower right corner, block matching is performed on the reference screen at the same position in the past screen of the same camera in step F3.

However, if the camera is not at the lower right corner, in step F27, the reference screen is centered on the same position of the past screen of the camera located at the lower right side (or lower end) of the camera to be processed, and Perform matching. As described above, by including the switch 12, the motion compensation prediction unit 30, the block matching unit 14, the global vector detection unit 15, the uncovered background prediction unit 16, and the reference screen determination unit 17 ', the uncovered background area can be reduced. In addition to being able to make reliable predictions, it is also possible to determine a reference screen for performing motion compensation on a past screen of another camera by using a predicted uncovered background area and a detected global motion compensation vector. The reference screen in this motion compensation is closer to the input screen. Therefore, the prediction error can be reduced and the coding efficiency can be improved. In addition, since all use past images,
There is an advantage that delay between images is less likely to occur than using the current image.

(C) Others In the above embodiments, the uncovered background area is comprehensively detected by focusing on the zoom parameter, the pan parameter, and the local movement. The anchor guard background area can be detected by focusing on one or a combination of one of the parameter and the local motion.

[0116]

As described above in detail, according to the motion compensation prediction method for a multi-view stereoscopic video of the present invention, when performing motion compensation prediction for a multi-view stereoscopic video of two or more eyes, first, the corresponding camera is used. From the current output and the past output from, a motion compensation vector and a global motion compensation vector parameter are calculated, and a local motion compensation vector obtained by removing the global motion compensation vector parameter from the motion compensation vector is obtained. Reduces the prediction error by detecting the uncovered background area using the global motion compensation vector parameter, and then performing motion compensation prediction using the past output from another camera for the uncovered background area. In addition, encoding efficiency can be improved, and furthermore, there is an advantage that delay between images is hardly generated.

When detecting the uncovered background area by using the global motion compensation vector parameter and reducing the size of the screen from the enlarged state, the zoom parameter of the global motion compensation vector parameter is used to reduce the screen size. By detecting the surrounding area as an uncovered background area, accurate motion compensation can be performed even when the screen is reduced from the enlarged state, and therefore, the encoding efficiency is improved, and the delay between each image is reduced. There are advantages such as reduction.

When detecting the uncovered background area by using the global motion compensation vector parameter, when moving the screen, the pan parameter of the global motion compensation vector parameter is used to move the screen. By detecting the part as an uncovered background area, even when moving the screen, accurate motion compensation can be performed,
Therefore, there are advantages such as an improvement in coding efficiency and a reduction in delay between images.

Further, when detecting an uncovered background area using the global motion compensation vector parameter, a local motion compensation vector is used, and the local motion compensation vector of the block currently being processed is changed to the surrounding block. If the current block is different from the local motion compensation vector, the current block to be processed is detected as an uncovered background area, thereby contributing to an improvement in coding efficiency.

When detecting the uncovered background area using the global motion compensation vector parameter, the local motion compensation vector is used to detect the local motion compensation vector of the block currently being processed. When the number of the difference exceeding the first threshold exceeds the second threshold as compared with the local motion compensation vector of, the current block to be processed is regarded as an uncovered background area. By performing the detection, the prediction error can be reduced and the coding efficiency can be improved. In addition, there is an advantage that delay between images is less likely to occur.

By the way, when using the past output from another camera for the uncovered background area,
By referring to the position of the block to be processed and determining which camera uses the past output, the reference screen in motion compensation becomes closer to the input screen, and as a result, the prediction error is reduced. The coding efficiency can be reduced and the coding efficiency can be improved.

When the past output from another camera is used for the uncovered background area, reference is made to the global motion compensation vector of the block to be processed to determine which camera uses the past output. The determination has the advantage that the prediction error can be reduced and the coding efficiency can be improved.

[Brief description of the drawings]

FIG. 1 is a principle block diagram of the present invention.

FIG. 2 is a block diagram showing a first embodiment of the present invention.

FIG. 3 is a flowchart illustrating a procedure for detecting an uncovered background area using a zoom parameter.

FIG. 4 is a flowchart illustrating a procedure for detecting an uncovered background area using a pan parameter.

FIG. 5 is a flowchart illustrating a procedure for detecting an uncovered background area by focusing on local motion.

FIG. 6 is a flowchart illustrating a procedure for detecting an uncovered background area.

FIG. 7 is a diagram for specifically explaining a detection procedure of an uncovered background area.

FIG. 8 is a diagram for specifically explaining a detection procedure of an uncovered background area.

FIG. 9 is a block diagram illustrating a reference screen determination unit according to the first example of the present invention.

FIG. 10 is a flowchart for explaining a reference screen determination procedure according to the first embodiment of the present invention.

FIG. 11 is a flowchart illustrating a reference screen determination procedure according to the first embodiment of the present invention.

FIG. 12 is a diagram illustrating a reference screen according to the first embodiment of the present invention.

FIG. 13 is a block diagram showing a second embodiment of the present invention.

FIG. 14 is a block diagram illustrating a reference screen determination unit according to a second example of the present invention.

FIG. 15 is a flowchart illustrating a reference screen determination procedure according to a second embodiment of the present invention.

FIG. 16 is a flowchart illustrating a reference screen determination procedure according to the second embodiment of the present invention.

FIG. 17 is a flowchart illustrating a reference screen determination procedure according to the second embodiment of the present invention.

FIG. 18 is a diagram illustrating a reference screen according to a second embodiment of the present invention.

FIG. 19 is a diagram showing a multi-view stereoscopic video system.

FIG. 20 is a diagram illustrating an output example of a multi-view (5 × 5) camera.

FIG. 21 is a diagram showing a holographic stereogram.

FIG. 22 is a block diagram showing a conventional motion compensation method.

FIG. 23 is a diagram showing block matching according to a conventional motion compensation method.

FIG. 24 is a block diagram illustrating a conventional parallax compensation method.

FIG. 25 is a flowchart for explaining conventional block matching and detection of a disparity compensation vector.

[Explanation of symbols]

 1a-1 to 1a-n Camera output 1-1 to 1-n Camera output 2,12 Switch 3,30 Motion compensation prediction unit 4,14,31,212 Block matching unit 5,15 Global vector detection unit 6,16 Covered background prediction unit 7, 17, 17 'Reference screen determination unit 32, 202, 214 Variable delay unit 33, 203 First operation unit 34, 204, 216, 218a, 218b Encoding unit 35, 205, 217, 220 Variable Long encoding unit 36, 206 Local decoding unit 37, 207 Second operation unit 38, 208 Memory 51-1 to 51-6, 61-1 to 61-5 Comparison unit 52, 62 ROM 53, 63 Switch 54 Third operation Unit 55 multiplication unit 56, 65-1, 65-2 sign inversion unit 64-1 global motion compensation vector calculation unit 64-2 first Calculation unit 64-3 Second multiplication unit 101 Subject 102 Camera 103 Encoder 104 Multiplexer 105 Demultiplexer 106 Decoder 107 Display 108 Image size conversion unit 109 Phase calculation unit 110 Holographic output system 201 Block matching and vector Detector 209 Square error calculator 209-1, 215 Subtractor 209-2 Square calculator 209-3 Adder 209-4, 211-2, 211-5 Delay element 210 Vector converter 211 Vector detector 211- 1 comparison unit 211-3, 211-4 selector 213 disparity compensation vector detection unit

Continuation of the front page (72) Inventor Kiichi Matsuda 1015 Kamikodanaka, Nakahara-ku, Kawasaki-shi, Kanagawa Prefecture Within Fujitsu Limited (58) Field surveyed (Int. Cl. ⁷ , DB name) H04N 13/00-15/00

Claims (7)

(57) [Claims] When performing motion compensation prediction for a multi-view stereoscopic image of two or more eyes, first, a motion compensation vector and a global motion compensation vector parameter are calculated from a current output and a past output from a corresponding camera. Calculating a local motion compensation vector by removing the global motion compensation vector parameter from the motion compensation vector, and detecting an uncovered background area using the global motion compensation vector parameter. A motion compensation prediction method for a multi-view stereoscopic video, wherein motion compensation prediction is performed using a past output from another camera for a covered background area. 2. When detecting an uncovered background area using the global motion compensation vector parameter, when the screen is reduced from an enlarged state, the zoom parameter of the global motion compensation vector parameter is used. 2. The method according to claim 1, wherein a peripheral portion of a screen is detected as the uncovered background area. 3. When the screen is moved to detect an uncovered background area using the global motion compensation vector parameter, the moving picture is moved using the pan parameter of the global motion compensation vector parameter. 2. The method according to claim 1, wherein a peripheral portion of the image is detected as the uncovered background area. 4. When detecting an uncovered background area using the global motion compensation vector parameter, a local motion compensation vector is used, and a local motion compensation vector of a block to be currently processed is set to a surrounding motion compensation vector. 2. The multi-view stereoscopic video according to claim 1, wherein, when the block is different from the local motion compensation vector of the block, the block currently being processed is detected as the uncovered background area. Motion compensation prediction method. 5. When detecting an uncovered background area using the global motion compensation vector parameter, a local motion compensation vector is used, and a local motion compensation vector of a block to be currently processed is set to a surrounding motion compensation vector. Compared with the local motion compensation vector of the block, if the number of the differences exceeding the first threshold exceeds the second threshold, the current block to be processed is replaced with the uncovered background. 5. The method according to claim 4, wherein the detection is performed as an area. 6. When a past output from another camera is used for the uncovered background area, it is determined which camera to use the past output with reference to the position of a block to be processed. 2. The method of claim 1, wherein the multi-view stereoscopic video is motion compensated and predicted. 7. When the past output from another camera is used for the uncovered background area, the past output from which camera is used by referring to the global motion compensation vector of the block to be processed. The method of claim 1, further comprising: determining a motion compensation prediction of the multi-view stereoscopic video.