CN103220532B - The associated prediction coded method of three-dimensional video-frequency and system - Google Patents

Abstract

The present invention proposes a joint predictive coding method and system for stereoscopic video. Among them, the method includes: S1: Input stereoscopic video and divide the stereoscopic video into multiple coded macroblocks; S2: Predict the depth prediction disparity of the current coded macroblock through the method of depth prediction, and perform depth-assisted viewing on the current coded macroblock Inter-predictive coding; S3: Perform traditional inter-view predictive coding on the current macroblock; S4: Perform time-domain predictive coding on the current coded macroblock; S5: Calculate the depth-assisted inter-view predictive coding and traditional inter-view prediction of the current coded macroblock respectively Rate-distortion performance in coding and time-domain predictive coding modes; S6: Select the predictive coding mode with the best rate-distortion performance as the prediction mode of the currently coded macroblock and perform coding. According to the method of the embodiment of the present invention, the disparity of coded macroblocks is estimated by depth to perform inter-view compensation prediction, which reduces the bit rate required for disparity coding in stereoscopic video coding and improves the efficiency of stereoscopic video coding.

Description

Joint predictive coding method and system for stereoscopic video

technical field

The present invention relates to the field of video coding, in particular to a joint predictive coding method and system for stereoscopic video.

Background technique

With the continuous development of video technology, stereoscopic video has gained widespread attention for its realistic visual effects. In stereoscopic video, the video data consists of a video sequence and a depth map sequence. Wherein, the video sequence usually includes two or even multiple video sequences. The depth map sequence contains the depth map corresponding to each channel of video. Therefore, in the application of stereoscopic video, how to effectively compress and transmit massive video and depth maps has become one of the key technical bottlenecks in the application of stereoscopic video.

In order to achieve efficient compression of stereoscopic video data, researchers proposed a multi-view video coding scheme. In this scheme, one video in the multi-view video is taken as the basic view, and the redundancy in the time domain is compressed using the traditional video coding scheme. For the video of other viewpoints, the coding scheme introduces an inter-view prediction mode, which compresses the time domain and inter-view redundancy of multi-view video through temporal prediction and inter-view prediction, thus effectively reducing the time required for encoding multi-view video. code rate. Since the depth map can be regarded as a multi-view grayscale video sequence, the multi-view video coding scheme is also used to code the depth map. In the current mainstream stereoscopic video coding scheme, the encoder compresses the multi-viewpoint video and the depth map respectively using a multi-viewpoint video coding scheme to obtain two code streams of video and depth map, and transmit the two code streams to the decoding end at the same time , to reconstruct multi-view video and depth map sequences. The decoding end further draws the virtual viewpoint according to the needs of the user, thereby forming the stereoscopic video sequence required by the user, and playing it on the corresponding stereoscopic video display.

Although multi-view video coding can effectively compress the temporal and inter-view redundancy of multi-view videos and depth maps, the redundancy between multi-view videos and depth maps cannot be effectively compressed. In stereoscopic video, the depth map represents the depth information of corresponding points in the video sequence. Under the premise of given shooting conditions, the disparity information of each coded macroblock can be obtained through depth value prediction. In stereoscopic video, the depth map can be regarded as the side information of multi-view video coding, so the disparity obtained by disparity search can be replaced by disparity calculated by depth, the coding bit rate required for coding disparity can be reduced, and the difference between multi-view video and depth map can be compressed. Redundancy between.

Currently, there are two stereoscopic video coding methods based on joint coding of multi-view video and depth map. One is that the encoder renders a virtual reference frame according to the depth map corresponding to the current video frame to be encoded and its reference video frame, so as to reduce the redundant information in the depth map and disparity coding. The other is a prediction method that obtains the relationship between temporal motion information and disparity information through the geometric constraint relationship between temporal motion information and inter-view disparity information.

Disadvantages of existing technologies include:

(1) An additional codec cache is required, which increases the space complexity of the codec

(2) The computational complexity is high, which increases the time complexity of the codec

Contents of the invention

The object of the present invention is to solve at least one of the above-mentioned technical drawbacks.

Therefore, an object of the present invention is to propose a joint predictive coding method for stereoscopic video.

Another object of the present invention is to propose a joint predictive coding system for stereoscopic video.

In order to achieve the above purpose, an embodiment of the present invention proposes a joint predictive coding method for stereoscopic video, including the following steps: S1: input stereoscopic video and divide the stereoscopic video into multiple coded macroblocks; S2: pass depth The prediction method predicts the depth prediction disparity of the current coded macroblock, and performs depth-assisted inter-view prediction coding on the current coded macroblock according to the depth prediction disparity; S3: Obtain the disparity vector through the inter-view matching method, and according to the Perform traditional inter-view predictive coding on the current macroblock with the disparity vector; S4: Obtain a motion vector through time domain motion estimation, and perform time domain predictive coding on the current coded macroblock according to the motion vector; S5: Respectively Calculate the rate-distortion performance of the currently coded macroblock in the depth-assisted inter-view predictive coding, traditional inter-view predictive coding, and time-domain predictive coding modes; and S6: Select the predictive coding mode with the best rate-distortion performance as the current The prediction mode of the macroblock is encoded and encoded.

According to the method of the embodiment of the present invention, the disparity of coded macroblocks is estimated by depth to perform inter-view compensation prediction, which reduces the bit rate required for disparity coding in stereoscopic video coding and improves the efficiency of stereoscopic video coding.

In an embodiment of the present invention, the method further includes: S7: judging whether the encoding of all encoded macroblocks is completed; S8: if not, repeating the steps S1-S5 for unencoded macroblocks until all encoded macroblocks The blocks are all encoded.

In an embodiment of the present invention, the rate-distortion performance of the time-domain predictive coding is obtained by the following formula, $J_{\mathrm{MCP}}(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)|+{\mathrm{\λ}}_{\mathrm{motion}}(r_m+r_h),$ in, is the motion vector, B _k is the current coded macroblock, and ref _m is The reference frame pointed to, X is each pixel in B _k , I is the brightness or chrominance component value corresponding to X, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, λ _motion is the Lagrangian multiplier of time domain prediction, r _m is the encoding bit rate required for encoding the motion vector, r _h is the encoding except motion The code rate required by other macroblock header information other than the vector.

In an embodiment of the present invention, the rate-distortion performance of the traditional inter-view predictive coding is obtained by the following formula, $J_{\mathrm{DCP}}(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)|+{\mathrm{\λ}}_{\mathrm{motion}}(r_d+r_h),$ in, is the disparity obtained by inter-view matching, B _k is the current coded macroblock, and ref _d is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, X is each pixel in B _k , I is the luminance or chrominance component value corresponding to X, and λ _motion is the Lagrang of traditional inter-view prediction day multiplier, rd is the encoding bit rate required for encoding and _searching for the disparity vector.

In an embodiment of the present invention, the rate-distortion performance of the depth-assisted inter-view predictive coding is obtained by the following formula, $J_{\mathrm{DADCP}}(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)|+{\mathrm{\λ}}_{\mathrm{motion}}\×r_h^{\′},$ in, Disparity is calculated for depth, B _k is the current coded macroblock, and ref _z is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, X is each pixel in B _k , I is the luminance or chrominance component value corresponding to X, and λ _motion is the pull of depth-assisted inter-view prediction Grangian multiplier, r _h ′ is the code rate required for encoding the macroblock header information in the disparity compensation prediction mode based on the depth prediction disparity.

In order to achieve the above object, another embodiment of the present invention proposes a joint predictive coding system for stereoscopic video, including: a division module for inputting stereoscopic video and dividing the stereoscopic video into multiple coded macroblocks; The prediction module is used to predict the depth prediction disparity of the currently coded macroblock through the method of depth prediction, and perform depth-assisted inter-view prediction coding on the current coded macroblock according to the depth prediction disparity; the second prediction module is used to encode the current coded macroblock performing traditional inter-view predictive encoding on the current macroblock; a third prediction module, configured to perform time-domain predictive encoding on the currently encoded macroblock; and a calculation module, configured to calculate the depth-assisted Rate-distortion performance in inter-view predictive coding, traditional inter-view predictive coding and time-domain predictive coding modes; and a selection module, used to select the predictive coding mode with the best rate-distortion performance as the prediction mode of the currently coded macroblock and perform coding.

According to the system of the embodiment of the present invention, the disparity of coded macroblocks is estimated by depth to perform inter-view compensation prediction, which reduces the bit rate required for disparity coding in stereoscopic video coding and improves the efficiency of stereoscopic video coding.

In an embodiment of the present invention, the system further includes: a judging module, used to judge whether the coding of all coded macroblocks is completed; a processing module, used to reuse the division module, the first prediction module, the second prediction module, the third prediction module, the calculation module and the selection module until all coded macroblocks are coded.

In an embodiment of the present invention, the rate-distortion performance of the time-domain predictive coding is obtained by the following formula, $J_{\mathrm{MCP}}(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)|+{\mathrm{\λ}}_{\mathrm{motion}}(r_m+r_h),$ in, is the motion vector, B _k is the current coded macroblock, and ref _m is The reference frame pointed to, X is each pixel in B _k , I is the brightness or chrominance component value corresponding to X, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, λ _motion is the Lagrangian multiplier of time domain prediction, r _m is the encoding bit rate required for encoding the motion vector, r _h is the encoding except motion The code rate required by other macroblock header information other than the vector.

In an embodiment of the present invention, the rate-distortion performance of the traditional inter-view predictive coding is obtained by the following formula, $J_{\mathrm{DCP}}(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)|+{\mathrm{\λ}}_{\mathrm{motion}}\×(r_d+r_h),$ in, is the disparity obtained by inter-view matching, B _k is the current coded macroblock, and ref _d is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, X is each pixel in B _k , I is the luminance or chrominance component value corresponding to X, and λ _motion is the Lagrang of traditional inter-view prediction day multiplier, rd is the encoding bit rate required for encoding and _searching for the disparity vector.

In an embodiment of the present invention, the rate-distortion performance of the depth-assisted inter-view predictive coding is obtained by the following formula, $J_{\mathrm{DADCP}}(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)|+{\mathrm{\λ}}_{\mathrm{motion}}\×r_h^{\′},$ in, Disparity is calculated for depth, B _k is the current coded macroblock, and ref _z is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, X is each pixel in B _k , I is the luminance or chrominance component value corresponding to X, and λ _motion is the pull of depth-assisted inter-view prediction Grangian multiplier, r _h ′ is the code rate required for encoding the macroblock header information in the disparity compensation prediction mode based on the depth prediction disparity.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

FIG. 1 is a flowchart of a joint predictive coding method for stereoscopic video according to an embodiment of the present invention;

Fig. 2 is a schematic diagram of drawing a virtual viewpoint according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a coding prediction structure according to an embodiment of the present invention; and

Fig. 4 is a structural block diagram of a joint predictive coding system for stereoscopic video according to an embodiment of the present invention.

detailed description

Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

FIG. 1 is a flowchart of a joint predictive coding method for stereoscopic video according to an embodiment of the present invention. As shown in FIG. 1, the joint predictive coding method for stereoscopic video according to an embodiment of the present invention includes the following steps:

Step S101, input a stereoscopic video and divide the stereoscopic video into a plurality of coded macroblocks.

Specifically, the stereoscopic video is input, pre-processing such as correction and alignment is performed on it, and then the processed stereoscopic video is divided into multiple coded macroblocks.

Step S102 , predicting the depth prediction disparity of the currently coded macroblock through a depth prediction method, and performing depth-assisted inter-view prediction coding on the current coded macroblock according to the depth prediction disparity.

Specifically, it is assumed that a stereoscopic video sequence only contains videos and depth map sequences of left and right viewpoints. The baseline distance between the left and right viewpoints is c, and the camera focal lengths of the left and right viewpoints are both f. The currently coded macroblock is B _k . B _k contains n _j pixels, and the depth value corresponding to each pixel is The depth prediction disparity of the currently coded macroblock B _k is predicted through the depth value of each pixel. Assuming that the depth value of the current coded macroblock B _k is the maximum likelihood value of the depth values corresponding to all pixels contained in B _k , z _k can be expressed as $z_k=\underset{z_k^j}{\arg \max }\mathrm{prob}\left(\right\{z_k^j|j=\mathrm{1,2},...,{\mathrm{no}}_{B_k}\}),$ in, is the depth value of each pixel.

Fig. 2 is a schematic diagram of virtual viewpoint rendering according to an embodiment of the present invention. As shown in FIG. 2 , after the depth value corresponding to B _k is obtained, the disparity of the currently coded macroblock can be calculated through the mapping relationship between depth and disparity. The predicted disparity of the currently coded macroblock can be expressed as, Among them, d _k is the calculated disparity, f is the focal length, and c is the baseline distance between the left and right viewpoints. For a coding mode with quarter-pixel precision, d _k is rounded to the nearest quarter-pixel point position as the depth prediction disparity of the current coded macroblock.

In step S103, a disparity vector is obtained by means of inter-view matching, and conventional inter-view predictive coding is performed on the current macroblock according to the disparity vector.

In step S104, a motion vector is obtained by means of time-domain motion estimation, and time-domain predictive coding is performed on the currently coded macroblock according to the motion vector.

Step S105, respectively calculate the rate-distortion performance of the current coded macroblock in depth-assisted inter-view predictive coding, traditional inter-view predictive coding and time-domain predictive coding modes

Specifically, the encoder will calculate the rate-distortion performance in different prediction modes. Let the motion vector of the current coded macroblock B _k be Search parallax for The depth prediction disparity is

The rate-distortion performance of the time-domain predictive coding of the current macroblock search disparity is obtained by the following formula, $J_{\mathrm{MCP}}(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)|+{\mathrm{\λ}}_{\mathrm{motion}}(r_m+r_h),$ in, is the motion vector, B _k is the current coded macroblock, and ref _m is The reference frame pointed to, X is each pixel in B _k , I is the brightness or chrominance component value corresponding to X, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, λ _motion is the Lagrangian multiplier of time domain prediction, r _m is the encoding bit rate required for encoding the motion vector, r _h is the encoding except motion The code rate required by other macroblock header information other than the vector.

The rate-distortion performance of the traditional inter-view predictive coding of the current macroblock search disparity is obtained by the following formula, $J_{\mathrm{DCP}}(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)|+{\mathrm{\λ}}_{\mathrm{motion}}\×(r_d+r_h),$ in, is the disparity obtained by inter-view matching, B _k is the current coded macroblock, and ref _d is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, X is each pixel in B _k , I is the luminance or chrominance component value corresponding to X, and λ _motion is the Lagrang of traditional inter-view prediction day multiplier, rd is the encoding bit rate required for encoding and _searching for the disparity vector.

In stereoscopic video, depth information can be regarded as side information for video coding. Therefore, we can assume that the encoder and decoder can simultaneously obtain the same reconstructed depth map. Therefore, the depth prediction disparity does not need to be encoded into the code stream. Therefore, the rate-distortion performance corresponding to the depth-assisted inter-view prediction coding of the current macroblock through the depth prediction parallax can be expressed as, $J_{\mathrm{DADCP}}(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)|+{\mathrm{\λ}}_{\mathrm{motion}}\×r_h^{\′},$ in, Disparity is calculated for depth, B _k is the current coded macroblock, and ref _z is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, X is each pixel in B _k , I is the luminance or chrominance component value corresponding to X, and λ _motion is the pull of depth-assisted inter-view prediction Grangian multiplier, r _h ′ is the code rate required for encoding the macroblock header information in the disparity compensation prediction mode based on the depth prediction disparity.

Step S106, selecting the prediction mode corresponding to the rate-distortion performance with the smallest rate-distortion performance as the prediction mode of the currently coded macroblock and performing coding.

Specifically, the encoder selects the prediction mode with the best rate-distortion as the prediction mode of the currently coded macroblock. Its selection process can be expressed as, $J=\min (J_{\mathrm{MCP}}(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m),J_{\mathrm{DCP}}(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d),J_{\mathrm{DADCP}}(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)),$ in, and Denote the rate-distortion performance of temporal prediction, the rate-distortion performance of traditional inter-view prediction, and the rate-distortion performance of depth-assisted inter-view prediction, respectively.

In one embodiment of the present invention, a standard test video sequence named "Book Arrival" in standard definition format is used for the video sequence of stereoscopic video encoding, and the pixels of the video sequence in standard definition format are 1024×768. The decoder adopts H.264/AVC (Multi-view Video Coding, multi-view video extension version) standard reference software JMVC (Joint Multi-view Video Coding, multi-view video coding), encoder GOP (Group of Pictures, group of pictures) ) frame number is 8, and the time-domain predictive coding of coding adopts the prediction structure of Hierarchical B (hierarchical bidirectional predictive coding frame, referred to as hierarchical B frame). FIG. 3 is a schematic diagram of the coding prediction structure according to an embodiment of the present invention. As shown in Figure 3, the virtual view point is drawn using two channels of color video and a depth map adjacent to the virtual view point. In this implementation example, two videos of viewpoint 10 and viewpoint 8 of the "Book Arrival" sequence are used as the multi-view video input sequence, wherein viewpoint 10 is called the left reference viewpoint, and viewpoint 8 is called the right reference viewpoint. The value range of the multi-view video and multi-view depth map coding quantization parameter QP is an integer between 0 and 51. The baseline distance of the left and right viewpoints is 10, and the focal length of the camera is 100.

Assume that the currently coded macroblock B _k is an 8×8 macroblock in one frame of view 8 video in the “Book Arrival” sequence. Its corresponding depth value is shown in the following 8×8 matrix.

$z_{B_k}=\left[\begin{array}{cccccccc}62& 63& 62& 61& 63& 58& 62& 63\\ 61& 62& 62& 63& 61& 64& 65& 64\\ 62& 61& 57& 61& 63& 59& 63& 63\\ 67& 61& 58& 62& 61& 61& 66& 64\\ 63& 62& 62& 58& 60& 62& 63& 62\\ 61& 62& 61& 61& 61& 63& 60& 58\\ 62& 62& 61& 62& 62& 62& 60& 62\\ 64& 63& 61& 62& 62& 62& 60& 61\end{array}\right].$ For the current coded macroblock B _k , the maximum likelihood value z _k of the corresponding depth values of all pixels contained in B _k is, $z_k=\underset{z_k^j}{\arg \max }\mathrm{prob}\left(\right\{z_k^j|j=\mathrm{1,2},...,{\mathrm{no}}_{B_k}\})=62.$ After obtaining the depth value of the currently coded macroblock B _k , the predicted disparity of the current coded macroblock is, For the encoding mode with quarter-pixel precision, after d _k is rounded to the nearest pixel, the disparity should be, d _k ′=[d _k ]=16.25. The encoder then performs traditional inter-view prediction based on the predicted disparity information. For the currently coded macroblock, its predicted disparity is 16.25. The encoder finds the corresponding reference macroblock in the corresponding frame of the viewpoint 10, and performs prediction. Let the sum of the absolute values of the forecast residuals be 50. In addition, the encoder will also perform compensation prediction, instant domain prediction and traditional inter-view prediction on the current macroblock. In the time-domain prediction, it is advisable to set the motion vector of the current macroblock as 32, and the sum of the absolute values of the residuals of the time-domain prediction as 80. In the traditional inter-view prediction of inter-view, it is assumed that the disparity obtained by the encoder through block matching search is 16, and the sum of the absolute values of the residuals of the traditional inter-view prediction is 45.

The encoder will then compare the rate-distortion performance of different inter-macroblock predictions. Assume that the number of bits required to encode the motion vector of the current macroblock B _k is rm = 10, the number of bits required to encode the disparity obtained by the block matching search of B _k is r _d = 8, and the number of bits required to encode the header information of _{B k} The number of bits is r _h =20. Then, in the traditional inter-view prediction based on the depth prediction disparity, the number of bits required to encode the header information of B _k is r _h ′=21. An additional bit is used to identify that the current macroblock adopts traditional inter-view prediction based on depth prediction disparity. In the rate-distortion optimization process, the value of the Lagrangian multiplier λ _motion is set to 1.5.

Therefore, for a macroblock B _k , the rate-distortion performance of its time-domain prediction is, $J_{\mathrm{MCP}}(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)|+{\mathrm{\λ}}_{\mathrm{motion}}(r_m+r_h)=80+1.5\×(10+20)=125.$

The rate-distortion performance of conventional inter-view prediction for _Bk is, $J_{\mathrm{DCP}}(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)|+{\mathrm{\λ}}_{\mathrm{motion}}\×(r_d+r_h)=45+1.5\×(8+20)=87$

When using depth-predicted disparity for predictive coding, the rate-distortion performance of depth-assisted inter-view predictive coding of _Bk is, $J_{\mathrm{DADCP}}(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)|+{\mathrm{\λ}}_{\mathrm{motion}}\×r_h^{\′}=50+1.5\×\mathrm{twenty\; one}=81.5.$

Afterwards, the encoder selects the optimal inter prediction coding mode by comparing the rate-distortion performance of different prediction modes. For the current macroblock B _k , $\begin{array}{c}J=\min (J_{\mathrm{MCP}}(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m),J_{\mathrm{DCP}}(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d),J_{\mathrm{DADCP}}(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z))\\ =\min \left(\mathrm{125,87,81.5}\right)\\ =81.5\end{array}.$ Therefore, the optimal inter-frame predictive coding mode is depth-assisted inter-view predictive coding. After obtaining the optimal inter prediction mode, the encoder will perform a second rate-distortion optimization selection. The encoder will further compare the rate-distortion performance of the inter-frame prediction mode and the intra-frame prediction mode, and finally select the mode with the best rate-distortion to encode the current macroblock.

According to the method of the embodiment of the present invention, the disparity of coded macroblocks is estimated by depth to perform inter-view compensation prediction, which reduces the bit rate required for disparity coding in stereoscopic video coding and improves the efficiency of stereoscopic video coding.

Fig. 4 is a structural block diagram of a joint predictive coding system for stereoscopic video according to an embodiment of the present invention. As shown in FIG. 4 , the joint predictive coding system for stereoscopic video includes a division module 100 , a first prediction module 200 , a second prediction module 300 , a third prediction module 400 , a calculation module 500 and a selection module 600 .

The dividing module 100 is used for inputting stereoscopic video and dividing the stereoscopic video into multiple coded macroblocks.

Specifically, the stereoscopic video is input, pre-processing such as correction and alignment is performed on it, and then the processed stereoscopic video is divided into multiple coded macroblocks.

The first prediction module 200 is configured to predict the depth prediction disparity of the currently coded macroblock through a depth prediction method, and perform depth-assisted inter-view prediction coding on the current coded macroblock according to the depth prediction disparity.

Specifically, it is assumed that a stereoscopic video sequence only contains videos and depth map sequences of left and right viewpoints. The baseline distance between the left and right viewpoints is c, and the camera focal lengths of the left and right viewpoints are both f. The currently coded macroblock is B _k . B _k contains n _j pixels, and the depth value corresponding to each pixel is The depth prediction disparity of the currently coded macroblock B _k is predicted through the depth value of each pixel. Assuming that the depth value of the current coded macroblock B _k is the maximum likelihood value of the depth values corresponding to all pixels contained in B _k , z _k can be expressed as in, is the depth value of each pixel.

Fig. 2 is a schematic diagram of virtual viewpoint rendering according to an embodiment of the present invention. As shown in FIG. 2 , after the depth value corresponding to B _k is obtained, the disparity of the currently coded macroblock can be calculated through the mapping relationship between depth and disparity. The predicted disparity of the currently coded macroblock can be expressed as, Among them, d _k is the calculated disparity, f is the focal length, and c is the baseline distance between the left and right viewpoints. For a coding mode with quarter-pixel precision, d _k is rounded to the nearest quarter-pixel point position as the depth prediction disparity of the current coded macroblock.

The second prediction module 300 is configured to obtain a disparity vector through an inter-view matching method, and perform traditional inter-view predictive coding on the current macroblock according to the disparity vector.

The third prediction module 400 is configured to obtain a motion vector through a time-domain motion estimation method, and perform time-domain predictive coding on the currently coded macroblock according to the motion vector.

The calculation module 500 is used to calculate multiple rate-distortion performances of the current coded macroblock in inter-view prediction and compensation prediction.

Specifically, the encoder will calculate the rate-distortion performance in different prediction modes. Let the motion vector of the current coded macroblock B _k be Search parallax for The depth prediction disparity is

The rate-distortion performance of the motion compensation prediction of the current macroblock can be expressed as, the rate-distortion performance of the motion compensation prediction is obtained by the following formula, $J_{\mathrm{MCP}}(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)|+{\mathrm{\λ}}_{\mathrm{motion}}(r_m+r_h),$ in, is the motion vector, B _k is the current coded macroblock, and ref _m is The reference frame pointed to, X is each pixel in B _k , I is the brightness or chrominance component value corresponding to X, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, λ _motion is the Lagrangian multiplier of time domain prediction, r _m is the encoding bit rate required for encoding the motion vector, r _h is the encoding except motion The code rate required by other macroblock header information other than the vector.

The rate-distortion performance of the search parallax compensation prediction of the current macroblock is obtained by the following formula, $J_{\mathrm{DCP}}(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)|+{\mathrm{\λ}}_{\mathrm{motion}}(r_d+r_h),$ in, is the disparity obtained by inter-view matching, B _k is the current coded macroblock, and ref _d is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, X is each pixel in B _k , I is the luminance or chrominance component value corresponding to X, and λ _motion is the Lagrang of traditional inter-view prediction day multiplier, rd is the encoding bit rate required for encoding and _searching for the disparity vector.

In stereoscopic video, depth information can be regarded as side information for video coding. Therefore, we can assume that the encoder and decoder can simultaneously obtain the same reconstructed depth map. Therefore, the depth prediction disparity does not need to be encoded into the code stream. Therefore, the rate-distortion performance corresponding to the parallax compensation prediction of the current macroblock through the depth prediction parallax can be expressed as, $J_{\mathrm{DADCP}}(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)|+{\mathrm{\λ}}_{\mathrm{motion}}\×r_h^{\′},$ in, Disparity is calculated for depth, B _k is the current coded macroblock, and ref _z is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, X is each pixel in B _k , I is the luminance or chrominance component value corresponding to X, and λ _motion is the pull of depth-assisted inter-view prediction Grangian multiplier, r _h ′ is the code rate required for encoding the macroblock header information in the disparity compensation prediction mode based on the depth prediction disparity.

The selection module 600 is configured to select the prediction mode corresponding to the rate-distortion performance with the smallest rate-distortion performance as the prediction mode of the currently coded macroblock and perform coding.

Specifically, the encoder selects the prediction mode with the best rate-distortion as the prediction mode of the currently coded macroblock. Its selection process can be expressed as, $J=\min (J_{\mathrm{MCP}}(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m),J_{\mathrm{DCP}}(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d),J_{\mathrm{DADCP}}(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)),$ in, and Denote the rate-distortion performance of temporal prediction, the rate-distortion performance of traditional inter-view prediction, and the rate-distortion performance of depth-assisted inter-view prediction, respectively.

In one embodiment of the present invention, the current coded macroblock B _k is assumed to be an 8×8 macroblock in one frame of view 8 video in the “Book Arrival” sequence. Its corresponding depth value is shown in the following 8×8 matrix.

$z_{B_k}=\left[\begin{array}{cccccccc}62& 63& 62& 61& 63& 58& 62& 63\\ 61& 62& 62& 63& 61& 64& 65& 64\\ 62& 61& 57& 61& 63& 59& 63& 63\\ 67& 61& 58& 62& 61& 61& 66& 64\\ 63& 62& 62& 58& 60& 62& 63& 62\\ 61& 62& 61& 61& 61& 63& 60& 58\\ 62& 62& 61& 62& 62& 62& 60& 62\\ 64& 63& 61& 62& 62& 62& 60& 61\end{array}\right].$ For the current coded macroblock B _k , the maximum likelihood value zk of the corresponding depth values of all pixels contained in B _k is, $z_k=\underset{z_k^j}{\arg \max }\mathrm{prob}\left(\right\{z_k^j|j=\mathrm{1,2},...,{\mathrm{no}}_{B_k}\})=62.$ After obtaining the depth value of the currently coded macroblock B _k , the predicted disparity of the current coded macroblock is, $d_k=\frac{\mathrm{fc}}{z_k}=\frac{100\×10}{62}=16.13.$

For the encoding mode with quarter-pixel precision, after d _k is rounded to the nearest pixel, the disparity should be, d _k ′=[d _k ]=16.25. The encoder then performs traditional inter-view prediction based on the predicted disparity information. For the currently coded macroblock, its predicted disparity is 16.25. The encoder finds the corresponding reference macroblock in the corresponding frame of the viewpoint 10, and performs prediction. Let the sum of the absolute values of the forecast residuals be 50. In addition, the encoder will perform compensation prediction, instant domain prediction and traditional inter-view prediction on the current macroblock. In the time-domain prediction, it is advisable to set the motion vector of the current macroblock as 32, and the sum of the absolute values of the time-domain prediction residuals as 80. In the traditional inter-view prediction of inter-view, it is assumed that the disparity obtained by the encoder through block matching search is 16, and the sum of the absolute values of the residuals of the traditional inter-view prediction is 45.

In an embodiment of the present invention, the calculation module 500 calculates the rate-distortion performance in different predictive coding modes. For a macroblock B _k , the rate-distortion performance of its time-domain prediction is, $J_{\mathrm{MCP}}(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m)|+{\mathrm{\λ}}_{\mathrm{motion}}(r_m+r_h)=80+1.5\×(10+20)=125.$

The rate-distortion performance of conventional inter-view prediction for _Bk is, $J_{\mathrm{DCP}}(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d)|+{\mathrm{\λ}}_{\mathrm{motion}}\×(r_d+r_h)=45+1.5\×(8+20)=87$

When using depth-predicted disparity for predictive coding, the rate-distortion performance of depth-assisted inter-view predictive coding of _Bk is, $J_{\mathrm{DADCP}}(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)=\underset{x\∈B_k}{\mathrm{\Σ}}|I-I_p(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z)|+{\mathrm{\λ}}_{\mathrm{motion}}\×r_h^{\′}=50+1.5\×\mathrm{twenty\; one}=81.5.$

The selection module 600 compares the rate-distortion performance in different prediction modes through the encoder, and selects the optimal prediction coding mode. For the current macroblock B _k , $\begin{array}{c}J=\min (J_{\mathrm{MCP}}(\stackrel{\&Right\; Arrow;}{m},{\mathrm{ref}}_m),J_{\mathrm{DCP}}(\stackrel{\&Right\; Arrow;}{d_{\mathrm{the\; s}}},{\mathrm{ref}}_d),J_{\mathrm{DADCP}}(\stackrel{\&Right\; Arrow;}{d_z},{\mathrm{ref}}_z))\\ =\min \left(\mathrm{125,87,81.5}\right)\\ =81.5\end{array}.$ Therefore, the optimal inter-frame predictive coding mode is depth-assisted inter-view predictive coding. After obtaining the optimal inter-frame prediction coding mode, the encoder will perform a second rate-distortion optimization selection. The encoder will further compare the rate-distortion performance of the inter-frame prediction mode and the intra-frame prediction mode, and finally select the mode with the best rate-distortion to encode the current macroblock.

According to the system of the embodiment of the present invention, the disparity of coded macroblocks is estimated by depth to perform inter-view compensation prediction, which reduces the bit rate required for disparity coding in stereoscopic video coding and improves the efficiency of stereoscopic video coding.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be construed as limitations to the present invention. Variations, modifications, substitutions, and modifications to the above-described embodiments are possible within the scope of the present invention.

Claims (8)

1. a joint predictive coding method of stereoscopic video, is characterized in that, comprises the following steps: S0: Encode the depth map sequence using a multi-view video coding method, and input the decoded depth map information as side information for stereoscopic video color image coding; S1: Input a stereoscopic video color image sequence and divide each frame image in the stereoscopic video color image sequence into a plurality of coded macroblocks; S2: Predict the depth prediction disparity of the currently coded macroblock by a depth prediction method, and perform depth-assisted inter-view prediction coding on the current coded macroblock according to the depth prediction disparity; S3: Obtain the disparity vector of the currently coded macroblock by means of inter-view matching, and perform traditional inter-view predictive coding on the current macroblock according to the disparity vector; S4: Obtain the motion vector of the currently coded macroblock by means of time domain motion estimation, and perform time domain predictive coding on the current coded macroblock according to the motion vector; S5: Calculate the rate-distortion performance of the currently coded macroblock in the depth-assisted inter-view predictive coding, traditional inter-view predictive coding, and time-domain predictive coding modes; S6: Select the predictive coding mode with the best rate-distortion performance as the predictive mode of the currently coded macroblock and perform coding; S7: judging whether the coding of all the coded macroblocks is completed; S8: If it is not completed, repeat the steps S1-S5 for the uncoded macroblocks until all coded macroblocks are coded; Wherein, the depth-assisted inter-view predictive coding specifically includes calculating the depth prediction disparity for each pixel in the current coded macroblock, and performing a quarter of the depth prediction disparity vector of the current coded macroblock Quantization with one-pixel accuracy, and performing pixel-by-pixel predictive coding according to the depth prediction parallax to obtain a predictive coding residual. 2. the joint predictive coding method of stereoscopic video as claimed in claim 1, is characterized in that, the rate-distortion performance of described temporal domain predictive coding obtains by following formula, ${\mathrm{<span\; class="notranslate">\; J</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$ in, is the motion vector, B _k is the current coded macroblock, and ref _m is The reference frame pointed to, X is each pixel in B _k , I is the brightness or chrominance component value corresponding to X, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, λ _motion is the Lagrangian multiplier of motion compensation prediction, r _m is the encoding bit rate required for encoding the motion vector, r _h is the encoding except motion The code rate required by other macroblock header information other than the vector. 3. The joint predictive coding method of stereoscopic video as claimed in claim 1, is characterized in that, the rate-distortion performance of described traditional inter-view predictive coding is obtained by the following formula, ${\mathrm{<span\; class="notranslate">\; J</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$ in, is the disparity obtained by inter-view matching, B _k is the current coded macroblock, and ref _d is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, X is each pixel in B _k , I is the luminance or chrominance component value corresponding to X, and λ _motion is the Lagrangian predicted by motion compensation multiplier, rd is the encoding bit rate required for encoding and searching the _disparity vector. 4. The joint predictive coding method of stereoscopic video as claimed in claim 1, is characterized in that, the rate-distortion performance of the depth-assisted inter-view predictive coding is obtained by the following formula, ${\mathrm{<span\; class="notranslate">\; J</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}\mathrm{<span\; class="notranslate">\; A</span>}\mathrm{<span\; class="notranslate">\; D.</span>}\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>$ Wherein, B _k is the current coded macroblock, X is each pixel in B _k , and I is the brightness or chrominance component value corresponding to X, Calculate the disparity vector for the quantized depth corresponding to the pixel point X, ref _z is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, λ _motion is the Lagrangian multiplier of motion compensation prediction, r′ _h is the coded macroblock header information in the parallax compensation prediction mode based on depth prediction parallax required code rate. 5. A joint predictive coding system for stereoscopic video, comprising: Depth map codec module for encoding and decoding depth map sequences; A division module for inputting stereoscopic video and dividing the stereoscopic video into a plurality of coded macroblocks; The first prediction module is used to predict the depth prediction parallax of each pixel in the current coded macroblock through a depth prediction method, and perform depth-assisted inter-view prediction coding on the current coded macroblock according to the depth prediction parallax; The second prediction module is configured to obtain the disparity vector of the current encoding module through an inter-view matching method, and perform traditional inter-view prediction coding on the current macroblock according to the disparity vector; The third prediction module is used to obtain the motion vector of the current coding module through the method of time domain motion estimation, and perform time domain predictive coding on the current coded macroblock according to the motion vector; A calculation module, configured to calculate the rate-distortion performance of the currently coded macroblock in the depth-assisted inter-view predictive coding, traditional inter-view predictive coding, and time-domain predictive coding modes; The selection module is used to select the prediction coding mode with the best rate-distortion performance as the prediction mode of the current coded macroblock and perform coding; a judging module, configured to judge whether all coded macroblocks have been coded; and The processing module is used to repeatedly use the division module, the first prediction module, the second prediction module, the third prediction module, the calculation module and the selection module until the coding of all the coded macroblocks is completed when the coding is not completed; Wherein, the first prediction module calculates the depth prediction disparity for each pixel in the current coded macroblock, and quantizes the depth prediction disparity vector of the current coded macroblock with quarter-pixel precision, And perform pixel-by-pixel predictive coding according to the depth prediction parallax to obtain a predictive coding residual. 6. The joint predictive coding system of stereoscopic video as claimed in claim 5, is characterized in that, the rate-distortion performance of described temporal domain predictive coding obtains by following formula, ${\mathrm{<span\; class="notranslate">\; J</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$ in, is the motion vector, B _k is the current coded macroblock, and ref _m is The reference frame pointed to, X is each pixel in B _k , I is the brightness or chrominance component value corresponding to X, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, λ _motion is the Lagrangian multiplier of motion compensation prediction, r _m is the encoding bit rate required for encoding the motion vector, r _h is the encoding except motion The code rate required by other macroblock header information other than the vector. 7. The joint predictive coding system of stereoscopic video as claimed in claim 5, is characterized in that, the rate-distortion performance of described traditional inter-view predictive coding is obtained by the following formula, ${\mathrm{<span\; class="notranslate">\; J</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>$ in, is the disparity obtained by stereo matching, ref _d is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, X is each pixel in B _k , B _k is the current coded macroblock, I is the luminance or chrominance component value corresponding to X, and λ _motion is The Lagrangian multiplier of motion compensation prediction, rd is the encoding bit rate required for encoding and searching the _disparity vector. 8. The joint predictive coding system of stereoscopic video as claimed in claim 5, is characterized in that, the rate-distortion performance of the depth-assisted inter-view predictive coding is obtained by the following formula, ${\mathrm{<span\; class="notranslate">\; J</span>}}_{\mathrm{<span\; class="notranslate">\; D.</span>}\mathrm{<span\; class="notranslate">\; A</span>}\mathrm{<span\; class="notranslate">\; D.</span>}\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&Element;</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; p</span>}}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; ref</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>$ Wherein, B _k is the current coded macroblock, X is each pixel in B _k , and I is the brightness or chrominance component value corresponding to X, is the quantized depth prediction disparity vector corresponding to the pixel point X, and ref _z is the frame of reference pointed to, for The luminance or chrominance component value of the corresponding pixel in the pointed reference frame, λ _motion is the Lagrangian multiplier of motion compensation prediction, r′ _h is the coded macroblock header information in the parallax compensation prediction mode based on depth prediction parallax required code rate.