JP5355606B2 - Stereo video encoding method, apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a stereo video encoding method which realizes high compression of stereo video by an inter-view prediction, even when a base view and non-base views are encoded independently of each other by using an existing encoder. <P>SOLUTION: A stereo video encoding method is the one which accepts data consisting of a base view and non-base views as its input and outputs them as one bit stream. An inter-view prediction vector is generated by an inter-view prediction of a base view and non-base views, and the inter-view prediction vector is superposed, as data, on the non-base views to generate superposed non-base views. The superposed non-base views and the base view are combined to generate new non-base views. The base view is encoded by a specific encoding method, and the new non-base views are encoded by a specific encoding method, from this encoded data is removed the encoded data of the base view. The encoded base view and the encoded data which has had the encoded data of the base view removed are multiplexed, from which one bit stream is output. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to a stereo video encoding method, apparatus, and program for encoding a digital video of a stereo video in real time.

  As a technique for compressing and encoding a digital signal of a moving image, H.264 is used. H.264 High Profile is used for Blu-ray, One Seg, and the like. In order to compress a stereo moving image, that is, a two-viewpoint moving image using such an encoding technique, when the base view and the non-base view are encoded as independent moving images, sufficient compression efficiency cannot be realized. . In order to increase the compression efficiency of stereo moving images, it is essential to realize inter-view prediction.

  For example, as a technique for compressing and encoding a digital signal of a stereo moving image using inter-view prediction, H.264, which is also adopted in Blu-ray and the like. H.264 stereo high profile is attracting attention. H. In the H.264 stereo high profile, the left-eye video and the right-eye video can be encoded as one base view and one non-base view, respectively. The base view is H.264. H.264 high profile compatibility is maintained. H.264 can be decoded as a high profile. The non-base view is encoded with reference to a frame included in another view (inter-view prediction). By using inter-view prediction, high compression of a stereo moving image can be realized (Non-Patent Document 1).

Anthony Vetro, Thomas Wiegand, Gary J. et al. Sullivan: Overview of the Stereo and Multiview Video Coding Extensions of the H.C. H.264 / MPEG-4 AVC Standard, Proceedings of the IEEE Vol. 99, no. 4, April 2011 (To be issued)

  However, H.C. In the H.264 stereo high profile, there is no particular provision on the implementation, and if the base view and the non-base view are not encoded independently, the base view must be shared from the non-base view. There is a problem that an existing encoder such as the H.264 high profile encoding method cannot be used.

  Therefore, the present invention relates to H.264. H.264 performs inter-view prediction outside the H.264 high profile encoding apparatus, superimposes inter-view prediction vector data of a non-base view on image data as additional information on the non-base view, and encodes the non-base view. H.264 high profile encoding device by inputting into the existing H.264 encoding device. The present invention provides a stereo video encoding method, apparatus, and program for realizing high compression of a stereo video by inter-view prediction even when a base view and a non-base view are independently encoded by an H.264 high profile encoding apparatus.

In order to solve the above-described problems, the present invention inputs two types of video data, a base view that is one image of a stereo image and a non-base view that is the other image, and outputs the data as one bit stream. A video encoding method in a video encoding device, wherein the video encoding device generates an inter-view prediction vector by inter-view prediction of the base view and the non-base view, and the inter-view prediction A second step of superimposing data on a non-base view and outputting a superimposed non-base view, and a new non-base view by combining the superimposed non-base view and the base view used in the first step And a fourth step for encoding the base view used in the first step with a specific encoding method. And flop, and a fifth step of encoding the non-base view that is output by the third step in a particular coding scheme, from the encoded data by said fifth step, used in the first step The sixth step of deleting the encoded data of the base view, and the seventh step of multiplexing the encoded data of the fourth step and the sixth step and outputting one bit stream are executed.

In the video encoding method described above, in the video encoding method described above, in the first step, inter-view prediction is performed using an image input at a timing of an I picture in a GOP structure of a base view, and the video encoding is performed. The apparatus was further used in the ninth step in the fifth step and the ninth step in the fifth step using the parameters used in the encoding in the fourth step. And a tenth step of performing encoding using parameters.

  Further, the present invention provides the above-described video encoding method, the time information for encoding in units of pictures as parameters in the ninth step and the tenth step, and the code used when the base view image is encoded It is characterized by using a conversion parameter.

  The present invention is also characterized in that, in the video encoding method described above, the encoding parameter includes a parameter for determining a quantization matrix and a quantization value.

  The present invention also relates to a video encoding device that inputs two types of video data, a base view that is one image of a stereo image and a non-base view that is the other image, and outputs the data as one bit stream. An inter-view prediction unit that generates an inter-view prediction vector by inter-view prediction of the base view and the non-base view, and superimposes the inter-view prediction vector on a non-base view to generate a superimposed non-base view And an inter-view prediction vector superimposing unit that generates a new non-base view by combining the superimposed non-base view and the base view used in the inter-view prediction unit, and a base used in the inter-view prediction unit Generated by a first encoding unit that encodes a view using a specific encoding method and the inter-view prediction vector superimposing unit. A second encoding unit that encodes the new non-base view using a specific encoding method, and a base view used by the inter-view prediction unit from encoded data obtained by the second encoding unit. A base view stream deleting unit that deletes the encoded data of the first encoding unit, the encoded data obtained by the first encoding unit, and the encoded data obtained by the base view stream deleting unit, And a multiplexing unit that outputs a stream.

  The present invention is also a video encoding program for causing a computer to execute processing used to realize the above-described video encoding method.

As described above, according to the present invention, even when a base view and a non-base view are independently encoded using an existing encoding device, high compression of a stereo moving image by inter-view prediction is realized. can do.
In addition, according to the present invention, it is possible to make the parameters the same between the two encoding apparatuses, and as a result, it is possible to prevent encoding degradation.

It is a schematic block diagram which shows the structure of the stereo moving image encoder by one Embodiment of this invention. It is a figure explaining the operation | movement which produces | generates an inter-view prediction vector and superimposes on a non-base view. H. H.264 high profile encoder 7 and H.264. It is a figure explaining the method of transmitting / receiving data between H.264 high profile encoding apparatuses 8. FIG. It is a figure explaining the parameter 33 and the parameter 34. FIG. It is a figure which demonstrates concretely about the time information and the encoding parameter 43, the time information, and the encoding parameter 44. FIG.

  A stereo video encoding apparatus according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of a stereo video encoding apparatus according to an embodiment of the present invention.

  Reference numeral 1 denotes a base-view image frame, and reference numeral 2 denotes a non-base-view image frame, which is a stereo image frame input to the inter-view prediction unit 3. Here, the base view corresponds to, for example, the left eye image, and the non-base view corresponds to the right eye image. The inter-view prediction unit 3 calculates a prediction vector for the base view from the non-base view.

  Reference numeral 4 denotes an inter-view prediction vector superimposing unit that superimposes a prediction vector on a non-base view. Reference numeral 5 denotes a base-view image frame, which is the same as the base-view image frame 1 of FIG. 6 is a new non-base view in which the superimposed non-base view processed by the inter-view prediction vector superimposing unit 4 and the base view image used by the inter-view prediction unit 3 are combined. 7 shows the base view as H.264. H.264 encoding with high profile H.264 high profile encoding device.

  8 shows the superimposed non-base view as H.264. H.264 encoding with high profile H.264 high profile encoding device. 9 encodes a base view and a non-base view. H.264 high profile encoder 7 and H.264. 2 is a data path for transmitting / receiving data to / from the H.264 high profile encoding device 8. 10 is H. This is a syntax correction unit that replaces the syntax with the header of the H.264 stereo high profile. Usually, syntax such as a header can be realized by replacing programs. The syntax correction unit 10 is configured to display the H.264 of the superimposed non-base view. H.264 High Profile Bitstream The header is rewritten to a H.264 stereo high profile header.

  Reference numeral 11 denotes a base view stream deletion unit that removes an extra input base view stream. As a result, only the non-base view bit stream (13) is output. Reference numeral 12 denotes a base view bit stream, and reference numeral 13 denotes a non-base view bit stream. Reference numeral 14 denotes a multiplexing unit that multiplexes the base view bit stream 12 and the non-base view bit stream 13. 15 is an H.264 multiplexed by the multiplexing unit 14. This is a bit stream of H.264 stereo high profile.

FIG. 2 specifically illustrates a method of generating an inter-view prediction vector from a base view and a non-base view and superimposing the inter-view prediction vector on the non-base view.
In this figure, 21 is a sequence of a series of base views, and 22 is a sequence of a series of non-base views. Reference numerals 23 and 25 are pictures at the timing of becoming I pictures in GOP (Group Of Pictures) in base view encoding, and are included in the base view sequence 21. Reference numeral 24 denotes a non-base view picture input simultaneously with the picture 23, and is included in the non-base view sequence 22. 26 is a non-base view picture input simultaneously with the picture 25 and is included in the non-base view sequence 22.

  Reference numeral 27 denotes an inter-view prediction unit, which performs inter-view prediction on pictures at timings encoded with I pictures such as 23 and 24. By providing an inter-view prediction interval, the amount of computation per time in inter-view prediction can be reduced, and the circuit scale can be reduced. 28 superimposes the inter-view prediction vector obtained by the inter-view prediction unit 27 on the image data of the non-base view, so It is an inter-view prediction vector superimposing unit that encodes with H.264 high profile.

  The inter-view prediction unit 27 corresponds to the inter-view prediction unit 3 in FIG. 1, and the inter-view prediction vector superimposing unit 28 corresponds to the inter-view prediction vector superimposing unit 4 in FIG.

In FIG. H.264 high profile encoding device 7 and the H.264 high profile encoding device 7 of FIG. A method for transmitting and receiving data to and from the H.264 high profile encoding device 8 is specifically illustrated.
31 is an H.264 encoding base view. H.264 high profile encoding device. 32 is an H.32 encoding non-base view. H.264 high profile encoding device. 33 is a parameter for encoding picture 23 and picture 25 in FIG. H.264 high profile encoding device 31. 34 is a parameter used when coding the picture 24 and the picture 26 in FIG. H.264 high profile encoding device 32.

  35 is an H.35 code when encoding a base view and a non-base view. H.264 high profile encoding device 31 and H.264. 2 is a data path for transmitting and receiving data to and from the H.264 high profile encoding device 32. Specifically, the data path 35 is H.264. H.264 high profile encoding device 31 and H.264. Data transmission / reception between the hosts of the H.264 high profile encoding device 32 is realized. For example, the data path 35 transmits and receives data for each picture. In order to realize this encoding by data transmission / reception, the input of a series of pictures to the non-base view is delayed.

  H. The H.264 high profile encoding device 31 is similar to the H.264 high profile encoding device 31 shown in FIG. H.264 high profile encoding device 7, H.264. The H.264 high profile encoding device 32 is similar to the H.264 high profile encoding device 32 shown in FIG. This corresponds to the H.264 high profile encoding device 8.

FIG. 4 is a diagram for explaining specific parameters for the parameters 33 and 34 in FIG.
41 is an H.264 encoding base view. H.264 high profile encoding device. 42 encodes a non-base view. H.264 high profile encoding device. 43 is time information and encoding parameters when encoding the picture 23 and the picture 25 in FIG. Reference numeral 44 denotes time information and encoding parameters used when encoding the picture 24 and the picture 26 in FIG.

  Reference numeral 45 denotes a data path for transmitting and receiving data when encoding the base view and the non-base view. Specifically, the data path 45 is H.264. H.264 high profile encoder 41 and H.264. Data transmission / reception between the hosts of the H.264 high profile encoding device 42 is realized. For example, the data path 45 transmits and receives data for each picture. In order to realize this encoding by data transmission / reception, the input of a series of pictures to the non-base view is delayed.

  H. The H.264 high profile encoding apparatus 41 is similar to the H.264 high profile encoding apparatus 41 shown in FIG. H.264 high profile encoder 31, H.264. The H.264 high profile encoding device 42 is similar to the H.264 high profile encoding device 42 shown in FIG. This corresponds to the H.264 high profile encoding device 32.

FIG. 5 is a diagram specifically explaining the time information and the encoding parameter 43, the time information and the encoding parameter 44 of FIG.
51 is an H.264 encoding base view. H.264 high profile encoding device. 52 is an H.264 encoding non-base view. H.264 high profile encoding device. 53 are parameters for determining time information, quantization matrix, and quantization when coding the picture 23 and the picture 25 in FIG. 54 are parameters for determining time information, a quantization matrix, and quantization used when coding the picture 24 and the picture 26 in FIG.

  55 is an H.55 code when encoding a base view and a non-base view. H.264 high profile encoder 51 and H.264. 2 is a data path for transmitting / receiving data to / from the H.264 high profile encoding device 52. Specifically, the data path 55 is H.264. H.264 high profile encoder 51 and H.264. Data transmission / reception between the hosts of the H.264 high profile encoding device 52 is realized. For example, the data path 55 transmits and receives data for each picture. In order to realize this encoding by data transmission / reception, the input of a series of pictures to the non-base view is delayed.

  H. The H.264 high profile encoding device 51 is similar to the H.264 high profile encoding device 51 shown in FIG. H.264 high profile encoding device 41, H.264. The H.264 high profile encoding device 52 is similar to the H.264 high profile encoding device 52 shown in FIG. This corresponds to the H.264 high profile encoding device 42.

  In the above-described embodiment, the base view and the non-base view are encoded by independent encoding devices. Here, H. A case of encoding with H.264 high profile will be described. Using the base view as an input image, In the H.264 high profile encoding method, information for encoding a base view serving as a reference image used for inter-view prediction in a non-base view is retained. A prediction vector for inter-view prediction between the base view and the non-base view is input as additional information to the image data of the non-base view. A base view that is a reference image of a non-base view and a non-base view to which a non-base view and a prediction vector for inter-view prediction are added is used as an input image. In the method of encoding to H.264 high profile, information obtained by encoding the inter-view prediction vector and the base view serving as the reference image is used. H.264 encoding with high profile.

  The base view, which is a reference image for inter-view prediction, performs the same encoding using the data stored when encoding with the base view. A base view that is a reference image of a non-base view and a non-base view to which a prediction vector of a non-base view and inter-view prediction is added is referred to as H.264. The bit stream of the base view that becomes the reference image of the non-base view is deleted from the bit stream encoded by the H.264 high profile, and the bit stream is made only of the non-base view. Furthermore, the base view and non-base view bit streams are multiplexed into one bit stream. As a result, H.C. The H.264 high profile encoding method and apparatus are used to enable high compression of stereo moving images.

  Further, by recording a program for realizing the function of the stereo moving image encoding apparatus in FIG. 1 on a computer-readable recording medium, and causing the computer system to read and execute the program recorded on the recording medium. An encoding process may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

1, 5 Base-view image frame 2 Non-base-view image frame 3, 27 Inter-view prediction unit 4, 28 Inter-view prediction vector superimposition unit 6 New non-base view 7, 8, 31, 32, 41, 42, 51 52H. H.264 high profile encoding device 9, 35, 45, 55 Data path 10 Syntax modification unit 11 Base view stream deletion unit 12 Base view bit stream 13 Non-base view bit stream 14 Multiplexing unit 15 Bit stream 21 Base view Sequence 22 Sequence of non-base view 23, 25, 26 Picture 33, 34 Parameter 43, 44 Time information and encoding parameter 53, 54 Time information, quantization matrix, parameter for determining quantization

Claims (6)

A video encoding method in a video encoding apparatus that receives data of two types of video, a base view that is one image of a stereo image and a non-base view that is the other image, and outputs the data as one bit stream,
The video encoding device is
A first step of generating an inter-view prediction vector by inter-view prediction of the base view and the non-base view;
A second step of superimposing data on the inter-view prediction vector on a non-base view and outputting a superimposed non-base view;
A third step of outputting the superimposed non-base view and the base view used in the first step as a new non-base view;
A fourth step of encoding the base view used in the first step with a specific encoding method;
A fifth step of encoding the non-base view output in the third step with a specific encoding method;
A sixth step of deleting the encoded data of the base view used in the first step from the encoded data in the fifth step;
A video encoding method for executing the seventh step of multiplexing the encoded data of the fourth step and the sixth step and outputting one bit stream. In the first step, inter-view prediction is performed using an image input at the timing of becoming an I picture in the GOP structure of the base view,
The video encoding device further includes:
A ninth step in which the parameters used in the encoding in the fourth step are used in the encoding in the fifth step;
A tenth step of performing encoding according to the parameters used in the ninth step in the fifth step;
The video encoding method according to claim 1, further comprising: The time information for encoding in units of pictures and the encoding parameter used when the base view image is encoded are used as parameters in the ninth step and the tenth step, respectively. Video encoding method.   The video encoding method according to claim 3, wherein the encoding parameter includes a parameter for determining a quantization matrix and a quantization value. A video encoding device that inputs data of two types of video, a base view that is one image of a stereo image and a non-base view that is the other image, and outputs the data as one bit stream,
An inter-view prediction unit that generates an inter-view prediction vector by inter-view prediction of the base view and the non-base view;
The inter-view prediction vector is superimposed on the non-base view to generate a superimposed non-base view, and a new non-base view is combined with the base view used by the inter-view prediction unit. An inter-view prediction vector superimposing unit generated as a base view;
A first encoding unit that encodes the base view used in the inter-view prediction unit using a specific encoding method;
A second encoding unit that encodes a new non-base view generated by the inter-view prediction vector superimposing unit using a specific encoding method;
A base view stream deletion unit that deletes encoded data of the base view used in the inter-view prediction unit from the encoded data obtained by the second encoding unit;
A multiplexing unit that multiplexes the encoded data obtained by the first encoding unit and the encoded data obtained by the base-view stream deletion unit, and outputs one bit stream. A video encoding device.   A video encoding program for causing a computer to execute processing used to realize the video encoding method according to any one of claims 1 to 4.

Images