JP2006140618A - Three-dimensional video information recording device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a three-dimensional video information recording device and a program capable of efficiently and functionally reproducing both two-dimensional video and three-dimensional video. <P>SOLUTION: Depth information called a differential pack (D_PACK) 12 is packed and MPEG multiplexed in a format conforming with the DVD video standards to obtain a format in which three-dimensional video can be output by using the differential pack 12 and two-dimensional video being normal as the DVD video standards can be output when the differential pack is not used. Further, the depth information is described in a format which conforms with the DVD video standards and linked to DVD-video zone20 and DVD-others Zone21 to record the three-dimensional video and two-dimensional video compatibly with the DVD video standards. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a 3D video information recording apparatus and program that preferably realize recording and reproduction of 3D video information while maintaining compatibility with 2D video information.

  Conventionally, for example, Patent Document 1 discloses that on the screen, both the pixel points corresponding to each other in both directions with respect to the direction of the epipolar line connecting the corresponding pixel points of the stereo image related to the target object and the direction orthogonal thereto are included. A method of calculating a distance to a target object by performing a two-dimensional search to obtain a disparity vector is disclosed. With such a method for extracting depth, it has become possible to obtain depth information necessary for stereoscopic image display.

In Patent Document 2, data for binocular stereoscopic video is recorded as main data, conventional video is recorded as the other data, and the other parallax image is recorded as MPEG private stream and auxiliary data in the user area. A method for recording eye stereoscopic video data is disclosed.
JP 2000-28355 A Japanese Patent Laid-Open No. 9-327041

  However, in the method disclosed in Patent Document 2, since the image recorded as auxiliary data is predictively encoded as difference data from the image recorded in the main, the image quality of one-eye image is improved. There is also a device method such as switching alternately depending on the scene, but it was not an essential improvement.

  Conventionally, 3D video playback of content that can be displayed and reproduced by 3D video technology that can express space is recorded in a state compatible with the so-called DVD video standard, and is switched by a playback device for efficient and functional However, there is no device capable of reproducing both 2D video and 3D video, and there has been a demand for a device capable of reproducing both 2D video and 3D video.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a 3D video information recording apparatus and program capable of efficiently and functionally reproducing both 2D video and 3D video.

  The 3D video information recording apparatus according to the present invention comprises means for recording a video object of a 2D video, means for recording management information in which information used for special reproduction of the video object is described in a management information area, And means for multiplexing and recording information relating to the depth for each predetermined unit of each picture of the three-dimensional video as stream data different from the video of the video object.

  Further, the 3D video information recording apparatus of the present invention comprises means for recording a video object of 2D video, means for recording management information in which information used for special reproduction of the video object is described in a management information area, Means for recording information relating to the depth for each predetermined unit of each picture of the 2D video in a management information area for 3D video different from the management information area.

  In the 3D video information recording apparatus according to the present invention, the depth-related information may be obtained by performing pixel difference encoding, predictive encoding, run length encoding, or each picture of the 2D video in the plane of each picture of the 2D video. Recording is performed after encoding using at least one of encoding using in-plane orthogonal transform.

  The 3D video information recording program according to the present invention includes a step of recording a video object of a 2D video on a recording medium, and management information describing information used for special reproduction of the video object. Causing the computer to execute a step of recording in a region and a step of multiplexing information relating to the depth for each predetermined unit of each picture of the two-dimensional video as stream data different from the video object video and recording it on a recording medium. It is characterized by.

  The 3D video information recording program according to the present invention includes a step of recording a video object of a 2D video on a recording medium, and management information describing information used for special reproduction of the video object. And a step of causing the computer to record information on the depth for each predetermined unit of each picture of the 2D video in a management information area for 3D video different from the management information area on the recording medium. It is characterized by making it.

  In the 3D video information recording program of the present invention, the depth information may be obtained by performing pixel difference encoding, predictive encoding, run length encoding, or each picture of 2D video in the plane of each picture of 2D video. Recording is performed after encoding using at least one of encoding using in-plane orthogonal transform.

  The 3D video information recording apparatus of the present invention has means for multiplexing and recording information relating to the depth for each predetermined unit of each picture of the 2D video as stream data different from the video of the video object, so-called DVD. Since the video standard can be recorded in a compatible state, a system that can efficiently and functionally reproduce both 2D video and 3D video can be constructed.

  In addition, the 3D video information recording apparatus of the present invention has means for recording information about the depth for each predetermined unit of each picture of the 2D video in a management information area for 3D video different from the management information area. In addition, since recording can be performed with compatibility with the so-called DVD video standard, it is possible to construct a system capable of efficiently and functionally reproducing both 2D video and 3D video.

  In addition, according to the 3D video information recording program of the present invention, the information about the depth for each predetermined unit of each picture of the 2D video is multiplexed and recorded on the recording medium as stream data different from the video of the video object. Therefore, it is possible to record in a state compatible with the so-called DVD video standard, and it is possible to construct a system that can efficiently and functionally reproduce both 2D video and 3D video.

  Further, according to the 3D video information recording program of the present invention, the information about the depth for each predetermined unit of each picture of the 2D video is stored in the management information area for 3D video different from the management information area on the recording medium. Since recording is performed, recording can be performed in a state compatible with the so-called DVD video standard, and a system capable of efficiently and functionally reproducing both 2D video and 3D video can be constructed.

  In addition, by having depth information, it is possible to build a system without depending on the stereoscopic display method on the playback side, such as being limited to the twin-lens method. Further, since the data for stereoscopic viewing is created from the depth information, a safe system with little influence on the living body in stereoscopic viewing such as an image deterioration of only one eye can be constructed.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Example 1
A first embodiment of a 3D video information recording apparatus and program according to the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing a configuration of a 3D video information recording apparatus according to a first embodiment of the present invention, FIG. 2 is a diagram showing an example of a system using two imaging cameras, and FIG. 3 is for obtaining parallax of two screens 4 is an explanatory diagram of epipolar constraint conditions, FIG. 4 is an explanatory diagram of a format compatible with depth information and DVD, FIG. 5 is a block diagram showing a configuration of an encoding apparatus to which MPEG is applied, and FIG. 6 is an encoding apparatus shown in FIG. 7 is a block diagram showing the configuration of a decoding apparatus that decodes encoded data, FIGS. 7-1 to 7-7 are explanatory tables of MPEG video streams and video layers, and FIG. 8 is an MPEG multiplexed transport stream. FIG. 9 is a flowchart showing a 3D video information recording program using the 3D video information recording apparatus shown in FIG.

  As shown in FIG. 1, the 3D video information recording apparatus according to the present embodiment includes a pair of left and right imaging cameras 1A and 1B that capture a left image and a right image of a target object, and a two-dimensional image from the imaging camera 1A or 1B. A video compressor 2 that receives an image and the like and compresses the image, a disparity vector extractor 3 that extracts a disparity vector from the left image and the right image captured by the imaging cameras 1A and 1B, and a size of the disparity vector are calculated. Depth information calculator 4 that outputs depth information to depth information formatter 5, depth information formatter 5 that formats depth information, and information multiplexer that multiplexes depth information compressed by video compressor 2 6, a recorder 7 that records the multiplexed information on the recording medium 8, and a controller 9 that controls each unit provided in the apparatus.

  In the 3D video information recording apparatus shown in FIG. 1, first, a left image Pl and a right image Pr of a target object imaged by a pair of left and right imaging cameras 1A and 1B are obtained by a parallax vector extractor 3 in Japanese Patent Laid-Open No. 9-33249. Correspondence for each pixel is performed in accordance with an evaluation function such as a correlation function disclosed in the publication.

  Here, the two imaging cameras 1A and 1B are arranged so that their optical axes are included in the same XZ plane. As long as they are correctly arranged, the corresponding points need only be searched for on the scanning line which is an epipolar line, but in reality, it is rather rare that one pixel is arranged on the scanning line without any error. Therefore, the parallax vector extractor 3 performs parallax vector search calculation in the horizontal direction that is the epipolar line direction and a certain degree of vertical direction in the left image Pl and the right image Pr.

  A method of calculating the epipolar line direction will be described with reference to FIGS. As shown in FIG. 2, the two imaging cameras 1 </ b> A and 1 </ b> B are placed in parallel with a distance of the eye away from the target object 11, and a parallax vector is extracted from the obtained two images to obtain depth information. Calculate

  For example, as shown in FIG. 3, corresponding points that have been discriminated in advance or feature points that are very easy to discriminate are shown as a point R (Xl, Yl) in the left image Pl and a point S (Xr, Y1) in the right image Pr. Yr). The direction of the epipolar line Ep is obtained by connecting these two points R and S with a straight line.

  Here, since the direction of the epipolar line Ep is substantially parallel to the X-axis direction line, it is basically obtained by a horizontal search. For example, the position of the minimum value is obtained at the position X by using the difference sum of the pixels of a small block of 4 horizontal pixels and 2 vertical pixels, or the square sum of the differences as an evaluation parameter. If there is a possibility of vertical installation error in the camera, it is unavoidable to expand the vertical search range to the extent calculated by the product of the horizontal search range and Tanθ, assuming that the camera is inclined by the angle θ from the epipolar line Ep. Absent.

  A search is performed within the set search range. For example, when the point R (Xl, Yl) on the left image corresponds to the point S (Xr, Yr) on the right image, the search is performed at the point R (Xl, Yl). The disparity vector is represented as V (X1-Xr, Y1-Yr). Such a disparity vector V is obtained for all macroblocks on the left image. This process is performed over the entire screen, and the finally selected disparity vector V is output to the depth distance calculator 4.

  The depth distance calculator 4 calculates the magnitude of the parallax vector V, searches for a small block of, for example, horizontal 4 pixels and vertical 2 pixels, and obtains the position. The obtained horizontal position, vertical position, and depth direction are output to the depth information formatter 5 as (X, Y, Z).

  The depth information formatter 5 expands the depth information in the small block into pixels. The pixels of the small blocks may all have the same depth value, or may be processed so as to be slightly smoothly connected to adjacent depth data (for example, a low pass filter or the like is applied). For each image, the pixel depth data is arranged in the raster order from the upper left to the lower right and is formatted.

  FIG. 4 shows a schematic format example of the DVD video standard. Although details will be described later, the contents of pack data of 2 kB constituting depth information called differential pack (D_PACK) will be described here.

  When a plurality of differential packs (D_PACK) 12 are collected, data for each frame is obtained. There is a start code (StartCode) 14 at the beginning of the frame layer 13, for example, starting from 000001FF in hexadecimal. This is because the start code 14 is a system for encoding data such as MPEG using run length coding, discrete cosine transform (DCT), or variable length coder (VLC) as compared with data coding described later. Special data is selected and created so that it can be distinguished from other data.

  After the start code 14, in principle, the following four types are conceivable. (1) Describe 8 bits of depth information of each pixel in raster order. (2) When the value of each pixel shows the same value, the run length of how many continues is described in 7 bits and then the value is shown. (3) Each pixel is regarded as an 8-bit grayscale image and is encoded in the same manner as an MPEG I-picture image. However, in this case, since there is no color signal, the contents of the macro block (MB) in MPEG encoding use six 8 × 8 DCT blocks, whereas in the present invention, only four Y signals are used. Is encoded as follows. (4) In addition to the above MPEG encoding, encoding is performed using not only an I picture image but also a P picture and a B picture. FIG. 4 shows (2) of these.

  The first DE 15 indicates whether or not the depth information is valid. If this flag is valid, 1 is described. In the case of 1, the following depth information is valid. If invalid, 0 is described. In the case of 0, the following depth information is invalid, and the depth does not change at all with respect to the pixels, and is set uniformly to an OFFSET value described later.

  The next OFFSET 16 adds OFFSET to the depth information data with a value from −128 to +127. If the depth information data after applying the OFFSET is depth information exceeding 0 to 255 regions, the data width of the depth information is appropriately reduced to 8 bits using a limiter. That is, if DE15 is set to 0 and OFFSET16 is set to 0, the depth information of all the pixels in this frame is all set to 0 even if some depth information is described in a format described later.

  Next, 8 bits (run length from 1 to 256) are indicated by NumOfSkipPixel 17, and then depth information is indicated by values from −128 to +127 by ZP18. For example, if four pixels having depth data of 100 continue, instead of transmitting 100, 100, 100, 100 and 4 times, the same information is obtained only by setting NumOfSkipPixel = 4 and then ZP = 100. Can be transmitted.

  In the application of (2), the depth data of adjacent pixels are often very similar, so it is possible to record the difference data as described above by taking the difference. It is expected that the depth information is further compressed and the encoding efficiency is improved. Further, if MPEG is used, the encoding efficiency is further improved.

  Here, MPEG will be briefly described. MPEG is the name of an organization that examines video coding standards established in 1988 by ISO / IEC JTC1 / SC2 (International Organization for Standardization / International Electrotechnical Standards Meeting Technical Committee 1 / Technical Committee 2, now SC29). It is an abbreviation for “Moving Pictures Expert Group”.

  MPEG1 (MPEG Phase 1) is a standard for storage media of about 1.5 Mbps, and is intended for video compression for low transfer rates of JPEG and ISDN video conferences and videophones for the purpose of still image coding. H. It inherits the basic technology of H.261 (CCITT SGXV, standardized by the current ITU-T SG15), and introduces a new technology for storage media. These were established in August 1993 as ISO / IEC 11172.

  MPEG2 (MPEG Phase 2) is a general-purpose standard designed to be compatible with various applications such as communication and broadcasting, in November 1994, ISO / IEC 13818, H.264. It is established as 262.

  MPEG is created by combining several technologies. First, the time redundant part is reduced by taking the difference between the image decoded by the motion compensator and the input image.

  There are three modes of prediction from the past, the future, and both the past and the future. These modes can be switched and used for each macro block (MB) of 16 pixels × 16 pixels. The prediction direction is determined by the picture type given to the input image. There are two modes in which the prediction from the past and the macroblock are independently encoded without prediction are P pictures. The B picture has four modes: prediction from the future, prediction from the past, prediction from both the past and the future, and independent encoding. It is the I picture that all the macroblocks encode independently.

  In motion compensation, a motion region is subjected to pattern matching for each macroblock, a motion vector is detected with half-pel accuracy, and prediction is performed after shifting by the amount of motion. The motion vector has a horizontal direction and a vertical direction, and is transmitted as additional information of the macroblock together with an MC (Motion Compensation) mode indicating where the prediction is from.

  A picture from an I picture to a picture before the next I picture is called a GOP (Group Of Picture), and when used in a storage medium or the like, generally about 15 pictures are used.

  FIG. 5 is a block diagram showing the configuration of an encoding apparatus to which MPEG is applied. In FIG. 5, an input image signal is sent to a calculator 41 and a motion compensation predictor 50 described later.

  The computing unit 41 obtains the difference between the image signal decoded by the motion compensation predictor 50 and the input image signal, and sends the difference image signal to the DCT unit 42.

  The difference image signal is subjected to orthogonal transformation in the DCT unit 42. DCT (Discrete Cosine Transform) is an orthogonal transform that discretely transforms an integral transformation with a cosine function as an integral kernel into a finite space. In MPEG, a macroblock is divided into four and two-dimensional DCT is performed on an 8 × 8 DCT block. In general, a video signal has many low-frequency components and few high-frequency components. Therefore, when DCT is performed, coefficients are concentrated in a low frequency.

  The quantized image data (DCT coefficient) subjected to DCT is quantized by the quantizer 43. Quantization is a quantized value obtained by multiplying a value obtained by weighting the 8 × 8 two-dimensional frequency called the quantization matrix with a visual characteristic and a value called a quantization scale that is a scalar multiplication of the whole, and the DCT coefficient is quantized. Divide by value. When inverse quantization is performed by the decoder, a value approximating the original DCT coefficient is obtained by multiplying by the quantized value.

  The quantized data is variable length encoded by the VLC unit 44. Of the quantized values, the direct current (DC) component uses DPCM (differential pulse code modulation) which is one of predictive coding. The alternating current (AC) component is zigzag scanned from low to high, and a zero run length and effective coefficient value are taken as one event. Huffman encoding is performed.

  The variable-length encoded data is stored in the temporary buffer 45 and output as encoded data at a predetermined transfer rate. The generated code amount for each macroblock of the output data is transmitted to the code amount controller 51, and the error code amount with respect to the generated code amount with respect to the target code amount is fed back to the quantizer 43 to quantize the scale. The amount of code is controlled by adjusting.

  The quantized image data is inversely quantized by the inverse quantizer 46, inversely DCTed by the inverse DCT unit 47, and stored in the temporary image memory 49. Then, the motion compensated predictor 50 calculates a difference image. Used as a reference decoded image.

  FIG. 6 is a block diagram showing a configuration of a decoding apparatus that decodes encoded data encoded by the encoding apparatus shown in FIG. In FIG. 6, the encoded stream is buffered in the buffer 52, and the data from the buffer 52 is input to the VLD unit 53.

  The VLD unit 53 performs variable length decoding to obtain a direct current (DC) component and an alternating current (AC) component. The alternating current (AC) component data is arranged in an 8 × 8 matrix in the zigzag scan order from the low range to the high range. This data is input to the inverse quantizer 54 and is inversely quantized by the quantization matrix. The inversely quantized data is input to the inverse DCT unit 55, inversely DCTed, and temporarily output as image data (decoded data).

  The decoded data is temporarily stored in the image memory 57, and then used as a reference decoded image for calculating a difference image in the motion compensation predictor 58. The above is the description of MPEG.

On the other hand, if the description of the configuration of the 3D video information recording apparatus of the present embodiment shown in FIG. 1 is continued, with the above-described configuration, 128 is offset into the depth data and is regarded as a Y signal from 0 to 255. Compression can be used. In the syntax of the MPEG bitstream, it is described in (A) on the fifth line from the back of the MacroBlock layer shown in FIG.
for (i = 0; i <6; i ++)
In the present invention,
for (i = 0; i <4; i ++)
By doing so, only four Y signals can be provided for the macroblock.

  That is, the color difference signal processing portion corresponding to (B) in the 10th to 14th rows of the block layer shown in FIG. 7-7 is omitted.

  On the other hand, the image of the imaging camera 1B (or 1A) is output to the video compressor 2. The video compressor 2 receives the two-dimensional image from the imaging camera 1B (or 1A) and the depth information from the depth information formatter 5, and performs, for example, MPEG standard image compression.

  The depth information is output in a user data area or a private stream so that compatibility can be obtained in the MPEG regulations.

  Next, means for recording depth information while conforming to the MPEG standard will be described. As described above, the compressed depth information is multiplexed with video and audio as another stream by performing MPEG multiplexing in the differential pack (D_PACK) 12 shown in FIG. Alternatively, it may be described in the User data area of the Picture Layer of the MPEG bit stream shown in FIG. Further, the frame layer may be recorded in user data of one picture.

  For example, in the MPEG image compression standard, user data areas are set in the picture layer, the GOP layer, and the like. These are recorded in a data packet such as user_data or private_data_byte which is set as a predetermined area in which data unrelated to video and audio can be embedded in MPEG syntax, or private_stream which can be arbitrarily set by the user.

  For example, as shown in FIG. 7-4, a picture layer in MPEG1 video defines a mechanism that allows user_data to be recorded in units of 8 bits after sending user_data_start_code before the slice layer. Also, as shown in FIG. 8, in the system layer of the multiplexed transport stream such as MPEG2, when 1 is set in transport_private_data_flag, it can be clearly indicated that private_data exists, and the data length also protrudes from the transport packet. It is possible to send private_data with the data length set in transport_private_data_length under the restriction that there is no such.

  In addition to this, there are several mechanisms for recording user-specific data in the MPEG system, such as sending private data by declaring a dedicated packet with stream_id set to private_stream. Depth information can be recorded for each picture in these areas, and any mechanism may be used.

  An example using MPEG1 video user_data will be described in more detail. user_data_start_code is defined in MPEG as 0x000001B2 before the slice layer. After sending the code, a code of, for example, 0x0f0f0f0f2428fdaa, which is a uniquely identifiable code indicating the presence of the function value used for authentication of the present invention in the user data area, is transmitted. This code is recorded for the purpose of identification when user_data is used in another application, and the value of the code has no particular meaning. After the code, the frame layer structure shown in FIG. 4 is recorded in raster order on the picture layer for each MPEG picture.

  The data multiplexed by the information multiplexer 6 is recorded on the recording medium 8 by the recorder 7.

  Next, the 3D video information recording program of this embodiment will be described with reference to FIG. FIG. 9 is a flowchart showing a 3D video information recording program using the 3D video information recording apparatus shown in FIG. Since the detailed processing contents of the steps are the same as those described in the description of the block diagram of the 3D video information recording apparatus shown in FIG. 1, only the order of the steps will be briefly described here.

  First, in step S110, the control unit 9 controls the image data from the imaging cameras 1A and 1B to be input to the parallax vector extractor 3 for a predetermined time and stored in the memory of the control unit 9. Further, the control unit 9 causes the image of the imaging camera 1B (or 1A) to be output to the video compressor 2.

  Next, in step S120, the control unit 9 performs control so that the parallax vector of the image input to the parallax vector extractor 3 is extracted. Here, the parallax vector search calculation in the horizontal direction, which is the epipolar line direction, and a certain vertical direction described in the description of FIG. 1 is obtained for all macroblocks on the image. This process is performed over the entire screen, and the finally selected disparity vector V is output to the depth distance calculator 4.

  Next, in step S130, the control unit 9 controls the depth distance calculator 4 to calculate the macro block depth distance. That is, the size of the parallax vector is calculated, and for example, the position is determined by searching for a small block of 4 horizontal pixels and 2 vertical pixels. Then, the obtained horizontal position, vertical position, and depth direction are output to the depth information formatter 5 as (X, Y, Z).

  Next, in step S140, the control unit 9 controls the depth information formatter 5 to format the depth information sent from the depth distance calculator 4 as in the format example shown in FIG.

  Thereafter, in step S150, the control unit 9 controls the video compressor 2 to compress the two-dimensional image input from the imaging camera 1B (or 1A) and the depth information input from the depth information formatter 5. .

  Next, in step S160, the control unit 9 performs control so that the depth information compressed by the video compressor 2 is MPEG-multiplexed by the information multiplexer 6. For example, as described above, a 2 kB pack structure called the differential pack 12 shown in FIG. 4 is used for MPEG multiplexing and multiplexed with video and audio as another stream.

  Next, in step S170, the control unit 9 controls the data multiplexed by the information multiplexer 6 to be recorded on the recording medium 8 by the recording unit 7 in a predetermined unit. For a DVD, the unit is 2 kB.

  In step S180, the control unit 9 determines whether there is still input image data from the imaging cameras 1A and 1B. If there is such data (YES), the control unit 9 returns to step S110, and if this is not present (NO). Ends the program.

  As described above, according to the 3D video information recording apparatus and program of this embodiment, the depth information is recorded in the user data area of the MPEG1 video. Therefore, if the user data is not used, a 2D video is output and the user data is used. For example, a 3D video is output, and a system that can efficiently and functionally reproduce both 2D video and 3D video can be constructed.

  Next, an apparatus for reproducing 3D video information from the recording medium 8 on which 3D video information is recorded by the 3D video information recording apparatus of this embodiment will be described with reference to FIG. FIG. 10 is a block diagram showing a configuration of a 3D video information reproducing apparatus for reproducing 3D video information recorded on a recording medium by the 3D video information recording apparatus shown in FIG.

  First, data is read from the recording medium 8 by the regenerator 61 and output to the information separator 62. The information separator 62 separates the video signal and other signal packets and outputs the video signal to the video decoder 63. The video decoder 63 decodes the video and simultaneously outputs the user data of the video picture layer to the depth information extractor 64.

  Then, the depth information extractor 64 extracts depth information recorded in the format shown in FIG. 4 from the user data. The depth information and the decoded video signal are output to the view field converter 65.

  Next, the visual field converter 65 generates a parallax image according to the stereoscopic display method of the stereoscopic image display 66 from the depth information and the two-dimensional decoded image.

  Note that each unit provided in the 3D video reproduction apparatus shown in FIG. 10 is controlled by the control unit 67.

  In order to generate a parallax image, a visual field conversion method is used as a conversion method of a coordinate system in CG. This is to obtain an image in which the viewpoint is changed by the conversion formula to the viewpoint coordinate system, and if there is depth information, it can be generated from any viewpoint.

For example, the coordinates of the viewpoint _{(x i, y i, z} i), the coordinates of the gazing point _{(x a, y a, z} a) and. If the distance between the viewpoint and the point of gaze is (x _f , y _f , z _f ),
x _f = x _i −x _a
y _f = y _i −y _{a
z f} = z i -z _a
It becomes.

First, the position of the origin is moved by parallel movement. This conversion to _{T 1.} T ₁ is a transformation that simply translates (−x _a , −y _a , −z _a ).

Next, the direction of the coordinate value is changed by rotation. As shown in FIG. 11, a vector of O _f direction from O _a only first α angular vector of the z-axis from O _a rotated y-axis, then allowed to rotate in the β angle x-axis. Actually, becomes the direction of rotation reversed so move the coordinates of O _f, the conversion to -β rotate transform that -α rotating the y axis T _2, x-axis and T _3.

Here, alpha since is angle between foot and O _a obtained by projecting the O _f the xy plane,

It becomes.

Moreover, beta by _O a _O length between ^{_{f (x f 2 + y f}} 2) 1/2 and _O f _{O f} 'between length _{y f,}

It becomes.

The end of conversion, the positive direction of the z-axis of the coordinate system such that the forward direction, to convert T ₄ for reversing the direction of the z-axis relative to the xy plane. This simply sets z to -z.

When these four transformation matrices T _{1 to} T ₄ are multiplied, the viewpoint coordinate transformation matrix T is

It becomes.

  For example, if the stereoscopic display method of the stereoscopic image display 66 is a binocular stereoscopic display method using a parallax barrier, α that can be set according to the viewing distance is substituted into this equation, and β and γ are set to 0. Images with parallax for the right eye and for the left eye can be generated.

  In addition, in the case of IP (Integral Photography), the size of an image obtained by capturing an element image corresponding to a plurality of lens arrays with a camera corresponding to the lens position. At the same time, it can be calculated and generated using the transformation matrix of the viewpoint coordinates. The stereoscopic image data generated in this way is output to the stereoscopic image display 66 to reproduce the stereoscopic image.

  Here, typical parallax barrier methods and IP methods among the stereoscopic image display methods will be described.

  The parallax barrier can be realized by liquid crystal. FIG. 12 is an explanatory diagram of the parallax barrier system. This is a stack of two liquid crystal panels. One liquid crystal panel 71 has a thin slit-like opening, and the other liquid crystal panel 72 is arranged on the back side with an appropriate interval. On the liquid crystal panel 72, images for the left and right eyes are alternately arranged, and when viewed through this opening from a predetermined viewpoint, an image separated into the right eye and the left eye can be perceived. As a result, different images can be input to the right eye and the left eye, and can be perceived as a stereoscopic image.

  However, even though the eye is always focused on the screen of the liquid crystal panel 71, the image feels different from this position. It has been pointed out that it is easy to get sick and video sickness. In recent years, four stereoscopic physiological factors ((1) binocular parallax: when looking at an object, the left and right eyes of a human being in different directions (2) Focus adjustment: The property of changing the lens thickness by controlling the lens thickness as the distance from the viewing object changes (3) Convergence: Distant The property that the eyeball is accompanied by the movement of the eyeball rotating inward or rotating outward due to a close change (4) Motion parallax: The property of seeing the difference in the image due to the user moving himself or changing the viewing angle) Scheme to meet the convergence adjusting contradiction (inconsistency position to meet the convergence point and focus) has been proposed.

  Among them, as a promising method, a method announced by Lippmann in 1908 is an IP method. The IP obtains depth information of an object using a two-dimensionally arranged lens array (also referred to as a fly-eye lens, eyelid lens, compound eye lens, or the like).

  In the 1990s, a technology for generating moving images by IP was developed by replacing the recording by conventional photographic plates with electronic technology, and using a gradient index lens array (also called GRIN lens array) and a high-vision camera. While capturing a subject and acquiring elemental image groups, each image was transmitted and displayed in real time on a liquid crystal display, and was successfully imaged in space by a fly-eye lens. (NHK Broadcasting Technology Laboratories “Basics of 3D Video” Ohm Corporation 1995).

  FIG. 13 is an explanatory diagram of the principle of the IP method. As shown in FIG. 13A, a large number of minute lens elements 73 are arranged at the time of photographing, and when one pixel of the minute camera shoots and reproduces an image of light in one direction, as shown in FIG. The image from the micro camera is reproduced on the LCD 74 or the like, and one point of each of the micro cameras is gathered to form a reproduction image viewed from one direction as a whole.

  It is possible to create horizontal and vertical motion parallax by arranging minute element lenses 73 two-dimensionally, and it is also possible to have parallax only in the horizontal direction by arranging them in the horizontal direction.

  In the IP method, a plurality of element images viewed through a plurality of lenses are created by viewpoint conversion based on the depth information, and the element images are arranged in an array, as shown in FIG. Stereoscopic reproduction is realized by displaying the element image array on the LCD as if it were picked up by the method shown.

  Actually, as shown in FIG. 14, the IP lens array 76 of the LCD 75 is configured close to each other, and is designed as a thin display with a configuration similar to that for viewing a two-dimensional image.

  Next, a 3D video information playback program for playing back 3D video information information with the 3D video information playback apparatus shown in FIG. 10 will be described with reference to the flowchart shown in FIG. Since the detailed processing contents of the steps are the same as those described in the description of the block diagram of the 3D video information reproducing apparatus shown in FIG. 10, only the order of the steps will be briefly described here.

  First, in step S210, the controller 67 causes the player 61 to read the multiplexed data from the recording medium 8, and in step S220, the controller 67 causes the information separator 62 to perform MPEG separation. To control. In step S230, the control unit 67 performs control so that the video data is decoded from the information separated by the video decoder 63.

  In step S240, the control unit 67 controls the depth information extractor 64 to detect the depth information from the separated information.

  Next, in step S250, the control unit 67 performs control so that the field-of-view converter 65 performs field-of-view conversion of the video image using the depth information of the pixels. This is to obtain an image in which the viewpoint is changed by an equation for conversion to the viewpoint coordinate system.

  Thereafter, in step S260, the control unit 67 controls the stereoscopic image display 66 to perform stereoscopic display of video. The stereoscopic display method may be a binocular stereoscopic display method using a parallax barrier or an IP method.

  In step S270, the control unit 67 determines whether or not there is still display image data on the recording medium 8. If there is (YES), the process returns to step S210. If not (NO), the program is returned to the program. Exit.

(Example 2)
A second embodiment of the 3D video information recording apparatus and program of the present invention will be described with reference to FIG. FIG. 16 is a block diagram showing the configuration of the 3D video information recording apparatus according to the second embodiment of the present invention. In FIG. 16, the same components as those of the three-dimensional video information recording apparatus according to the first embodiment shown in FIG.

  As shown in FIG. 16, the 3D video information recording apparatus of the present embodiment is a two-dimensional image from a pair of left and right imaging cameras 1A and 1B that capture a left image and a right image of a target object, and the imaging camera 1A or 1B. A video compressor 2 that inputs an image and the like, compresses the image, a disparity vector extractor 3 that extracts a disparity vector from the left image and the right image captured by the imaging cameras 1A and 1B, and calculates the magnitude of the disparity vector Depth information calculator 4 that outputs depth information to depth information formatter 5, depth information formatter 5 that formats depth information, and output from depth information formatter 5 is converted to a DVD format and recorder 7. A DVD formatter 10 that outputs to the video compressor, an information multiplexer 6 that multiplexes the depth information compressed by the video compressor 2, and a DVD. Comprises a recording unit 7 for recording information from the formatting unit 10 and the information multiplexer 6 to the recording medium 8, and a control unit 9 that controls each unit provided in the apparatus.

  The operation of the block having the same configuration as that of the 3D video information recording apparatus shown in FIG. 1 is the same. However, as shown in FIG. 14, the information from the depth information formatter 5 is output to the DVD formatter 10, and the DVD formatter In 10, it is converted into the DVD format described below, sent to the recorder 7, and recorded on the recording medium 8.

  The 3D video information recording apparatus according to the present embodiment has a format compliant with the DVD video standard, and multiplexes information on the depth for each predetermined unit of each picture of the 2D video as stream data different from the video object video. And means for recording the data.

  As shown in FIG. 4, the DVD format has a recording layer as a volume space and is divided into a volume and file structure 19, a DVD-video zone 20, and a DVD-others zone 21, and the DVD-video zone 20 includes a video manager (VMG). ) 22 and a structure of video title set (VTS) 23 exists.

  The video manager 22 describes information such as identification information of the succeeding video title set 23 such as video manager information, start addresses and end addresses of various information itself, and from which video stream playback is started. In the video title set 23, Control Data 24 such as address information and identification information of audio and video data to be reproduced is described.

  The video manager 22 and the control data 24 are management information areas in which information essential for reproduction is recorded. Data in this area is recorded by a DVD formatter or a DVD format step, and is recorded by a DVD format decoder or a DVD format decoding step. Played.

  Thereafter, there is a set of video and audio multiplexed MPEG streams called a video object set (VOBS) 25, and there is a small unit MPEG stream called a video object (VOB) 26. Below the video object 26, there are further subdivided units of cells (CELL) 27 and further video object units (VOBU) 28, which have a structure substantially corresponding to the group of pictures (GOP) of the MPEG stream. Yes. The playback time of the video object unit 28 corresponds to the playback time of video data composed of one or more GOPs included in the video object unit 28, and the playback time is about 0.4 to 1.0 seconds. belongs to.

  In the video object unit (VOBU) 28, a navigation pack (NV_PACK) 29, an audio pack (A_PACK) 30, and a video pack (V_PACK) 31 are MPEG-multiplexed. In the first navigation pack 29, stream search information and the like are described. The video pack 31 includes data in which video compressed data is packed, and the audio pack 30 includes data in which audio compressed data is packed.

  Next, the 3D video information recording program of the present embodiment will be described with reference to FIG. FIG. 17 is a flowchart showing a 3D video information recording program using the 3D video information recording apparatus shown in FIG. Since the detailed processing contents of the steps are the same as those described in the description of the block diagram of the 3D video information recording apparatus shown in FIGS. 1 and 16, only the order of the steps will be briefly described here.

  First, in steps S310 to S360, processing similar to that in steps S110 to S160 in the flowchart shown in FIG. 9 is performed.

  Thereafter, in step S370, the control unit 9 controls the DVD formatter 10 to convert the depth information formatted by the depth information formatter 5 into the DVD format shown in FIG.

  Next, in step S380, the control unit 9 records the data multiplexed by the information multiplexer 6 and the data converted into the DVD format by the DVD formatter 10 in a predetermined unit by the recording unit 8. To control the recording. For a DVD, the unit is 2 kB.

  In step S390, the control unit 9 determines whether there is still input image data from the imaging cameras 1A and 1B. If there is (YES), the control unit 9 returns to step S310, and if not (NO). Ends the program.

  Thus, according to the 3D video information recording apparatus and program of the present embodiment, the above-described depth information data called the differential pack (D_PACK) 12 is packed and MPEG-multiplexed in a format compliant with the DVD video standard. If the differential pack 12 is used, a 3D image is obtained, and if this is not used, a standard 2D image can be output as a DVD video standard. Therefore, both 2D and 3D images can be efficiently and functionally used. It is possible to construct a system that enables playback of images.

  Next, an apparatus for reproducing 3D video information from the recording medium 8 on which 3D video information is recorded by the 3D video information recording apparatus of this embodiment will be described with reference to FIG. FIG. 18 is a block diagram showing a configuration of a 3D video information reproducing apparatus for reproducing 3D video information recorded on a recording medium by the 3D video information recording apparatus shown in FIG.

  First, data is read from the recording medium 8 by the playback device 61, and an MPEG stream is extracted from the DVD format by the DVD format decoder 68 and output to the information separator 62.

  After that, the same processing as that of the 3D video information reproducing apparatus shown in FIG.

  Next, a program for reproducing 3D video information by the 3D video information reproducing apparatus shown in FIG. 18 will be described based on the flowchart shown in FIG. Since the detailed processing contents of the steps are the same as those described in the description of the block diagram of the 3D video information reproducing apparatus shown in FIGS. 10 and 18, only the order of the steps will be briefly described here.

  First, in step S410, the control unit 67 causes the player 61 to read the multiplexed data from the recording medium 8, and in step S420, the control unit 67 causes the DVD format decoder 68 to decode the DVD format. Control.

  Thereafter, steps S430 to S480 perform the same processing as steps S220 to S270 of the flowchart shown in FIG.

(Example 3)
A third embodiment of the 3D video information recording apparatus of the present invention will be described with reference to FIG. FIG. 20 is an explanatory diagram of a compatible format of depth information and DVD. In FIG. 20, the same components as those of the format shown in FIG.

  The 3D video information recording apparatus according to the present embodiment has a format compliant with the DVD video standard, and information on the depth for each predetermined unit of each picture of the 2D video is for a 3D video different from the management information area. Means for recording in the management information area. This is a format that conforms to the DVD video standard, but does not record in the differential pack (D_PACK), but records the depth information in a freely usable area in a DVD standard conforming format called DVD others zone. Is.

  The configuration of the 3D video information recording apparatus of this embodiment is the same as that of the 3D video information recording apparatus of Embodiment 2 shown in FIG.

  In the DVD-others zone 21, a structure of a video manager (DVMG) 81 and a video title set (DVTS) 82 is described. The video manager 81 describes information such as identification information of the subsequent video title set 82 such as video manager information, start addresses and end addresses of various information itself, and from which video stream playback is started. The video title set 82 includes DVTS information (DVTSI) 83 in which control data such as address information and identification information of audio and video data to be reproduced is described.

  The video manager 81 and the DVTS information 83 are management information areas in which information necessary for reproduction is recorded. Data in this area is recorded by a DVD formatter or a DVD format step, and a DVD format decoder or a DVD format decoding step. Played by.

  After that, there is a set of video and audio multiplexed MPEG streams called a video object set (DVOBS) 84, and there is a small unit MPEG stream called a video object (DVOB) 85. Below the video object 85, there are units of subdivided cells (DCELL) 86, and further a video object unit (DVOBU) 87, which has a structure in which several frames of the frame layer of depth information are collected.

  It has the same structure as the 2D video data in the DVD-video zone 20, and each video object unit, cell, video object, etc. has the same number of frames (same playback time length), so that accessibility such as search is possible. Can be increased.

  As described above, according to the 3D video information recording apparatus of the present embodiment, the depth information data is described in a format linked to the DVD-video zone 20 and the DVD-others zone 21 in a format compliant with the DVD video standard. A video and 2D video can be recorded in compliance with the DVD video standard, and a system capable of efficiently and functionally reproducing both 2D video and 3D video can be constructed.

  A 3D video information recording program using the 3D video information recording apparatus of this embodiment is the same as the flowchart shown in FIG.

  Further, the 3D video information reproducing apparatus shown in FIG. 18 in the second embodiment reproduces the 3D video from the recording medium 8 on which the 3D video information is recorded by the 3D video information recording apparatus of the present embodiment. The program for reproducing the 3D video information from the recording medium 8 on which the 3D video information is recorded by the 3D video information reproducing apparatus of this embodiment is the same as the flowchart shown in FIG.

Example 4
Embodiment 4 of the 3D video information recording apparatus of the present invention will be described with reference to FIG. FIG. 21 is a block diagram showing the configuration of the 3D video information recording apparatus according to the fourth embodiment of the present invention.

  As shown in FIG. 21, the 3D video information recording apparatus according to the present embodiment creates 3D video data based on CG, and includes a CPU 91 having a built-in image signal generator 92 for generating CG images. The CG image generated by the image signal generator 92 is input, the video compressor 2 that compresses the image, the depth information formatter 5 that formats the depth information, and the depth information compressed by the video compressor 2 An information multiplexer 6 that multiplexes, a recorder 7 that records the multiplexed information on a recording medium 8, and a control unit 9 that controls each unit provided in the apparatus.

  Since CG does not require a camera for imaging, all processing is performed within the CPU 91. There is an image signal generator 92 in the CPU 91, and each is activated by dedicated software. The CG image data basically includes position information and depth information in advance for a small-sized image such as a polygon.

  Therefore, the position information of the polygon portion corresponding to the 4 × 2 macroblock described above can be easily calculated. The video data is input to the video compressor 2 and compressed. The compressed video data is multiplexed by the information multiplexer 6 together with the information formatted by the depth information formatter 5 as in the 3D video information recording apparatus of the first embodiment shown in FIG. Recording is performed on the recording medium 8 by the recorder 7.

  According to the 3D video information recording apparatus of this embodiment, 3D video data based on CG is created and depth information is recorded in the user data area of MPEG1 video. The same effect as the device can be obtained.

(Example 5)
Embodiment 5 of the 3D image information recording apparatus of the present invention will be described with reference to FIG. FIG. 22 is a block diagram showing the configuration of the 3D video information recording apparatus according to the fifth embodiment of the present invention.

  As shown in FIG. 22, the 3D video information recording apparatus according to the present embodiment receives a CPU 91 incorporating an image signal generator 92 for generating an image and a CG image generated by the image signal generator 92. A video compressor 2 that performs compression, a depth information formatter 5 that formats depth information, a DVD formatter 10 that converts the output from the depth information formatter 5 into a format and sends it to a recorder 7, and a video compressor 2, an information multiplexer 6 that multiplexes the depth information compressed in 2, a DVD formatter 10, a recorder 7 that records the output from the information multiplexer 6 on a recording medium 8, and each unit provided in the apparatus. And a control unit 9 for controlling.

  The 3D video information recording apparatus of the present embodiment takes a format conforming to the DVD video standard, and the operation of the block having the same configuration as that of the 3D video information recording apparatus shown in FIG. As shown in FIG. 4, the output from the depth information formatter 5 is input to the DVD formatter 10, formatted into the format shown in FIGS. 4 and 20, and then recorded on the recording medium 8 by the recorder 7.

  According to the 3D video information recording apparatus of this embodiment, if 3D video data based on CG is created and converted into a DVD format as shown in FIG. 4, the depth information data is in a format compliant with the DVD video standard. Are packed and MPEG-multiplexed, so that the same effect as the 3D video information recording apparatus of the second embodiment can be obtained.

  Also, as shown in FIG. 20, if the DVD is formatted, the depth information data is described in a format linked to the DVD-video zone 20 and the DVD-others zone 21 in a format compliant with the DVD video standard. An effect is obtained.

  In the first to fifth embodiments, the final information is recorded on the recording medium. However, the communication and broadcast-specific packetization is performed, and the communication (broadcast) packetizer 93 is used as shown in FIG. Thus, it can be easily configured to transmit to the communication (broadcast) network 94. In this case, packet information that has been packetized peculiar to communication or broadcasting is received and received and reproduced from the communication (broadcasting) network 94 via the communication (broadcasting) packet release unit 95 as shown in FIG. be able to.

  In the above description, the stereoscopic image display method has been described by the IP method or the parallax barrier method. I just need it.

  The 3D video information recording program of the present invention may be realized by a computer. This program may be read from a recording medium and loaded into a computer, or may be transmitted via a communication network and loaded into a computer.

  Further, in the present invention, it is possible to transmit data via any transmission medium such as communication and broadcasting without recording data on the recording medium. In this case, the recording apparatus should be used as a transmission apparatus. You can also. The playback device can also be used as a receiving device.

  In addition, the recording medium on which the signal data of the present invention is recorded has an effect peculiar to the medium in which the depth information of the three-dimensional video is recorded. It is possible to suitably realize the system that performs this.

  In addition, the definition of the medium includes not only a narrowly-defined medium that can record data, but also an electromagnetic wave and light for transmitting signal data. The information recorded on the recording medium includes data itself such as an electronic file in an unrecorded state.

  Further, although the video depth information has been described as being recorded for each picture, it may be about every 0.5 second or about every 1 second. In that case, it can be realized by using user data of the GOP layer of MPEG.

  Although the present invention has been described centering on video, it is effective even when audio data is present together with video and audio and video are multiplexed by MPEG multiplexing. The same applies to data such as sub-pictures and control information.

It is a block diagram which shows the structure of the three-dimensional video information recording device of Example 1 of this invention. It is a figure which shows the example of the system using two imaging cameras. It is explanatory drawing of the epipolar constraint conditions for calculating | requiring the parallax of 2 screens. It is explanatory drawing of the compatible format of depth information and DVD in Example 1. FIG. It is a block diagram which shows the structure of the encoding apparatus with which MPEG is applied. It is a block diagram which shows the structure of the decoding apparatus which decodes the encoding data encoded with the encoding apparatus shown in FIG. 4 is an explanatory table (No. 1) of an MPEG video stream video layer. 4 is an explanatory table (2) of an MPEG video stream video layer. 4 is an explanatory table (No. 3) of an MPEG video stream video layer. 4 is an explanatory table (part 4) of an MPEG video stream video layer. FIG. 10 is an explanatory table (No. 5) of an MPEG video stream video layer. FIG. 7 is an explanatory table (No. 6) of an MPEG video stream video layer. FIG. 10 is an explanatory table (No. 7) of an MPEG video stream video layer. FIG. It is an explanatory table of the multiplexed transport stream system layer of MPEG. It is a flowchart which shows the three-dimensional video information recording program of Example 1 of this invention. FIG. 2 is a block diagram showing a configuration of a 3D video information reproducing apparatus that reproduces 3D video information recorded on a recording medium by the 3D video information recording apparatus shown in FIG. 1. It is a vector diagram explaining visual field conversion. It is explanatory drawing of a parallax barrier system. It is explanatory drawing of the principle of IP system. It is explanatory drawing of the display of an IP system. It is a flowchart which shows the 3D video information reproduction | regeneration program which reproduces | regenerates 3D video information information with the 3D video information reproduction | regeneration apparatus shown in FIG. It is a block diagram which shows the structure of the three-dimensional video information recording device of Example 2 of this invention. It is a flowchart which shows the 3D video information recording program of Example 2 of this invention. FIG. 17 is a block diagram showing a configuration of a 3D video information reproducing apparatus that reproduces 3D video information recorded on a recording medium by the 3D video information recording apparatus shown in FIG. 16. FIG. 19 is a flowchart showing a program for playing back 3D video information by the 3D video information playback device shown in FIG. 18. FIG. FIG. 10 is an explanatory diagram of a compatible format of depth information and DVD in the third embodiment. It is a block diagram which shows the structure of the three-dimensional video information recording device of Example 4 of this invention. It is a block diagram which shows the structure of the three-dimensional video information recording device of Example 5 of this invention. It is a block diagram which shows the structure which transmits to a communication (broadcasting) network, without recording on a recording medium. It is a block diagram which shows the structure of the three-dimensional video information reproduction apparatus which reproduces | regenerates video information from a communication (broadcast) network.

Explanation of symbols

1A, 1B Imaging Camera 2 Video Compressor 3 Parallax Vector Extractor 4 Depth Distance Calculator 5 Depth Information Formatter 6 Information Multiplexer 7 Recorder 8 Recording Medium 9 Control Unit 10 DVD Formatter 12 Differential Pack (D_PACK)
13 Frame layer 14 Start code 15 DE
16 OFFSET
17 NumOfSkipPixel
18 ZP
19 Volume and File structure
20 DVD-video zone
21 DVD-others zone
22 Video Manager (VMG)
23 Video title set (VTS)
24 Control Data
25 Video Object Set (VOBS)
26 Video object (VOB)
27 cells (CELL)
28 Video Object Unit (VOBU)
29 Navigation Pack (NV_PACK)
30 audio pack (A_PACK)
31 Video pack (V_PACK)
61 Regenerator 62 Information Separator 63 Video Decoder 64 Depth Information Extractor 65 Field of View Converter 66 Stereo Image Display 67 Control Unit 68 DVD Format Decoder 81 Video Manager (DVMG)
82 Video Title Set (DVTS)
83 DVTS Information (DVTSI)
84 Video Object Set (DVOBS)
85 Video object (DVOB)
86 cells (DCELL)
87 Video Object Unit (DVOBU)
91 CPU
92 Image signal generator 93 Communication (broadcast) packetizer 94 Communication (broadcast) network 95 Communication (broadcast) packet release unit

Claims (6)

Means for recording a two-dimensional video object;
Means for recording management information in which information used for special reproduction of the video object is described in a management information area;
3D video information recording apparatus comprising: means for multiplexing and recording information relating to depth for each predetermined unit of each picture of a 2D video as stream data different from the video of the video object. Means for recording a two-dimensional video object;
Means for recording management information in which information used for special reproduction of the video object is described in a management information area;
3D video information recording apparatus comprising: means for recording information relating to depth in a predetermined unit of each picture of a 2D video in a management information area for 3D video different from the management information area .   The depth-related information is at least one of pixel differential encoding, predictive encoding, run length encoding, or encoding using in-plane orthogonal transform of each picture of the 2D video. The 3D video information recording apparatus according to claim 1, wherein the recording is performed after encoding using one of the two. Recording a two-dimensional video object on a recording medium;
Recording management information describing information used for special playback of the video object in a management information area of a recording medium;
A three-dimensional video information recording program for causing a computer to execute a step of multiplexing information relating to depth for each predetermined unit of each picture of a two-dimensional video as a stream data different from the video of the video object and recording it on a recording medium . Recording a two-dimensional video object on a recording medium;
Recording management information describing information used for special playback of the video object in a management information area of a recording medium;
3D video information for causing a computer to execute a step of recording information on depth for each predetermined unit of each picture of the 2D video in a management information area for 3D video different from the management information area in the recording medium. Recording program. The depth-related information is at least one of pixel differential encoding, predictive encoding, run length encoding, or encoding using in-plane orthogonal transform of each picture of the 2D video. 6. The three-dimensional video information recording program according to claim 4 or 5, wherein the three-dimensional video information recording program is recorded after encoding using one of the two.

Images