JP2012120220A - Video information recording method, video information reproducing method, video information recording apparatus, video information reproducing apparatus, and optical disc - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stereoscopic video recording method, a stereoscopic video recording medium, a stereoscopic video reproducing method, a stereoscopic video recording apparatus, and a stereoscopic video reproducing apparatus that are capable of more optimally recording and reproducing stereoscopic video.SOLUTION: A piece of solving means of the present invention is a stereoscopic video recording method for recording stereoscopic video composed of left-eye video 2 and right-eye video 1 utilizing parallax video in a recording medium using a stereoscopic video recording apparatus 6. Then, from video content containing the stereoscopic video, a variation amount of a parallax angle having a predetermined value or greater, a variation time required for a variation in the parallax angle, and the number of variation times of the parallax angle are obtained, an evaluation value corresponding to a degree of eye fatigue is calculated based on the variation amount, the variation time, and the number of variation times, and the video content is encoded so that the evaluation value is within a predetermined range and is recorded in the recording medium.

Description

  The present invention relates to a stereoscopic video recording method, a stereoscopic video recording medium, a stereoscopic video playback method, a stereoscopic video recording device, and a stereoscopic video playback device.

  As an encoding technique for compressing a digital video signal progresses, it is possible to realize an optical disc apparatus excellent in searchability and operability by recording the compressed video signal on an optical disc. Such an optical disc apparatus records video as a digital signal, so there is no deterioration due to dubbing compared to the case of recording an analog video signal, and it is optical recording and reproduction, so it is non-contact and excellent in reliability.

  On the other hand, as an encoding method for compressing data of the digital video signal as described above, for example, there is a method based on the MPEG (Moving Picture Coding Experts Group) method, but this compression method is compared with an in-plane compression method such as motion JPEG. Although the compression efficiency is good, since motion compensation prediction in the time direction is used, it is possible to search only in units of video (GOP) consisting of a plurality of video groups, and access on the disk is also made to I picture consisting of in-plane compression. There were constraints such as having to access first. Therefore, Patent Document 1 proposes to devise a data format on the optical disc.

  Further, the data format of Patent Document 1 is only for filing a two-dimensional plane image. When a 3D image is filed, the right and left eye images are separately displayed in the first and second fields of the TV signal, and the two images are displayed only by the respective eyes with deflection glasses or the like. It is necessary to adopt the method of Patent Document 2.

JP 2005-260988 A JP 2007-166651 A

  When viewing stereoscopic (hereinafter also referred to as 3D) video for a long time, the frequency of moving the eyes in the focus direction increases, and there is concern that symptoms such as eye fatigue and 3D sickness will occur frequently, especially in the depth direction. When the movement of the points is intense, there are cases where 3D viewing is hindered. In particular, in the display of subtitles and the like, it is necessary to switch the focus of the eyes one after another, and there is a concern that the symptoms of eye fatigue and 3D sickness are further increased with respect to normal 3D images.

  In recording of stereoscopic video information, when a stereoscopic video using the parallax between the right-eye video and the left-eye video is filed on the optical disc, planar (hereinafter also referred to as 2D) video and stereoscopic (3D) video are mixed. There was a problem that could not support the video stream. In particular, when a display device is switched to a playback device that is playing back a 3D image or when a new connection is attempted, information necessary for 3D display cannot be linked and sent to the display device side. There were problems such as being unable to switch.

  In addition, when recording stereoscopic video information, when stereoscopic video using the parallax between right-eye video and left-eye video is filed on an optical disc, it is possible to record stereoscopic video implemented by broadcasting, and to set display devices and playback devices. However, there is a problem that image control information is not provided.

  In addition, in the distribution contents such as movies, the pre-viewing business such as a movie theater and the media distribution are set to have different release times, and a code for limiting the viewable area is provided from the situation of the movie distribution. In addition, when the distribution content is a stereoscopic video, it is further affected by the spread status of the corresponding display device, so that it is necessary to further divide the viewable area by 3D video and 2D video. However, the conventional system has a problem that does not correspond to this.

  In addition, in a stereoscopic video stream, information related to content, information related to copyright, information related to 3D display method, information related to OSD display, multi-angle information, etc. cannot be superimposed on the stream, so that it is not displayed on the playback device side. There was a problem that device settings could not be changed instantaneously on the device side.

  In addition, in 3D video information using parallax, there are left eye information and right eye information as compared to normal 2D video information, and the amount of information is doubled as it is, so that there is a problem that file efficiency deteriorates. there were.

  In addition, when playing 3D video, eyes with a high degree of stereo are very tired, so warnings may be issued to the user in advance or viewing may be restricted based on the user's age, etc., depending on the degree of stereo of the content. There was a problem that could not be. Furthermore, there is a problem in that, when a 3D video is played back, eye strain is increased in a video with a high degree of 3D, so that it is impossible to reduce the 3D level of the content on the playback device side.

  Therefore, the present invention is an invention made to solve the above-described problem, and a stereoscopic video recording method, a stereoscopic video recording medium, a stereoscopic video reproduction method, and a method for more optimally recording and reproducing a stereoscopic video, It is an object to provide a stereoscopic video recording device and a stereoscopic video playback device.

  According to the solution means of the present invention, there is provided a video information recording method for recording a stereoscopic video using parallax consisting of a left-eye video and a right-eye video on an optical disc, wherein either the left-eye video or the right-eye video is used as a main video. The main video includes an I picture compressed in a frame, a P picture compressed by applying motion compensation by the I picture in the forward direction, and the I picture in the forward and backward direction, or the When recording 3D video information configured as a digital main video information block including a B picture compressed by motion compensation by a P picture on an optical disc, the sub video which is the other of the main video is recorded in the main video. As sub-picture information corresponding to a picture of the video, differential compression is further performed using motion detection with the main video, and video data is compared with the main video information block. It is generated as a digital sub-picture information block of a sub-picture with a reduced amount, and the main picture information block and the sub-picture information block are alternately arranged in time for each picture information block on the optical disc. The video data of the main video information block and the sub-video information block are rearranged alternately, and the graphics data related to the main video information block and the sub-video information block is related to the video information block and the sub-video information block. Recording data is generated so that the graphics data and the audio data are related and arranged together, and at the time of recording the stereoscopic video information, the main video information block and the sub-data block in the optical disc are recorded. An area separate from the video information block and the optical disc Graphics displayed together with the stereoscopic video in the main video information block and the sub video information block for the video supplementary information recording area related to the video information provided in two areas on the inner peripheral side and the outer peripheral side Recording of auxiliary information including depth information for specifying the jumping out of depth information or depth for each graphics information display unit to be displayed, and for the video information block unit of the video information block unit with respect to the video auxiliary information recording area A recording area for table information composed of auxiliary information is secured, and 2D video is recorded by recording 3D video auxiliary information indicating whether the video information block is a main video information block or a sub video information block that is 3D video. In the case of reproduction, the main video and graphic data not subjected to the differential compression are reproduced, and a stereoscopic video is reproduced. 3D video auxiliary information of the video information block in the additional information recording area for decoding the differentially compressed video and reproducing the graphic data with both the main video information block and the sub-video information block And video information of an optical disc that can be displayed by superimposing stereoscopic graphics on the stereoscopic video using the parallax by reproducing the graphics depth information for stereoscopic viewing of the graphics. A recording method is provided.

  According to another solution according to the present invention, there is provided a video information reproducing method for an optical disc on which a stereoscopic video using a parallax composed of a left-eye video and a right-eye video is recorded, wherein the information recorded on the optical disc is One of the video and the right-eye video is the main video, and the main video is an I picture that is data-compressed within a frame, and a P-picture that is data-compressed by applying motion compensation by the I picture in the forward direction in time. , A digital main video information block including a B picture that is data-compressed by applying motion compensation by the I picture or the P picture in the front-rear direction and a sub-picture that is the other of the main pictures Is a differential sub-compression digital sub-picture information block using motion detection as sub-picture information corresponding to the picture of the main picture. The main video information block and the further compressed sub-video information block are alternately arranged in time for each video information block on the optical disc, and the main video information block Graphics data related to the sub-video information block is arranged as a combination of the video information block with respect to the video data, and the main video information block and the sub-video information block in the optical disc are In a video ancillary information recording area related to the video information, which is provided in another area and in two areas on the inner peripheral side and the outer peripheral side of the optical disc, in the main video information block and the sub-video information block, Specifies the pop-up or depth of graphics information displayed with 3D images. Is recorded for each graphics information display unit in which supplementary information, which is depth information to be displayed, is displayed, and further, in the video supplementary information recording area, recording of table information including supplementary information related to the video information block in the video information block unit 2D video is reproduced on an optical disc in which 3D video supplementary information indicating whether the video information block is a main video information block or a sub video information block that is a 3D video is recorded in the table information region. In this case, when the recording area of the video data is reproduced, the sub-picture is jumped to reproduce the graphics data area where the main video information block and the graphics data are arranged, and also reproduce the stereoscopic video. The sub-video information subjected to the differential compression with the main video information block. Both the block and the graphics data area are reproduced, and the video information stored in the table information includes additional information including depth information specifying the pop-up or depth of the graphics information in the video additional information recording area. By reproducing 3D video supplementary information on whether the information block is a main video information block or a sub-video information block, the sub-video information block on which the differential compression has been performed in the video information block is decoded and displayed synchronously. The main video and the sub video, which are stereoscopically represented as parallax images, are separated, and in the separated video information, three-dimensional graphic information with a parallax is added to the parallax video in accordance with graphics depth information. An optical disc video information reproducing method is provided.

  According to another solution of the present invention, one of the left-eye video and the right-eye video is used as a main video, the main video is an I picture compressed in a frame, and the I picture forward in time A digital main video information block including a P picture compressed by adding motion compensation according to, and a B picture compressed by adding motion compensation by the I picture or the P picture in the longitudinal direction in time. In addition, a sub-picture that is the other of the main pictures is a sub-picture information block corresponding to a picture of the main picture as a digital sub-picture information block that is further differentially compressed using motion detection as sub-picture information. The main video information block and the further compressed sub video information block are configured for each video information block on the optical disc. For the graphics data related to the main video information block and the sub video information block, the recording data is generated so as to be collectively arranged as a combination of video data of the video information block, In addition, the video ancillary related to the video information provided in two areas on the optical disc, which are different from the main video information block and the sub-video information block, and on the inner peripheral side and the outer peripheral side of the optical disc. In each information recording area, each graphics information display unit in which auxiliary information including depth information for specifying the pop-up or depth of graphics information displayed together with the stereoscopic video in the main video information block and the sub-video information block is displayed. In the video ancillary information recording area, the video is recorded. A recording area of table information composed of supplementary information related to the video information block in information block units is secured, and 3D video supplementary information indicating whether the video information block is a main video information block or a sub video information block that is a 3D video is provided. In the optical disk apparatus for reproducing the optical disk recorded in the table information area, the auxiliary information includes an optical disk drive for reproducing recorded data from the optical disk, and the main video and the sub-video arranged alternately from the optical disk. System decoding means for extracting and separating based on the 3D video supplementary information, video decoding means for decoding the extracted and separated main video information block, and extracting and separating and differentially compressing the other main video The second sub-picture is decoded using the motion vector of the main picture Video decoding means, OSD generation means for generating graphics data, and the graphics data based on the depth information in the accompanying information reproduced from the optical disc drive in the generated graphics data. There is provided a video information reproducing apparatus having OSD depth generating means for generating stereoscopic graphics data with parallax and blending means for superimposing the stereoscopic graphics data on the stereoscopic video.

  According to another solution of the present invention, there is provided a video information recording apparatus for recording a stereoscopic video using a parallax composed of a left-eye video and a right-eye video on an optical disc, wherein the video information recording apparatus includes a left-eye video and a right-eye video. Either one of the videos as a main video, an I picture whose data is compressed in a frame in the main video, a P picture which is data compressed by adding motion compensation by the I picture forward in time, and temporal A video encoding means for generating a digital main video information block including a B picture compressed by applying motion compensation by the I picture or the P picture in the front-rear direction, and a sub-picture that is the other of the main pictures, The sub-picture information corresponding to the picture of the main picture is further subjected to differential compression using motion detection with the main picture, and the main picture information block A second video encoding means for generating a digital sub-picture information block of a sub-picture with a reduced amount of video data, and the main picture information block and the sub-picture information block for each picture information block on the optical disc; The video data of the main video information block and the sub video information block are alternately rearranged so as to be alternately arranged in time, and the graphics data related to the main video information block and the sub video information block Also, format encoding means and buffer memory means for generating data so as to be collectively arranged as a combination of video data of the video information block, OSD encoding means for generating graphics information, and writing data to the optical disc Data modulation means and optical head for In addition, the format encoding means, when recording the stereoscopic video information, is a region different from the main video information block and the sub-video information block in the optical disc, and on the inner and outer peripheral sides of the optical disc. Projection or depth of graphics information displayed together with the stereoscopic video in the main video information block and the sub video information block in the video supplementary information recording area related to the video information provided in two areas. Attached information including depth information to be specified is recorded for each display unit of graphics information to be displayed, and table information including incidental information related to the video information block in the video information block unit is recorded in the video incidental information recording area. A recording area is secured, and the video information block is the main video information. 3D video supplementary information indicating whether it is a block or a 3D video sub-video information block, and includes stereoscopic video information including sub-video information with a reduced video data amount and stereoscopic graphics information. A video information recording apparatus for recording on an optical disc is provided.

  According to another solution according to the present invention, the information recorded on the optical disc includes either one of the left-eye video and the right-eye video as a main video, the main video is an I picture compressed in a frame, and a time A P picture compressed by adding motion compensation by the forward I picture, and a B picture compressed by adding motion compensation by the I picture or the P picture temporally forward and backward. It is configured as a digital main video information block, and the sub video that is the other of the main video is further differentially compressed using motion detection as sub video information corresponding to the picture of the main video. The main video information block and the further compressed sub video information block are formed on the optical disc. The video data blocks are alternately arranged in time, and the graphics data related to the main video information block and the sub video information block are combined in combination with the video data of the video information block. And the video information provided in two areas on the optical disc that are different from the main video information block and the sub-video information block and on the inner circumference side and the outer circumference side of the optical disc. In the video ancillary information recording area related to the video, the supplementary information which is depth information for designating the pop-up or depth of graphics information displayed together with the stereoscopic video in the main video information block and the sub-video information block is displayed. And is recorded for each graphics information display unit. In the area, a recording area of table information composed of incidental information related to the video information block in the video information block unit is secured, and indicates whether the video information block is a main video information block or a sub-video information block that is 3D video An optical disc in which 3D video supplementary information is recorded in the table information area is provided.

  According to each of the above solutions, the stereoscopic video data processed by the compression processing with high compression efficiency can be optimally arranged on the optical disc in a form compatible with the planar video reproduction, and is used. be able to.

1 is a conceptual diagram of a stereoscopic video display system using a shutter according to Embodiment 1 of the present invention. It is a conceptual diagram of the three-dimensional video display system using the polarizing filter which concerns on Embodiment 1 of this invention. It is a conceptual diagram of the three-dimensional video display system using the rotation polarizing filter which concerns on Embodiment 1 of this invention. 1 is a conceptual diagram of a stereoscopic video display system using multiplexed information according to Embodiment 1 of the present invention. It is a schematic diagram of the three-dimensional image which concerns on Embodiment 1 of this invention. FIG. 3 is a schematic diagram of a stereoscopic video including subtitle display according to Embodiment 1 of the present invention. It is a figure for demonstrating the eye fatigue degree which concerns on Embodiment 1 of this invention. It is a figure for demonstrating the acceleration and the eye fatigue degree of the depth direction which concern on Embodiment 1 of this invention. It is a figure for demonstrating the display range of the caption which concerns on Embodiment 1 of this invention. It is a figure which shows the information recording area on the recording medium based on Embodiment 2 of this invention. It is a figure for demonstrating the video stream structure of the video title which concerns on Embodiment 2 of this invention. It is a figure for demonstrating the video stream structure of the video title which concerns on Embodiment 2 of this invention. It is a figure for demonstrating the video stream structure of the video title which concerns on Embodiment 2 of this invention. It is a figure for demonstrating the content information of the incidental information which concerns on Embodiment 2 of this invention. It is a figure for demonstrating the time code information of the incidental information which concerns on Embodiment 2 of this invention. It is a figure for demonstrating the arrangement | positioning information of incidental information which concerns on Embodiment 2 of this invention. It is a figure for demonstrating the video information of the incidental information which concerns on Embodiment 2 of this invention. It is a conceptual diagram of TV display using the incidental information which concerns on Embodiment 2 of this invention. It is a trial figure of the multi-angle information which concerns on Embodiment 2 of this invention. It is a figure for demonstrating the OSD information which concerns on Embodiment 2 of this invention. It is a figure for demonstrating the GOP table information of the video control information which concerns on Embodiment 2 of this invention. It is a figure for demonstrating the GOP table information and OSD information of the video control information which concern on Embodiment 2 of this invention. It is a figure for demonstrating the sequence information of the video control information based on Embodiment 2 of this invention, OSD information, and video attribute information. It is a block diagram of the stereoscopic video recording apparatus which concerns on Embodiment 3 of this invention. It is a figure for demonstrating the video stream structure which concerns on Embodiment 3 of this invention. It is a figure for demonstrating the video stream structure which concerns on Embodiment 3 of this invention. It is a block diagram of the stereoscopic video recording apparatus which concerns on Embodiment 3 of this invention. It is a parallax image conceptual diagram for demonstrating the principle of the compression of the difference information which concerns on Embodiment 3 of this invention. It is a block diagram of the stereoscopic video recording apparatus which concerns on Embodiment 3 of this invention. It is a schematic diagram for demonstrating compression of the parallax information image conversion which concerns on Embodiment 3 of this invention. It is a figure for demonstrating the video stream structure which concerns on Embodiment 3 of this invention. It is a figure for demonstrating the video stream structure which concerns on Embodiment 3 of this invention. It is a block diagram of the stereoscopic video recording apparatus which concerns on Embodiment 3 of this invention. It is a figure for demonstrating the video stream structure which concerns on Embodiment 3 of this invention. It is a figure for demonstrating the video stream structure which concerns on Embodiment 3 of this invention. It is a block diagram of the three-dimensional video reproduction device concerning Embodiment 4 of the present invention. It is a block diagram of the three-dimensional video reproduction device concerning Embodiment 4 of the present invention. It is a figure for demonstrating the stereoscopic video reproduction apparatus which concerns on Embodiment 4 of this invention. It is a figure for demonstrating the stereoscopic video reproduction apparatus which concerns on Embodiment 4 of this invention. It is a figure for demonstrating the stereoscopic video reproduction apparatus which concerns on Embodiment 4 of this invention.

(Embodiment 1)
Embodiment 1 of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the overall configuration of the stereoscopic video system according to the present embodiment. In the stereoscopic video system shown in FIG. 1, a stereoscopic video recording device 6 (hereinafter also simply referred to as a recording device 6), a TV, a projector, or the like that reproduces media on which stereoscopic video is recorded and outputs right-eye video 1 and left-eye video 2. Display device 3, a liquid crystal shutter or the like that can be switched between two transmitted and deflected light beams, and a video shutter 5 A and 5 B that are arranged in frame order via the shutter 4. It is comprised with the glasses 7A with which the deflection | deviation plate was comprised. FIG. 2 shows another configuration example of the stereoscopic video system according to the first embodiment. The stereoscopic video system shown in FIG. 2 differs from FIG. 1 in that it includes two display devices A and B, and projects on the deflection plates 9 and 10 and the display panel 12 for passing only light of specific deflection components having different directions. And glasses 7B having different right and left deflecting plates for viewing images 5C and 5D having the frame order obtained through the deflecting plates 9 and 10.

  FIG. 3 shows still another configuration example of the stereoscopic video system according to the first embodiment. In the stereoscopic video system shown in FIG. 3, unlike FIG. 1, a synchronous rotating member 13 having a circular rotating disk with a semicircular deflecting plate that passes only light of specific deflection components having different directions, and a synchronous rotating member 13. And a light source 14 for projecting light. FIG. 4 shows still another configuration example of the stereoscopic video system according to the first embodiment. The stereoscopic video system shown in FIG. 4 includes display devices 15 to 19 for projecting a plurality of videos based on the video signal 22 and a rotating mirror 21 for reproducing the projected stereoscopic video, unlike FIG. ing.

  FIG. 5 is a trial of a stereoscopic image perceived by a human. In FIG. 5, the depth positions 117 to 117 are arranged in order from the infinite distance 111, and the depth position 117 is the position where it protrudes most (close to the eyes). Further, FIG. 5 shows a person 118 displayed on the depth position 115, a person 119 displayed on the depth 114, a river 120 flowing from a distant place, and a mountain 121 visible in the distant place. FIG. 6 is a further display of subtitle display in addition to the display of FIG. 5, and illustrates subtitles 122 </ b> A to 122 </ b> C displayed at respective depth positions and depth ranges 123 </ b> A to 123 </ b> C of the subtitles.

  FIGS. 7A and 7B are diagrams showing the angular change acceleration of parallax, the time and number of changes, and the degree of eye fatigue. In FIG. 7A, the eye fatigue level 124 is the vertical axis, the product 125 of the angular change acceleration / change is the horizontal axis, the eye fatigue level 126 for the attention point and caption, the highest fatigue point 127 for the attention point and caption, The background image eye fatigue degree 128, the limit point 129 where intoxication occurs in the background video, the human eye safety area 130, the human eye danger area 131, and the human eye 3D sickness area 131 are respectively illustrated. Show. FIG. 7B is a schematic diagram for showing the parallax movement of the eye. The image 126A far from the point of interest, the image 126B near the point of interest, the viewing angle 126D when near, and far away. In this case, the viewing angle 126E, the human eye 126F, the depth 126GG where the video 126B is displayed, and the depth 126HH where the video 126A is displayed are illustrated.

  FIG. 8 is a graph showing the relationship between the acceleration in the depth direction of the point of interest and the movement time × number of times. In the graph shown in FIG. 8, the vertical axis is the acceleration 133 in the depth direction of the point of interest, the horizontal axis is the product of the movement time and the number of times (movement time × number) 134, the boundary 135 between the safety area 130 and the danger area 131, and the danger area. A boundary 136 between 131 and the 3D sickness generation region 132 is illustrated. FIG. 9 shows the relationship between the depth position and the depth position change amount in subtitle display. In FIG. 9, with the depth position 137 as the vertical axis and the depth change amount 140 as the horizontal axis, the infinity position 138, the eye position (frontmost) 139, the depth change amount limit value 141, the front depth limit (jump limit) ) 142 and a remote depth limit 143 respectively.

  Here, in general, stereoscopic video systems using a TV or a projector, as shown in FIGS. 1 to 3, often use parallax information of human eyes, and each video information is displayed on the left and right with glasses. It is projected and shown in three dimensions. In general, when shooting a video, two cameras are used to shoot a video for input to the left and right eyes using two cameras. Here, the left and right video information stored in the recording device 6 is first input to the display device of the TV or projector. At this time, when the analog interface is used as the information interface between the recording device 6 and the display device, it is necessary to transmit information separately on the left and right, but in the case of a digital interface such as HDMI, the information on the left and right are alternately serially transmitted. In addition, there is a method of compressing and transmitting on the recording device 6 side and decompressing on the TV side. Also. In the case of TV display, left and right video information is displayed by switching between left and right for each field. However, when a TV using a double-scan display in recent years is used, a playback video is divided into left and right for each field. Problems such as flicker can be solved, and smooth stereoscopic video playback is possible.

  Furthermore, as shown in FIG. 1, when the shutter 4 configured by liquid crystal or the like and capable of switching between two transmitted deflected lights is configured, the transmitted field image 5A is, for example, vertically deflected, and the image 5B is laterally deflected. By controlling the shutter 4, the deflection angle of light for each field can be changed. In this case, the glasses 7A may have different left and right deflection plates (vertical deflection and lateral deflection) attached thereto, and a signal corresponding to the timing at which the display device 3 controls the shutter 4 via the cable 3A is sent from the display device 3. The cable 3B supplied to the glasses 7A becomes unnecessary. On the other hand, when the shutter 4 is not used, it is necessary to provide a liquid crystal shutter on the side of the glasses 7A, and the field synchronization signal cable of the cable 3B is necessary. When using the liquid crystal shutter on the side of the glasses 7A, since deflected light is not used, there is an effect that the influence on the stereoscopic display can be kept small even if the angle of the glasses changes such as tilting the neck.

  In the method of FIG. 2, two display devices having a PLD element or a transmissive liquid crystal type are provided, and separate left and right images are displayed on each display device. In this case, deflection plates 9 and 10 having different deflection directions are attached to the front surfaces of the display devices (A and B) 7 and 8, respectively. As a result, the light emitted from the respective display light emitting portions has different deflections, and is projected onto the display panel 12 via the optical system 11, for example, the right eye is a vertically deflected image 5C and the left eye is It is possible to project a laterally deflected image 5D. Here, further, video information with parallax is input to each eye using the deflection glasses 7B.

  Further, in the system of FIG. 3, a deflecting light switching mechanism having a synchronous rotating member 13 that rotates in synchronization with the field display timing of the TV is formed in the portion of the light source incident on an optical element such as a PLD, and the field display timing is set. This is a system in which light having polarized light matched to is incident on a PLD element or the like. In this case, an image having different polarized light for each field is projected on the image display panel. Viewing this with the deflection glasses 7 of the same type as in FIG. 2 makes it possible to see the parallax image. In addition, as shown in FIG. 4, there is also a method for reproducing a stereoscopic image by projecting images taken from a plurality of angles by a plurality of display devices 15 to 19. In this case, it is necessary to store and reproduce a plurality of streams instead of two stereoscopic video streams.

  Further, the synchronous rotating member 13 that rotates in synchronization with the timing of the field display of the TV is configured with an optical filter that passes only specific wavelengths of RGB, and the other half of the disk is for the left eye and the other half is for the right eye. By shifting the RGB wavelengths, the wavelength of the light for the right eye and the left eye is changed, and the left and right images can also be obtained by configuring the glasses 7 with optical filters that pass only the wavelengths for the right eye and the left eye, respectively. It can be incident on each eye. In this case, the left and right RGB shifts can be corrected by color adjustment on the display side of the TV so as not to be inferior in color reproduction. Also, in the method of shifting the RGB wavelength left and right, the glasses 7 can be tilted even if the glasses are tilted. The light from is not attenuated.

  In the method using the rotating mirror 21 in FIG. 4, images from a plurality of viewpoints are projected onto the rotating mirror 21 by the display devices 15 to 19. However, it looks like the real thing (in the extreme case, you can see the hidden parts such as the back side).

  Next, actual stereoscopic video will be described. Even in the case of video reproduction using parallax, it is perceived by human eyes as shown in FIG. In this case, if the depth from the infinity 111 of the depth position to the eye position 117 is decomposed and expressed, the respective images are displayed on the depth plane from the depth positions 112 to 115. For example, a character that is a point of interest looks like a human 118 when it is in front, and looks like a human 119 when separated. The river 121 or the like, which is background information, looks smaller when it is farther away, and the larger mountain 121 or the like appears larger even in the background. For example, when a caption is displayed on a stereoscopic video as shown in FIG. 5, it is expressed as shown in FIG. The subtitle 122A in the vicinity is gradually moved away from the subtitles 122B and 122C. If the depth positions of the humans 119A to 119C, which are the characters of interest, change depending on the scene, if the subtitles are changed accordingly, the movement of the focal point of the eyes is reduced, so that fatigue is less likely to occur. Therefore, it is desirable to display the caption 122A in the scene of the human character 119A, the subtitle 122B in the scene of the human character 119B, and the subtitle 122C in the scene of the human character 119C. Since there is no change in depth position in the conventional 2D video, the distance between the user and the TV is the focus of the human eye and does not move the muscle in the focus direction of the eye, but the stereoscopic video uses parallax. This is because movement of the eye for the amount of parallax is required even if it is a thing.

  Further, as shown in FIG. 7A, in the movement of the attention point of a character or the like in the focal direction, eye fatigue occurs in proportion to the change speed, the time required for the change, and the number of changes. In particular, since the eye must follow the attention point, eye fatigue is intense, and it is considered that the peak of fatigue is reached when the parallax angle change speed, the time required for the change, and the number of changes are small. In particular, the near side image has a large parallax angle like a viewing angle 126D shown in FIG. 7B, and the far side image has a small parallax angle like a viewing angle 126E. When the perspective changes, the parallax angle changes, and it is necessary for the eyes 126F to focus on the target by setting this angle, and the parallax angle must be followed to the change in perspective. In conventional TVs that display flat images, there is no perspective image, so the parallax angle necessary for recognizing the depth direction of the eyes is always constant. The eye burden is increased because the eye movement must be applied. However, if the movement is faster than the response of the eye, it cannot be followed, and the fatigue is also reduced. In addition, although the background information does not originally follow the background information, it is estimated that the parallax angle changing speed at which the distance changes, the time taken for the change, and the number of times of the change increase. Figure 8 shows the relationship between the depth direction acceleration and the product of the travel time and the number of times, and even if the depth direction acceleration is small, if the number of times and the distance increase, the occurrence of dangerous areas and sickness can be seen, If the level falls below a certain level, it is estimated that even if the product of the travel time and the number of times increases, the tired state is not obtained.

  Here, the degree of eye fatigue as an evaluation value is such that when the screen size increases, the movement of the eye in the in-plane direction also increases and fatigue increases. Can be considered. First, the evaluation function 1 is such that the evaluation value (the degree of eye fatigue) is parallax angle change speed × time required for change × a <n> It is proportional to the number of changes. The evaluation function 2 is such that the evaluation value (fatigue degree of the eye) changes the parallax angle change speed × the time required for the change × a <n> It is proportional to the number of times x the screen size. When the size of the TV screen can be detected, the evaluation function 2 is used, and when it cannot, the evaluation function 1 is used. In the second and subsequent embodiments, the evaluation value (eye fatigue degree) is described as a depth change degree.

  When one stereoscopic video is produced, the amount of change in the parallax angle of the stereoscopic video in one video content, the time involved in the change, and the number of changes are set as evaluation coefficients so that they do not enter the danger area 131 of FIG. 3D video content can be produced by re-encoding. In addition, by describing the degree of depth change as an evaluation function in this stereoscopic video content as well, the degree of eye fatigue is presented before watching the movie, and the user is allowed to select 2D playback or 3D playback. I can do things. At this time, as a re-encoding method, a shooting device such as reducing a parallax interval of a camera that shoots a parallax image (reducing the distance between two cameras) or a pixel conversion process using parallax information described later. A method of performing image processing that reduces parallax, a method of limiting the pop-out amount at the time of content production with CG, etc. are performed for animation and the like.

  In order to grasp such a story of a movie or the like, for the subtitle display that must be read by the user, it is necessary to limit the amount of change in the depth direction as shown in FIG. This is because if the follow-up speed in the focus direction of the eye becomes too high, eye fatigue increases as shown in FIGS. 7 and 8, and 3D sickness is likely to occur. In addition, since the subtitles at positions that are too far away have a sense of incongruity as shown in FIG. 6 due to the size of the subtitles and the background, it may be better to limit the distant positions. In addition, there is a need to limit the near side near the eyes. This is especially true when the eye is too close to the eyes, because the eye angle changes greatly because of the viewing angle, and the eye fatigue is increased. Because there is also. In addition, when the TV screen to be displayed is enlarged, the amount of eye movement in the in-plane direction is increased, and the psychological effects such as “surprise” and “surprise” are increased. desirable. When the playback device and the TV are linked, information regarding the size of the TV screen is exchanged with the playback device, and the pop-up range limitation of subtitles and the like is severely limited. In addition, when a plurality of streams with different pop-out amounts are arranged, a configuration in which a stream with a small pop-out amount is selected when the TV screen is large, and a stream with a large pop-out amount is selected when the TV screen is small is considered. It is done. In addition, when the pop-out amount can be changed by setting on the apparatus side described later, it is conceivable that the setting is automatically performed in consideration of the TV size information and the user status (age etc.).

  At this time, a stereoscopic video parental level consisting of the above evaluation value and the maximum projection amount that is the viewing angle is specified, and according to the stereoscopic video parental level, the age limit for viewing and the danger notification for the elderly and the sick are to be given. Is possible. For example, as a stereoscopic video parental level, level 1 is assumed to be fatigue / dangerous, evaluation value (eye fatigue)> c, maximum pop-up amount> d, and normal parental level is high. Level 2 indicates that fatigue / risk is slightly high, and evaluation value (eye fatigue level)> c, maximum pop-up amount> d, normal parental level is below normal, or evaluation value (eye fatigue level)> e , Maximum pop-out amount> f, normal parental level is high. Level 3 assumes that fatigue / risk is medium, evaluation value (eye fatigue degree)> e, maximum pop-out amount> f, and normal parental level is below normal. Level 4 is assumed that there is no fatigue / danger, evaluation value (eye fatigue)> g, maximum pop-out amount> h, and normal parental level is below normal.

  In the example of the stereoscopic video parental level described above, there is a relationship of c> e> g, d> f> h, and the normal parental level (planar image parental level) is the current level for horror movies and the like. This refers to the viewing restriction for safety defined in the 2D video DVD and the like. In addition, such 3D image parental level settings can be set / changed at the time of product purchase or at the initial setting.For example, by storing the PIN number, it is possible to change the release later. More useful.

(Embodiment 2)
Next, the second embodiment will be described below with reference to the drawings. For stereoscopic images using parallax information as shown in FIG. 1 to FIG. 3, if a TV broadcast is made as it is, a screen like a double copy is displayed, and viewing is not performed unless a dedicated stereoscopic display device as described above is configured. I can't. Therefore, since performing stereoscopic video in broadcasting depends on the equipment infrastructure on the viewer side, a dedicated channel that cannot be generally viewed is provided, or a flag indicating that it is 3D video is superimposed on broadcasting information, etc. Is required. For this reason, it is normally distributed on a recording medium or the like, and it is convenient to view it on a dedicated player or a player equipped with this function. In this situation, a method and format for storing the above-described stereoscopic video on a recording medium will be described below.

  FIG. 10 shows a recording medium 26 according to the second embodiment. It goes without saying that the recording medium (video medium) according to the present invention may be an HDD medium in addition to an optical disk medium such as a DVD, BD, HD-DVD, or MO. In the case of an HDD, it is often impossible to carry it by itself, but it is advantageous in terms of capacity when recording broadcast 3D video information. On the other hand, in an optical disk medium such as a ROM medium, it is effective for distributing stereoscopic killer content before broadcasting or stereoscopic charged content. In the disc-shaped recording medium 26 shown in FIG. 10, an area for storing control information related to video information (video control information 23), an area for storing stereoscopic video (video title 24), and a normal 2D video are stored. It is divided into areas (video title 25).

  FIG. 11 shows an example of the structure of the video stream of the video title (video content) 24 part of FIG. In FIG. 11, the video title 27 is 2D video information 28, 2D video information 30 and 31 that can be selected by the user, and 3D video information that is automatically selected or selected by the user when the display device can display stereoscopic video. 29, 2D video information 32 reproduced following the video information 29-31, and last 2D video information 33 of the video title 27. In FIG. 11, as the GOP layer stream information, the supplementary information area 34 in which supplementary information related to the GOP video information is described at the top of the following GOP video information 35 and 36 is described, and the top of the GOP video information 38. A supplementary information area 37 in which supplementary information related to GOP video information is described, and a supplementary information area 39 in which supplementary information related to GOP video information is described at the top of the GOP video information 40 and 41 are illustrated. Yes.

  In FIG. 11, as stream information of the picture layer, the packet data part 42 describing the accompanying information, the I picture data 43 composed of the in-plane encoded data, the I picture data 43 and the P picture 45 in the time direction. A B picture 44, which is predicted encoded data, and a P picture 45, which is encoded data predicted in only one time direction from the I picture data 43, are illustrated. In FIG. 11, as the transport packet data layer, the same packet 46 as the packet data part 42 in the transport packet data part describing the incidental information, the first packet 47 obtained by dividing the I picture data 43 by the transport packet, A transport packet 48 in which the last data of the I picture data 43 is stored, a last portion 49 of the I picture data in the transport packet 48, and a padded portion 50 in the transport packet 48 are illustrated. Yes.

  FIG. 12 shows a data structure of a hierarchy of areas in which 3D video information 29 and 2D video information 30 and 31 in the video title (video content) 27 can be selectively selected. In FIG. 12, supplementary information 51 storing information related to the main video sequence arranged at the head in a region where 3D video information 29 and 2D video information 30 and 31 can be selectively selected, and a GOP video information sequence in the region. 52, additional information 53 in which information related to the GOP video information sequence 54 is stored, and additional information 55 in which information related to the last GOP video information sequence 56 in the area is stored are illustrated. Also, in FIG. 12, as picture layer stream information, I picture data 57 composed of intra-frame encoded data, B picture which is encoded data predicted in the time direction from I picture data 57 and P picture 59 58, a P picture 59 which is encoded data predicted from the I picture data 57 in the time direction of only one direction is shown.

  Also, in FIG. 12, as the transport packet data layer, the first packet 60 obtained by dividing the I picture data 57 by the transport packet, the transport packet 61 in which the last data of the I picture data 57 is stored, the transport packet The last part 62 of I picture data in 61 and the padded part 63 in the transport packet 61 are shown. In addition, an arrow A shown in FIG. 12 is a playback portion, an arrow B is a portion that jumps when 3D playback is performed, and an arrow C is a portion that is played back to perform 3D playback. GOP video information of the right-eye video and left-eye video The GOP video information is arranged. FIG. 13 shows the data structure of the area hierarchy in which the 3D video information 29 and 2D video information 31 in the video title 27 can be selectively selected, and is basically the same as FIG. Are denoted by the same reference numerals and description thereof is omitted.

  As shown in FIG. 10, the structure of data recorded on the optical disk or HDD medium includes an area of video control information 23 in which video-related supplementary information and sequences are recorded, and actual video titles (video contents) 24 and 25. It is composed of At this time, the entire 3D video is not necessarily a 3D video, and a case where the 3D video is mixed with a 2D video or a case where these videos are switched by a user's selection is assumed. In particular, in the DVD standard, it is possible to switch and display a user-selectable video information sequence such as multi-angle, and even in the case of 3D video information, it is considered that not all user devices are compatible with 3D video. Then, a 3D video stream is additionally constructed on the 2D video stream. When the user's display device is 3D-compatible, the 3D video stream is automatically identified by the link function of the HDMI terminal and the 3D video stream is selectively displayed, or the 3D video side is selectively determined by the user's button operation. A method of making it possible is conceivable. Of course, it is needless to say that all content may be in 2D video only or 3D video only, but the format requires consideration for such a composite format.

  Also, in the video information stream of the video title 24, it is desirable to provide an associated information area on the video information stream so as to perform access and management of information, switching of device settings, and the like. In particular, in a content in which 2D video and 3D video are mixed, it is necessary to determine whether the video stream is 2D video or 3D video on the TV side. It is possible to easily and automatically switch the setting on the side. When all settings are closed by a player / recorder that reproduces or records a recording medium, the control information may be described only in the video control information 23 arranged in a part of the disc. However, in the case of connection coordination with a TV, especially when a procedure such as switching between TVs is performed during playback, automatic control on the TV side is performed by superimposing the minimum necessary control information on the video information itself. Change setting can be performed. When there is no control information in the video information, the TV connection switching is detected, the control information is separately transmitted from the player / recorder, the TV side setting is changed, and then the video information is transmitted. These TV-side setting changes are performed on the display device side, such as switching the polarized light, so that it is necessary to have a mechanism for quickly performing the setting change processing of the display device. Nor.

  The incidental information 51 can also be used for information access management, and is fixed as Navi information in the DVD standard. Here, when 2D video and 3D video are mixed, the time series of the contents are arranged in parallel, such as 3D video information 29 and 2D video information 30 and 31 shown in FIG. Therefore, the supplementary information 34 at the head needs to be arranged at the head of the GOP data information group. First, by reading the content of the supplementary information, whether the information of the next GOP sequence is 2D video or 3D video, In the case of a 3D video, it is possible to determine whether the video is a left-eye video or a right-eye video, and arrangement information (where to access) in the GOP video information group. Here, the GOP video information group including the accompanying information 51 at the head is defined as a video unit that is larger than the GOP video information as a video unit.

  In addition, in the case of video information data that is also compressed in the time direction, such as MPEG, information exists in units of GOP video information starting from an I picture, so that video data access is in units of this GOP video information. Needless to say. Also, since the incidental information needs to be read first, it must be placed at the beginning of the GOP video information group. For example, when reproducing the 3D video information portion as shown in FIG. After the middle arrow A), the 2D video information 30, 31 is jumped to reproduce the 3D video information 29. At this time, the 2D video information 30, 31 jumps as indicated by an arrow B in the figure, and unnecessary memory is increased by preventing unnecessary information (in this case, 2D video information 30, 31) from being taken into the memory of the playback device. The 3D video information 29 indicated by the arrow C in the figure is reproduced so as to avoid this and prevent the video from being interrupted.

  Further, the supplementary information 51 at the head of the GOP video information is arranged at the head position of the I picture 57 in the state of the picture layer below it. Further, since it is convenient to divide these compressed video data into transport packets in order to have compatibility with digital broadcasts such as terrestrial waves and satellites / cables, the lowermost layer data is shown in FIG. The transport packets 60 and 61 are divided as follows. Even in this case, the incidental information 51 is described in the first transport packet of the GOP video information group 52. Needless to say, a newly defined private packet in the transport packet is used. Further, since the last transport packet 61 of the GOP video information group is not necessarily cut off in a certain transport packet unit, the last portion 63 is padded with “00” or “FF” to obtain the GOP video. It is preferable to complete packet data in units of information. Also, as shown in FIG. 13, when one 2D video 31 and one 3D video 29 are branched, the amount of GOP video information to be jumped by arrow B in the figure is smaller than that in FIG. The basic operation is the same as in FIG.

  The contents of the accompanying information will be further described. The supplementary information 51 shown in FIG. 14 includes content information 64, time code 65, arrangement information 66, information 67 about video information, information 68 about audio information, and information 69 about OSD information. 14 includes a content name 70, a copyright 71, encryption information 72, presence / absence 73 of 3D video, and valid area information 74.

  Further, the time code information area 65 shown in FIG. 15 includes a presentation time 65A and synchronization information 65B. The arrangement information 66 shown in FIG. 16 includes seamless information 75, jump destination information 76, angle information 77, and intra-GOP arrangement information 78. The video information 67 shown in FIG. 17 is related to resolution information 79, frame rate information 80, 3D video information 81, parental information 82, angle information 83, encryption information 84, information about 3D video format and presence 85, and 3D video frame rate. The information 86 includes 3D video information count 87, depth resolution information 88, depth change information 89, subtitle permission depth information 90, depth restriction information 100, and parallax amount restriction information 101.

  FIG. 18 is a schematic diagram when the supplementary information is displayed on a TV as a display device. FIG. 19A and FIG. 19B are schematic diagrams when multi-angle shooting is performed from a plurality of cameras. Also, the OSD information 69 shown in FIG. 20 includes OSD arrangement information 69A, OSD storage destination information 69B, font and font size designation 69C, in-plane OSD arrangement information 69D, depth direction OSD arrangement information 69E, and depth position. 69F, depth permission limit 69G, and depth zooming speed 69H.

  Here, the supplementary information 51 in FIG. 14 is described for each GOP video information group on the stream, and is transmitted together with video information even when HDMI transmission is performed to a TV or the like. Therefore, it goes without saying that necessary information is also included in the setting on the TV side particularly regarding 3D video display.

  Next, the content information 64 shown in FIG. 14 will be described. The content name 70 displays (1) content name, (2) performer name, (3) production time, (4) distribution company, (5) related work name, and (6) summary as OSD information on the TV side. There is a case to let you. If the content name 70 includes the accompanying information 51 superimposed on the video stream, the content name 70 can be displayed even when the TV-side input is switched to 3D video information. It becomes.

  The copyright information 71 shown in FIG. 14 includes (7) copyright owner, (8) distributor, (9) importer, and (10) capital participating company, so that the copyright owner of the video stream can be obtained. This information can be distributed at the same time, and even if the reproduction data is used illegally, the copyright holder's right can be claimed. In addition, since this information is superimposed on the video stream, information is always distributed to the TV even when the TV is switched, so that it is possible to display a copyright.

  Also, the encryption information 72 shown in FIG. 14 is (11) presence / absence of encryption and (12) encryption method, so that it is encrypted highly confidential information or no confidentiality such as commercials. Information can be sent to the destination device.

  Also, the 3D video information 73 shown in FIG. 14 includes (13) 3D video compatibility, (14) whether all 2D video is supported (whether playback is possible only by displaying 2D video), and (15) 3D video. In the case of correspondence, it is possible to display to the user that it is not compatible when connected to a TV that does not support 3D video by describing whether or not 3D video playback is prioritized. In addition, when the TV and HDMI are linked, the TV side is automatically switched to 3D video setting (for example, as shown in FIGS. 1 to 3, two video streams are automatically displayed for each field. ) When there is no 3D video function on the TV side, the TV or playback device can display that the TV is not supported, or can take measures such as ejecting the disc.

  14 describes (16) 2D video playback permission area and (17) 3D video playback permission area, thereby not only limiting the playback permission area of this disc, but also 2D video playback. It is also possible to specify a limited area where only video is allowed and 3D video compatible display is allowed. This is because there are cases where only 2D video playback is permitted in a specific region when the license conditions for 3D video playback are not in place. If the 3D video reproduction permission area is not permitted, even if the 3D video display device is connected, only the 2D video is played, or the disc is ejected.

  Next, the time code information 65 shown in FIG. 15 will be described. In the case of video content in which 2D video and 3D video are mixed, even if the switching is performed by the user (for example, 3D video to 2D video), it is continuously played back without interrupting or omitting the flow of video information. I have to let it. In addition, there may be a time search such as going back / forward 10 minutes before the user's instruction. Therefore, it is necessary to record the following presentation time 65A, which is playback time information from the start time of the title of the video, at the beginning of the GOP video information group. It is possible to display the remaining time on the TV side by describing the remaining time information until the end of the title playback or the total title playback time.

  Also, since 3D video is likely to induce eye fatigue as described in Embodiment 1, the time code from the start of 3D video playback (how many 3D videos have been viewed continuously) and the content of this video content You can display how many 3D images you watched in total, and can give a break instruction and danger display to prevent eye strain. Further, when there is GOP video information for each of the right eye and left eye in the 3D video, it is possible to specify a field corresponding to the playback order. That is, the presentation time 65A includes (18) time code (presentation time) from the start of the title, (19) remaining time information until the end of the title playback or the total playback time of the title, and (20) the time from the start of 3D video playback. The code (3D presentation time), (23) total 3D playback time, and (24) left and right video playback order or field designation are described. The synchronization information 65B specifies synchronization of video content and specifies the playback order or fields of left and right videos.

  Next, the arrangement information 66 shown in FIG. 16 will be described. In particular, when 2D video content and 3D video content coexist, it is necessary to skip information unnecessary for reproduction or to describe arrangement information in the GOP video information group at the head position of necessary data. In addition, when performing a special reproduction operation, it is necessary to consider that access from an in-plane compressed image is required first from the characteristics of compressed video in the time axis direction such as MPEG. Therefore, the seamless information 75 records (25) presence / absence of seamless reproduction (up to the next GOP video information group). The jump destination information 76 includes (26) jump destination (forward and reverse directions) address 1, address 2, etc., (27) jump destination time code information 1, time code information 2, etc. (multiple jump destination information) (28) The presence / absence of the jump destination 3D video information is recorded. Note that (28) if there is no 3D video information at the jump destination based on the information on the presence / absence of the 3D video information at the jump destination, the TV setting can be returned to 2D video during stream playback.

  The angle information 77 includes (29) address 1 and address 2 of GOP video information corresponding to a plurality of angles, and (30) time code information 1 and time code information 2 of GOP video information corresponding to a plurality of angles. To be recorded. In the intra-GOP arrangement information 78, (31) address information 1, address information 2, and the like as the arrangement information of P pictures in each GOP are recorded. As described above, by having the seamless information 75, it is possible to sequentially reproduce while connecting necessary angles, and reproduce only the I picture or only the I and P pictures according to the arrangement information in the GOP. By doing so, fast forward and fast reverse playback becomes possible.

  Next, the video information 67 shown in FIG. 17 will be described. In the video information 67, what is particularly required as stereoscopic video information is shown below. First, the resolution information 79 stores (32) 2D video playback resolution (in-plane direction), PinP image resolution (in-plane direction), and (33) 3D playback resolution (in-plane direction). . In the 3D video presence / absence method 85, (34) presence / absence of 3D and (35) 3D video mode designation (double scan rate designation, deflection glasses not present, liquid crystal shutter not present) are recorded. In the 3D frame rate 86, (36) a frame rate at the time of 2D video playback, and a frame rate at the time of 3D video playback are recorded.

  The number of 3D video information 87 records (37) the number of independent 3D video information streams to be played back in parallel. When there are n different angles, n = angle number is described. Based on this information, the number of angles can be displayed during reproduction, and the angle can be changed according to the selection from the user and recognized by displaying the angle number. The number of 3D video information 87 records (38) the number of 3D video streams and camera information when the left and right videos are sequentially switched. For example, as shown in FIG. 19A, when a video is captured using cameras D to H shifted by five parallaxes, or an animation image or the like is recorded as five parallax video information by CG. The number of these and the interval or angle of each camera are described. As an example of the description of the incidental information, general information-number of images 5-camera interval ** mm, image 1 from camera D 1-angle 1, image 2 from camera E 2, angle 2 from camera F 3-angle 3, camera G A video 4 -angle 4 and a camera H video 5 -angle 5.

  When there are five images shifted by the amount of parallax, the actual angle image is as follows: angle D is image 1 left, image 2 is right, angle E is image 2 left, image 3 is right, angle F is image 3 is left, video 4 is right, angle G is video 4 is left, video 5 is right and 5 pieces of parallax video information. As shown in FIG. You can play it. At this time, it is also possible to rotate the video by sequentially shifting the angle information. Accordingly, each video stream does not indicate one angle information, but new angle information can be constructed by a combination with a video having an adjacent parallax angle. In recent years, with the evolution of CG technology, it has become possible to easily create stereoscopic images with animation images. By preparing such multi-angle parallax information and accessing each by specifying the remote control from the user The viewpoint can be changed by shifting the angle.

  In the depth resolution 88, (39) depth resolution 1, resolution 2 and the like in 3D video are recorded. When there are a plurality of 3D video streams, a plurality of depth direction resolutions are described. For example, when the depth resolution is extremely low due to a CG image or the like and the image moves rapidly in time, the depth can be complemented in the time direction and displayed smoothly based on this information. Depth change degree 89 records (40) depth change degree 1, change degree 2 and the like in 3D video. When there are a plurality of 3D video streams, a plurality of depth direction change degrees are described. In particular, since the degree of change is related to human eye fatigue as shown in the first embodiment, it can be described in order to ensure safety, and can be used for a warning or a break instruction to the user. .

  In the permitted caption depth 90, (41) a permitted caption depth range (maximum viewing angle 1, minimum viewing angle 1, maximum viewing angle 2, minimum viewing angle 2, etc.) is recorded. When there are a plurality of 3D video streams, a plurality of depth direction change degrees are described. With regard to subtitle information, it is necessary to frequently adjust the focus at the subtitle focus position and the point of interest when viewing stereoscopic video, which will be described later. Is required. Depth information should be described as viewing angle information because it is difficult to quantify because the far distance becomes infinite when the actual distance is used. Moreover, since it is meaningless to describe detailed numerical values when it is close to infinity, a lower limit such as omitting the viewing angle of 1 deg or less may be provided. Based on this information, the player sets the depth position of the caption in the OSD display.

  In the depth restriction 100, (42) depth restriction (maximum viewing angle 1, maximum viewing angle 2, etc.) is recorded. A stereoscopic image popping out too close gives a sense of being surprised by a psychological effect. For this reason, the amount of projection of the 3D image itself, not the subtitles, is limited, and effects that are easy on the eyes and considerations that do not surprise the viewers. In this case, as shown in FIG. 17, the player records a viewing angle that is the maximum amount of pop-up in the video content in advance, thereby warning or restricting viewing to a small child or other viewer as shown in FIG. Is possible.

  The parallax amount limitation 101 describes (43) parallax amount limitation (maximum viewing angle 1, minimum viewing angle 1, maximum viewing angle 2, minimum viewing angle 2, etc. during shooting). When there are a plurality of 3D video streams, a plurality of depth direction change degrees are described. This information defines the range of the deviation amount of the reference angle because the parallax reference amount, which is the distance between the two cameras at the time of shooting, differs depending on the distance between human eyes. Thereby, it is possible to grasp in advance a sense of discomfort when a child or the like having a small interval between both eyes views.

  In this way, in order to eliminate the uncomfortable feeling, a method of preparing a plurality of stereoscopic images with different parallax reference amounts in the same video content and selecting them according to the distance between the eyes of the viewer can be considered. In recent years, CG technology has been revolutionized for a plurality of parallax reference amounts, and animation images and the like can be easily changed by a computer. In this case, by describing such parallax amount restriction information in the incidental information, the player prepares selection keys for ** to **, adults, etc. as shown in FIG. Accurate stereoscopic viewing can be achieved by matching the parallax reference amount in the original video content with the viewer. Further, it is possible to avoid eye fatigue caused by viewing a video with a shifted parallax for a long time. Furthermore, in the parental 82, a stereoscopic video parental level corresponding to 3D video is defined in addition to the normal planar video 2D parental level. The parental 82 includes (44A) a planar video parental level (a parental notation equivalent to the current DVD), (44B) a stereoscopic video parental level (the stereoscopic video parental level described in the first embodiment), and Record.

  Also, as shown in FIG. 20, in the OSD information 69, first, arrangement information 69A that is supplementary information of the OSD itself and an OSD information storage destination 69B that describes an address where the information of the OSD itself is stored are recorded. In the OSD display 69, first, the supplementary information is captured and understood by a microcomputer or the like, and then the actual OSD is acquired and displayed by the storage destination information.

  Here, (45) font font and font size are recorded in the information 69C such as font size. In the in-plane arrangement information 69D, (46) font arrangement information (X position, Y position) is recorded.

  The depth direction OSD arrangement information 69E includes (47) depth position 69F, (48) depth permission restriction 69G (to restrict eye fatigue in the first embodiment such as a far restriction position, a near restriction position, a depth variation restriction, and the like. (49) Depth zooming speed 69H (no zooming, zooming speed) is recorded. In addition, by regulating the zooming at a depth zooming speed of 69H, when switching from one subtitle to the next, the depth position is not changed instantaneously, but it is changed so that it zooms little by little, and tired eyes It is for reducing.

  Note that the incidental information in the 3D video information (1) to (43) is distributed together with the video information as being superimposed on the video information stream, but in a different area from the video information itself described below. The same description can be made for the described video control information 23. In addition, since all information can be read at the start of the player, various initial settings can be made, and more information can be described than superimposed on video information without being limited by bit rate or memory. Therefore, it is possible to describe more detailed control information.

  Next, the structure of the control information arranged separately from the video information in a certain area of the recording medium will be described. FIG. 21 is a diagram for explaining in detail the GOP table portion of the video control information 23 arranged together and the video-related information therein. The video control information 23 shown in FIG. 21 includes content information 64, copyright 71, encryption information 72, 3D video presence / absence 73, valid area information 74, GOP table information 102, sequence information 103, menu information 104, and menu OSD data. 105. The GOP table information 102 is in a table format as shown in FIG. 21 and includes GOP number, logical address, time code, sequence, arrangement, video, audio, and OSD fields.

  FIG. 21 particularly shows the configuration of the video column, which includes resolution information 79, frame rate information 80, 3D video information 81, parental information 82, angle information 83, and encryption information 84. Furthermore, in FIG. 21, 3D video information 81 includes 3D video format information and presence information 85, 3D video frame rate information 86, 3D video information number 87, depth resolution information 88, depth change information 89, subtitle permission information It is illustrated that the information includes information 90 related to depth, information 100 related to depth restriction, and information 101 related to parallax amount restriction.

  FIG. 22 is also a diagram for explaining in detail the GOP table portion of the video control information 23 arranged together and the video related information therein. FIG. 22 particularly shows the configuration of the OSD column, which is configured by the presence / absence 106 of subtitles and OSD information 69. The OSD information 69 includes OSD arrangement information 69A and OSD storage location information 69B. The OSD arrangement information 69A includes font and font size designation 69C, in-plane OSD arrangement information 69D, and depth-direction OSD arrangement information 69E. The OSD arrangement information 69E in the depth direction includes a depth position 69F, a depth permission limit 69G, and a depth zooming speed 69H.

  FIG. 23 illustrates the structure of the sequence information of the video control information arranged together with the video information in a certain area of the recording medium. The sequence information 103 is recorded in a table format. Show. In the video column of FIG. 23, resolution information 79, frame rate information 80, angle information 83, 3D video information 81, and parental information 82 are recorded. The 3D video information 81 includes information 85 regarding 3D video format and presence / absence, information 86 regarding 3D video frame rate, number 87 of 3D video information, information 100 regarding depth limitation, and information 101 regarding parallax amount limitation. On the other hand, subtitle presence / absence 106, subtitle font / color 107, subtitle display method 108, subtitle display depth limit 109, and subtitle data address 110 are recorded in the OSD column of FIG.

  Regarding the control information arranged separately from the video information in a certain area of the recording medium, all information including the information of the supplementary information 34 and 51 superimposed on the video information stream is described. This is because the control information is first read and various initial settings are made when the player recorder is started up.

  First, the video control information 23 is described as shown in FIG. 21, and the content information 64, the time code 65, the arrangement information 66, the information 67 about the video information, and the supplementary information 51 superimposed in the video information of FIG. It includes information 68 related to audio information and information 69 related to OSD information. However, in the video control information 23 that can store more information, table information relating to all GOPs such as the GOP table 102 can be described, and it is possible to grasp the information content in units of GOP video information without video playback. Is possible. Here, since the GOP table 102 is described as the table in FIG. 21 and has a logical address, the data / file identification information is detected from a signal read from a predetermined sector area, and the detected data is detected. Based on the file identification information, the position where the data file corresponding to the coding unit at the position indicated by the position identification signal is recorded on the disk medium is identified. The data file is read based on the identified position on the disk medium, and the image signal is reproduced by decoding the signal encoded in the encoding unit included in the read data file. Can do. As a result, it is possible to easily identify and reproduce the position where the coding unit at the desired time is recorded.

  Further, in the GOP table 102, the auxiliary information of the same items as the items described in the video streams (32) to (43) including the 3D video information in the auxiliary information regarding the video is described for each GOP video information. Is possible. Also, with regard to subtitle information, as shown in FIG. 22, subtitle presence / absence 106, subtitle font / color 107, subtitle display method 108, subtitle display depth limit 109, and subtitle data address 110 are described (44) to (49). Subtitle supplementary information can be written in units of GOP video information with the same information as that shown in FIG.

  The video control information 23 can also include sequence information in units of GOP video information as shown in FIG. Thus, at the start of playback, a sector address of a predetermined sector area in which information indicating the playback order of the data file is recorded is generated, and the playback order information is read from the data read by the data reading means. Information is detected, and then a sector address is generated based on the reproduction order information. Thereby, it is possible to reproduce an image signal recorded in sectors distributed on the recording medium. The incidental information shown in the above (32) to (49) can also be described in such sequence table information.

(Embodiment 3)
Next, the third embodiment will be described below. FIG. 24 is a block diagram of the stereoscopic video recording apparatus according to the third embodiment. The stereoscopic video recording apparatus shown in FIG. 24 has an AD converter 146 for digitizing video signals of right-eye video and left-eye video of stereoscopic video using parallax information, and movements necessary for image compression in the time direction. A vector detection (motion detection) 147, a DCT conversion circuit 148 necessary for in-plane compression, an adaptive quantization circuit 149 necessary for in-plane compression, and an inverse quantization circuit 150 in the local decoder are provided. Further, the stereoscopic video recording apparatus shown in FIG. 24 stores a variable length coding circuit 151 necessary for in-plane compression, a DCT inverse conversion circuit 152 in the local decoder, a frame memory 153 in the local decoder, and the compressed data. A buffer memory 154, an OSD information encoder 155, an audio encoder 156, a format encoder 157, a modulation means 158 for generating a signal to be written to the optical disk 165, and an LD modulation circuit 159. Further, the stereoscopic video recording apparatus shown in FIG. 24 includes an address header recognition circuit 160 for extracting an address to be recorded on the optical disk 165, a reproduction amplifier 161 for reproducing a signal from the optical head 164, an optical head 164, A servo circuit 162 for controlling the feed motor 163 and the rotary motor 166 and a system controller 167 for controlling and managing the sequence of the entire apparatus are provided.

  FIG. 25 is a diagram showing a stream structure of a stereoscopic video signal generated based on the stereoscopic video recording apparatus of FIG. The stream structure of the stereoscopic video signal shown in FIG. 25 is a structure in which the structure of the right-eye video GOP 168, the left-eye video GOP 169, the audio information information 68, and the OSD information 69 is sequentially repeated after the auxiliary information 51. The right-eye video GOP 168 shown in FIG. 25 includes a GOP header 170, picture headers 171, 173, I picture data 172, and B picture data 174. Further, the GOP header 170 shown in FIG. 25 includes a user data start code 175 in the GOP header 170 and 3D video information 176 in the GOP header 170. Furthermore, the 3D video information 176 shown in FIG. 25 includes information 177 indicating whether the video is left-eye video or right-eye video 85, information 85 regarding 3D video format and presence / absence, information 86 regarding 3D video frame rate, 87 regarding the number of 3D video information, depth limitation It consists of information 100 and information 101 related to the parallax amount restriction.

  FIG. 26 illustrates a lower structure portion of the stream structure of the stereoscopic video signal generated based on the stereoscopic video recording apparatus of FIG. In FIG. 26, the transport stream packets 178 to 182 of the GOP 168 for the right eye video are shown, and the transport stream packets 185 to 188 of the GOP 169 for the left eye video are shown. The last data of the transport packets 182 and 188 in which the last data of the GOP video information is described are the data 183 and 189, respectively, and padded portions 184 and 190 are added to the data 183 and 189, respectively. .

  Here, the stereoscopic video recording apparatus shown in FIG. 24 performs the same video compression for each of the left eye and the right eye. The right eye video digitized by the AD converter 146 is converted into each video by the process of the motion detection 147. A motion vector for each macroblock is extracted. In addition, since the video data is first subjected to in-plane encoding processing, after DCT conversion by the DCT conversion circuit 148, it is quantized by the adaptive quantization circuit 149, and variable length encoding is performed by the variable length encoding circuit 151. Sent to the buffer memory. At this time, the video data after adaptive quantization is restored to the original video signal by the local decoder by the inverse quantization circuit 150 and the DCT inverse transformation circuit 152, and further compared with the video compensated for motion in the frame memory 153, It is possible to perform compression using only the difference information on the subsequent screen for compressing in the time axis direction. Such compression methods are MPEG and H.264. It is a basic method in the compression method such as H.264 and is widely used.

  Here, in FIG. 24, the right-eye video and the left-eye video are input as independent video streams, and are encoded by separate encoding blocks. Therefore, in FIG. 24, the right eye and the left eye have a structure in which the same blocks are arranged in parallel. However, it is also possible to arrange a memory in the input part and temporarily store the left-eye video and the right-eye video, then process the same encoding circuit at double speed and perform the same processing with one encoding block. The encoded stereoscopic video information is additionally recorded with OSD information from the OSD encoder 155, audio information from the audio encoder 156, and additional information necessary for formatting from the format encoder 157 in the buffer memory 154, and an optical disk 165 as a recording medium. It is arranged in the data format to record. Here, the format encoder 157 additionally records incidental information necessary for recording the 3D video information according to the present invention in addition to the Navi information and menu information necessary for the conventional optical disc format.

  The video data in the format recorded on the optical disc is added with an error correction code as information to be physically written on the optical disc 165 by the modulation means 158 and modulated, and mounted on the optical head 164 by the LD modulation circuit 159. A signal for modulating the modulated laser is generated. At this time, the servo circuit 162 for stably recording on the optical disk 165 controls the feed motor 163 that moves the optical head 164, the rotation motor 166 that rotates the disk 165, and the objective lens actuator in the optical head 164 to adjust the track. And focusing. Further, it is necessary to read an address on the optical disk 165 at the time of recording. A minute signal obtained by photoelectrically converting a signal received by an optical head is reproduced by a reproduction amplifier 161, and address information is generated by an address header recognition circuit 160. These address information is processed in sequence by the system controller 167 together with the activation setting of each block. Particularly, write timing processing that requires high-speed timing is performed by dedicated hardware, and the sequence setting portion that requires programming is a CPU or the like. Done in

  Here, the video stream generated by the stereoscopic video recording apparatus has the structure shown in FIG. First, the compressed video data including the compressed video in the time axis direction such as MPEG includes an in-plane compressed code video generally called GOP. For example, it is configured as a video block of about 15 pictures. Since there are two right-eye video blocks and left-eye video blocks using disparity information here, the GOP 168 of the right-eye video and the GOP 169 of the left-eye video are sequentially arranged with the accompanying information 51 as the head. Note that the right-eye video and the left-eye video each show an example of 1 GOP, but may be composed of a plurality of GOPs as long as the number of GOPs is the same as long as the video conditions do not change. Further, the incidental information 51 is as described in the second embodiment, but here, it is also possible to define a new user data start code in the GOP header portion and describe the 3D video information 176 after that. is there.

  First, in the 3D video information 176, information (which may be a flag) 177 for identifying whether the video is a left-eye video or a right-eye video is arranged, the presence / absence 85 of the 3D video system described in Embodiment 2, and a 3D video frame The rate 86, the number of 3D video information 87, the depth information 100, and the parallax amount restriction information 101 are recorded. As shown in FIG. 26, the incidental information 51 is provided with a private packet (TSP1) 178 in the transport packet layer, which is separated and extracted by the transport decoder. With the configuration shown in FIG. 25, the same 3D information can be extracted even at the hierarchical level of MPEG data. When the playback device and TV are linked and automatically set 3D video, or when switching TV during video playback, if the additional information extraction setting in the transport decoder is valid on the TV side, this The information 51 becomes valid, and if the 3D video incidental information in the MPEG layer is valid, the 3D video information 176 becomes valid.

  The video stream shown in FIG. 25 is described in units of MPEG information layers, but FIG. 26 shows that described in units of transport packets one level lower. In FIG. 26, the video data in units of blocks of the GOPs 168 and 169 of the right-eye and left-eye videos starts from the beginning of the transport packet 178 and does not necessarily fit in data that is an integral multiple of the transport packet at the end of the GOP video information. The remaining data in the final transport packet 182 is padded, and the final data 183 is filled with information with padding data 184. Accordingly, it is possible to extract only the GOP 168 of the right-eye video by taking out the transport packet 179 to the transport packet 182. In the GOP 169 of the left eye video, the same processing as that of the right eye is performed, and only the left eye video can be extracted by extracting the transport packet 185 including the padding data 190 from the top of the transport packet 185.

  In the stereoscopic video recording apparatus in FIG. 24, the left-eye video and the right-eye video are encoded as they are. However, the left-eye video and the right-eye video are basically video information that is shifted by the amount of parallax and has a very high correlation. For example, if the left-eye video is configured to record only the difference information from the right-eye video, the entire information amount can be compressed. A block diagram of the stereoscopic video recording apparatus for this purpose is shown in FIG. The stereoscopic video recording apparatus shown in FIG. 27 employs the same configuration as the video compression block configuration shown in FIG. 24 with the block portion of the right-eye video as the main video. However, in the left-eye video, the difference between the output of the motion detection 147 of the left-eye video and the output of the frame memory 153 of the right-eye video is obtained, and only difference information between the left-eye video and the right-eye video is extracted. The difference information is subjected to the DCT transformation 148 and adaptive quantization 149 in the left-eye video line, and the variable-length encoding process 151 is performed, so that only the left-eye video difference information is recorded in the buffer memory 154. The subsequent processing until writing to the optical disc is the same as that in FIG. In the example shown in FIG. 27, the right-eye video is the main video and the left-eye video is the sub-video that takes the difference, but the left and right are reversed (the left-eye video is the main video and the right-eye video is the sub-video). It goes without saying that it is also good.

  In the stereoscopic video recording apparatus shown in FIG. 27, the method of further compressing the video information amount of one side by taking the difference between the left-eye video and the right-eye video has been described, but this video information is based on the parallax information. It is possible to further compress information. FIG. 28A and FIG. 28B schematically show the principle of parallax video. In FIG. 28A, the near-field image object 126G at the depth position 126M, the deep image object 126H at the depth position 126L, and the depth position (frontmost) 126N to both eyes 126F, and the viewing angle of the image object 126G. The viewing angle 126E of the image 126H is 126D. In FIG. 28B, the left eye image 126GA of the image object 126G, the right eye image 126GB of the image object 126G, the parallax amount 126I between the left eye image 126GA and the right eye image 126GB, and the same pixel point 126K of the left eye image and the right eye image. Yes. Further, in FIG. 28B, the left eye image 126HA of the image object 126H, the right eye image 126HB of the image object 126H, the parallax amount 126J between the left eye image 126HA and the right eye image 126HB, and the same pixel point 126KK of the left eye image and the right eye image. Yes.

  FIG. 29 is a block diagram of a stereoscopic video recording apparatus that further compresses one video. The stereoscopic video recording apparatus shown in FIG. 29 includes a parallax information calculation circuit 191A for calculating the parallax amount from the left-eye video and the right-eye video, a motion detection circuit 191C in the depth direction, and an estimated parallax information generation circuit 191D. ing. Furthermore, the stereoscopic video recording apparatus shown in FIG. 29 includes a reverse parallax calculation processing circuit 191B for converting the original left-eye video into the right-eye video from the estimated parallax information, and a right-eye video generated by the right-eye video and the reverse parallax calculation processing circuit 191B. Are provided with a DCT conversion circuit 191E that performs DCT conversion, an adaptive quantization circuit 191F, and a variable length coding 191G. Note that the part from the parallax information calculation circuit 191A to the variable length coding 191G is a part that performs the stereoscopic video compression processing.

  FIG. 30 is a diagram for explaining the encoding and decoding of the video stream in the compression method by the stereoscopic video recording apparatus of FIG. In FIG. 30, the parallax information calculation value 193, the left eye video 194A to 194G, the depth direction motion vector calculation value 196, the estimated parallax information 197, and the compressed video 195A to 195G of the right eye main video of the right eye video 192A to 192G, the left eye video and the right eye video. It is shown in the figure. Further, in FIG. 30, reproduction main images 198A to 198G based on the compressed images 195A to 195G of the right-eye main image, estimated parallax information 204 to 210 and reproduction sub-images 211 to 217 corresponding to the reproduction main images 198A to 198G, respectively, are illustrated. Has been. FIG. 31 is a diagram showing a video stream structure generated by the compression method by the stereoscopic video recording apparatus shown in FIG. 27 or FIG. The video stream structure shown in FIG. 31 is basically the same as that in FIG. 25, except that the difference video information 218 is used instead of the GOP 169 of the left-eye video and whether the 3D video format presence / absence 85 is GOP video information or difference information. The difference is that information 219 is added. FIG. 32 shows the stream structure at the transport level in the video stream structure generated by the compression method by the stereoscopic video recording apparatus shown in FIG. FIG. 32 is basically the same as FIG. 26 except that it is differential video information 218 instead of the GOP 169 of the left-eye video.

  Here, FIG. 28A shows the stereoscopic direction from the right-eye video and the left-eye video using parallax. In FIG. 28 (a), the parallax angle viewed from both eyes 126F looks different depending on the depth. For this reason, when the left and right parallax images are obtained, the front video object 126G is large as shown in FIG. 28B, the left eye image 126GA and the right eye image 126GB appear to be separated, and the parallax amount 126I also increases. On the other hand, the distant image object 126H is small, the left eye image 126HA and the right eye image 126HB appear to be separated, and the parallax amount 126J is also small.

  Therefore, if there is information on the parallax amount (126I or 126J) or the parallax angle information (126D or 126E), the right-eye video is estimated from the left-eye video (video generation by parallax information conversion) as shown in FIG. 28B (126K). And 126 KK). This condition is based on the premise that the brightness and color do not change depending on the viewing angle, and therefore, the information that cannot be estimated by this estimation about the wraparound of the video or the change of the shadow due to the angle.

  Here, in the stereoscopic video recording apparatus shown in FIG. 29, the parallax angle is extracted from the in-plane position information of the video object obtained from the motion detection block 147 of the left-eye video and the right-eye video, and the parallax information arithmetic circuit 191A Calculate disparity information in units of pixels. Further, in order to perform compression in the time axis direction, the depth direction motion detection circuit 191C extracts a motion vector in the depth direction for each screen. The estimated parallax information generation circuit 191D generates the estimated parallax information using the motion information and the parallax information in the depth direction. Further, as described above, the video on the opposite side (explained here as the left-eye video) cannot be completely reproduced from only one video (explained as the right-eye video here) information only by the parallax information, but the change due to the wraparound of the image ( Etc.), etc. will remain as information that cannot be estimated.

  For this reason, in the stereoscopic video recording apparatus shown in FIG. 29, in the stereoscopic video compression, the reverse parallax calculation processing circuit 191B first uses the parallax information to convert the video on the reverse side (described here as the left-eye video) using the parallax information. Decode reproduction is performed, and a difference between this and a video obtained by compressing the actually captured video on the opposite side (video on the frame memory 153 in the local decoder) is obtained. The information obtained by this difference is the information of the portion that cannot be reproduced due to the above-described wraparound of the image, and can cover the portion that cannot be completely reproduced by the disparity information even in the compressed stream using the disparity information. Although not shown in the figure, when a motion vector in the depth direction is extracted, the amount of change in parallax is also used as information. Therefore, as with a local decoder for normal information compression, an inverse quantization circuit and a DCT inverse transformation circuit, By using the frame memory and reproducing the original video from the motion vector in the depth direction by a local decoder and recomparison, the motion efficiency in the depth direction can be used to increase the compression efficiency.

  Here, video data in units of screens is shown as shown in FIG. Right-eye main images 192A to 192G are extracted from the right-eye camera in FIG. 30, and left-eye sub-images 194A to 194G are extracted from the left-eye camera. Here, the parallax information calculation circuit 191A in FIG. 29 calculates the parallax information 193 from the right-eye main videos 192A to 192G and the left-eye sub-pictures 194A to 194G. In the depth direction motion vector calculation 191C, the depth direction motion vector calculation value 196 is extracted from the change of the parallax information 193 in units of screens, and is generated as estimated parallax information 197. The estimated parallax information 197 itself may be a macroblock unit or a pixel unit of an image.

  On the other hand, at the time of video reproduction, the right-eye main videos 192A to 192G are encoded by video compression to become compressed videos 195A to 195G of the right-eye main video. Specifically, the compressed video of the right-eye main video includes an I-picture 195A of in-plane compressed video, P-pictures 195D and 195G subjected to temporal compression using in-plane motion vectors, and B pictures 195B, 195C, 195E, 195F. The compressed video of the right-eye main video is reproduced as the main video 198A to 198G by the normal compressed video decompression circuit. Then, reproduction sub-video (left-eye video) using the respective reproduction main video (right-eye video) 198A to 198G, estimated parallax information 204 to 210 for each screen, and information obtained by inverse quantization and inverse DCT conversion of the difference information for each screen. 211 to 217 are restored. Here, the restored video portion from the difference information plays a role of complementing a portion that cannot be reproduced by the estimated parallax information such as an image wraparound.

  The video stream using the compressed video in the stereoscopic video recording apparatus shown in FIG. 29 or FIG. 27 is shown as in FIG. 31, and is unitized as GOP video information unit data. This is because the right-eye video is originally unitized in units of GOP video information, and the differential compression data of the left eye needs to be adjusted to the unitization level of the image so that the right-eye video is used. Here, in the GOP header 170 in the GOP 168 of the right-eye video, incidental information regarding the stereoscopic video as described with reference to FIG. 25 is added. However, whether the left-eye video is compression information using the estimated parallax information 197 shown in FIG. 29, the differential compression information shown in FIG. 27, or is the method that does not compress the stereoscopic video shown in FIG. As shown in FIG. 31, it is necessary to describe information 219 indicating whether the information is GOP video information or difference information in the presence / absence 85 of the 3D video format. Further, when the stream structure is viewed at the level of the transport packet, it is shown as in FIG. 32. As in FIG. 26, the end portion of the GOP video data is padded within the transport packet, and the compressed video data in the stereoscopic direction is also displayed. Even so, the end portion is padded in the transport packet. In the above, the right-eye video is the main video and the left-eye video is the sub-video compressed in the stereoscopic direction, but the reverse is not a problem at all. The right-eye video is the main video, the left-eye video is the sub-video, and the left-eye video is The main video may be a video stream in which the right video is mixed with the sub video. However, when the mixture is permitted according to the standard, it is necessary to describe the identification information which video is the main video and which is the sub video.

  It is also possible to more easily convert the left-eye and right-eye stereoscopic video into a video stream. For example, the stereoscopic video recording apparatus shown in FIG. 33 includes a synthesis circuit 220 that is an image configuration processing unit that easily converts a video stream. FIG. 34 shows the structure of the video stream in the stereoscopic video recording apparatus shown in FIG. 33, and the left-eye or right-eye video GOP 221 is one GOP video information unit. FIG. 35 shows the structure of the transport packet level hierarchy of the video stream in the stereoscopic video recording apparatus shown in FIG.

  Here, the input left-eye and right-eye parallax images are once input to the synthesis circuit 220, and are inserted as two vertically long screens in one screen as shown in FIG. At this time, the pixels of each video are not simply thinned out in the video line, but are subjected to filter processing and compressed in the horizontal direction, and then combined with the left-eye right-eye video. In this manner, each screen is composed of two vertically long left-eye videos and right-eye videos, and thereafter, a stream is generated using a normal image compression processing method. Even in this case, in the video stream shown in FIG. 34, it is described in the supplementary information 51 or the 3D information area 176 in the GOP header 170 that the video information is compressed in the horizontal direction so that it is not reproduced as it is on a normal TV. It is necessary to do. Further, even when the video stream structure shown in FIG. 35 is adopted, at the level of the transport packet, as in FIG. 26 and FIG. 32, the padding data 184 is compared with the final data 183 of the transport packet at the end of the GOP video information. Process to fill in information. In the present invention, the stereoscopic video recording apparatus and the stereoscopic video recording method recorded on the optical disc are described, but it goes without saying that the same effect can be obtained even if the recording medium is a hard disk.

(Embodiment 4)
Next, the fourth embodiment will be described below with reference to the drawings. Although the stereoscopic video recording apparatus has been described in the third embodiment, the stereoscopic video reproducing apparatus will be described in the present embodiment. FIG. 36 is a block diagram of the stereoscopic video reproduction apparatus according to the present embodiment. 36, the demodulation / correction circuit 222, the address header recognition circuit 223, the IF (interface) 224 for connecting the optical disk drive part and the back end as the video / audio processing part, and the optical disk drive And a data buffer 225 for temporarily storing data from the portion. 36, a system decoder 226 for separating a stream of video / audio data, an MPEG / H264 decoder 227 for decompressing compressed video, an audio decoder 228, subtitle display, and the like. OSD decoder 229 and a depth generation circuit 229A for OSD information. Further, in the stereoscopic video reproduction device shown in FIG. 36, a 3D video processing circuit 230, a blending processing circuit 229B for covering the video with OSD information, a general-purpose IF 231 to the outside, a right-eye dedicated IF 232, a left-eye dedicated IF 233, A buffer circuit 234 and a system controller 235 for the entire back end are provided.

  FIG. 37 is a block diagram showing a portion for decoding the left-eye video from the compressed video in the stereoscopic direction shown in the third embodiment. In FIG. 37, a system decoder 236 for extracting disparity information and depth motion vector information of a video stream, an MPEG and H264 decoder 237 for decoding a compressed video stream such as MPEG and H264, disparity information 238, and motion vector information 239, a parallax information calculation circuit 240, and a left-eye image reproduction circuit 241. Note that the parallax information 238, the motion vector information 239, the parallax information calculation circuit 240, and the left-eye video reproduction circuit 241 constitute a 3D video processing circuit 230.

  Here, in the stereoscopic video reproduction apparatus shown in FIG. 36, first, the audio / video data and the auxiliary data described on the optical disk 165 are reproduced by the demodulation and correction circuit 222 in the optical disk drive. At this time, the servo circuit 162 operates so as to continue to extract the reproduction signal from the optical head 164 with high quality, and the address header recognition circuit 223 functions to instantly access a predetermined address. Here, the data reproduced from the optical disk drive is once inputted to the data buffer circuit 225 via the IF circuit 224 and then inputted to the system decoder 226. The system decoder 226 separates a stream such as video / audio data, and audio information is input to the audio decoder 228, OSD information is input to the OSD decoder 229, and video information is input to the MPEG / H264 decoder 227.

  Note that the OSD information is generated as OSD information having a depth by the supplementary information obtained from the system decoder 226 by the OSD depth generation circuit 229A. Also, the video stream decoded by the MPEG / H264 decoder 227 is processed as 3D video information by the 3D video processing circuit 230, blended with the OSD video having a depth by the blending circuit 229B, and HDMI is used when the transfer rate is low. The left-eye image can be output by the left-eye dedicated IF 233, and the right-eye image can be output by the right-eye dedicated IF 232.

  Further, when the one-side video as shown in the third embodiment is further compressed using the parallax information, the 3D video processing 230 in the stereoscopic projection playback device is configured as shown in FIG. Here, the parallax information calculation circuit 240 performs the parallax information calculation for each pixel or macroblock using the parallax information 238 and the depth motion vector 239 which are the compressed video information of the left eye extracted by the system decoder 236, and starts from the right-eye video. A conversion coefficient for generating a left-eye image is generated. Using the conversion coefficient, the left-eye video reproduction circuit 241 reproduces the left-eye video from the right-eye video generated by the MPEG / H264 decoder. Here, if the compressed left-eye video is only compressed by parallax information, only re-conversion based on the output of the parallax information calculation circuit 240 is performed, but DCT transformation and adaptive quantization as in the stereoscopic video recording apparatus shown in FIG. If the compressed information has been subjected to the above, it is necessary to incorporate an inverse quantization and inverse transform circuit in the left-eye video reproduction circuit 241.

  Next, a description will be given of a stereoscopic playback device that reproduces non-stereoscopic 2D video from left-eye and right-eye stereoscopic video. FIG. 38A is a block diagram of a stereoscopic playback device that reproduces 2D video. In FIG. 38A, a synthesis processing circuit 242 based on parallax information is provided. FIG. 38B is a diagram schematically illustrating an image formed by the composition processing circuit 242, and an image obtained by combining the left-eye image 232A and the right-eye image 233A is a 2D image 231A. Here, a general display device such as a TV does not always support 3D video, but rather supports 2D video. Therefore, when playing back a medium in which only 3D video is described, it is desirable that 2D video can be played back. As the simplest method, 2D video can be reproduced by displaying only the right-eye video or the left-eye video. For example, if the TV only supports 2D video, it is automatically detected in the link connection process between the player and the TV, and only one video is always played back.

  However, in this method, in the case of an image that is close to the eyes (appears popping out), the amount of parallax is large, and an image with a significantly shifted left and right position such as a left eye image 232A and a right eye image 233A shown in FIG. There was a problem that would become. Therefore, the parallax information in the left-eye and right-eye images can be combined to reproduce a video at an intermediate position, such as the 2D video 231A, so that a 2D video without a sense of incongruity can be reproduced. However, since both sides of the screen in this case cannot be calculated if the amount of parallax is large, if the original video is a left-eye video that is wide on the left side and a right-eye video is not shot on the right side, the video 241A The image looks like the part (both sides of the screen) is cut.

  Furthermore, as shown in the second embodiment, there is a concern that the amount of popping out of the screen is large, which may increase eye fatigue and startle feeling. Therefore, FIG. 39A shows a block diagram of a stereoscopic video reproduction apparatus in which the pop-out amount can be varied. 39A includes a left-eye video reproduction circuit 243 based on coefficient-changed parallax, a right-eye video conversion processing circuit 244 based on coefficient-changed parallax, a user interface 245 for changing the pop-out amount, and a parallax information coefficient change unit 246. ing. FIG. 39B is a diagram for explaining the variation of the pop-out amount of the stereoscopic video reproduction device. Further, FIG. 39 (c) is a diagram for explaining the result when the pop-out amount is changed by the circuit of FIG. 39 (a). FIG. 39D shows an OSD bar 246A for changing the pop-up amount displayed on the display device connected to the stereoscopic video reproduction device.

  In the method in which one-side video is compressed with parallax information as in the stereoscopic video recording apparatus shown in FIG. 29, the parallax information related to the pop-out amount is linked in units of pixels or macroblocks. Therefore, when there is an instruction to change the pop-out amount from the user, the instruction is input to the user interface 245 of FIG. 39D using, for example, an OSD screen as shown in the OSD bar 246A of the TV screen, and the disparity information coefficient The conversion unit determines how much the degree of popping is attenuated by the changing unit 246. The parallax calculation amount in the parallax information calculation circuit 240 is determined based on the conversion coefficient. If the left-eye video, the left-eye video reproduction circuit 243 using the coefficient-changed parallax, and if the right-eye video, the image conversion processing circuit 244 based on the coefficient variable parallax information. Thus, the parallax amount between the left-eye image and the right-eye image is converted into a small size so that the left-eye image 126GA and the right-eye image 126GB are changed from a broken line to a solid line as shown in FIG. As a result, the stereoscopic image obtained from the outputs of the dedicated IFs 232 and 233 is reproduced with a small pop-out amount like the stereoscopically viewed triangular figure in FIG.

  Further, in the stereoscopic video playback device shown in FIG. 39A, when the parallax information is recorded on the video stream, the pop-out amount is converted using this, but there is no parallax information. Can be considered. Therefore, the stereoscopic video playback device shown in FIG. 40 shows a configuration in which the pop-out amount can be controlled even when parallax information is not recorded on the video stream. In the stereoscopic video reproduction device shown in FIG. 40, MPEG and H264 decoders 237A and 237B, a parallax information extraction unit 247, a parallax conversion unit 248 for a right-eye video, and a parallax conversion unit 249 for a left-eye video, respectively, for left-eye video and right-eye video. And. In the stereoscopic video reproduction apparatus shown in FIG. 40, the parallax information may be newly detected from the decoded video of the left-eye video and the right-eye video by the parallax video extraction unit 247. Also, this disparity information is generated by the disparity information calculation unit 240 via the user interface 245 and the disparity information coefficient changing unit 246 as in the case of FIG. 39A, and the disparity conversion of the right-eye image is performed. Are supplied to the unit 248 and the parallax conversion unit 249 of the left-eye video.

  In the present embodiment, an apparatus and a reproducing method for reproducing stereoscopic video information recorded on an optical disc have been described. Needless to say, the same effect can be obtained even if a hard disk is used as a storage medium.

  1 Right-eye image, 2 Left-eye image, 3, 8 Display device, 4 Shutter, 5 Image, 6 Recording device, 7 Glasses, 9, 10 Deflection plate, 11 Display optical system, 12 Display panel, 13 Synchronous rotating member, 14 Light source, 15 to 20 Display device, 21 Rotating mirror, 22 Image signal, 23 Video control information, 24, 25 Video title, 26 Recording medium, 146 AD converter, 147 Motion detection circuit, 148 DCT conversion circuit, 149 Adaptive quantization circuit, 150 Inverse quantization circuit, 151 variable length coding circuit, 152 DCT inverse conversion circuit, 153 frame memory, 154 buffer memory, 155 OSD encoder, 156 audio encoder, 157 format encoder, 158 modulation circuit, 159 LD modulation circuit, 160 address header Recognition circuit, 161 playback amp 162, servo circuit, 163 feed motor, 164 optical head, 165 optical disk, 166 rotation motor, 167 system controller, 191A parallax information calculation circuit, 191B reverse parallax calculation processing circuit, 191C depth direction motion detection circuit, 191D estimated parallax information generation circuit 191E DCT conversion circuit, 191F adaptive quantization circuit, 191G variable length coding circuit, 222 demodulation correction circuit, 223 address header recognition circuit, 224 interface circuit, 225 data buffer, 226, 236 system decoder, 227, 237 MPEG / H264 Decoder, 228 Audio Decoder, 229 OSD Decoder, 229A OSD Depth Generation Circuit, 229B Blending Circuit, 230 3D Video Processing Circuit, 231 General Purpose IF, 23 , 233 Dedicated IF, 234 buffer, 235 System decoder, 238 Disparity information generation circuit, 239 Motion vector generation circuit, 240 Disparity information calculation circuit, 241 Left-eye image reproduction circuit, 242, 244 Compositing processing circuit based on disparity information, 243 Left-eye image Reproduction circuit, 245A OSD bar, 246 parallax information coefficient changing unit, 245 user interface, 247 parallax information extracting unit, 248 right-eye video parallax conversion circuit, 249 left-eye video parallax conversion circuit.

Claims (5)

A video information recording method for recording a stereoscopic video using parallax consisting of a left-eye video and a right-eye video on an optical disc,
One of the left-eye video and the right-eye video is the main video, and the main video is an I picture that is data-compressed within a frame, and a P-picture that is data-compressed by adding motion compensation using the I picture in the forward direction in time. And when recording stereoscopic video information configured as a digital main video information block including a B picture compressed by applying motion compensation by the I picture or the P picture in the longitudinal direction in time, on an optical disc,
The sub-picture, which is the other of the main pictures, is further subtracted using motion detection as sub-picture information corresponding to the picture of the main picture using motion detection, and compared with the main picture information block. Generate as sub-picture digital sub-picture information block with reduced video data amount,
Further, the video data of the main video information block and the sub video information block are arranged such that the main video information block and the sub video information block are alternately arranged on the optical disc for each video information block in terms of time. While rearranging alternately,
The graphics data related to the main video information block and the sub video information block is recorded data so as to be collectively arranged as a combination of the video information block, the graphics data, and the audio data. As well as
At the time of recording the stereoscopic video information, the main video information block and the sub-video information block in the optical disc are provided in two different regions, the inner peripheral side and the outer peripheral side of the optical disc, For the video supplementary information recording area related to the video information, supplementary information including depth information for designating the pop-up or depth of graphics information displayed together with the stereoscopic video in the main video information block and the sub-video information block. In addition to recording for each graphics information display unit to be displayed,
A table information recording area composed of auxiliary information related to the video information block in the video information block unit is secured for the video auxiliary information recording area, and the video information block is a main video information block or a secondary video information block. By recording 3D video incidental information indicating whether it is a video information block,
When playing back 2D video, play back the main video and graphic data that have not been differentially compressed,
When playing back a stereoscopic video, play back both the main video information block and the sub-video information block and graphic data,
By reproducing the 3D video supplementary information of the video information block in the additional information recording area for decoding the differentially compressed video and the graphics depth information for stereoscopically viewing the graphics,
An optical disc video information recording method capable of displaying stereoscopic graphics superimposed on a stereoscopic video using the parallax. A method for reproducing video information on an optical disc on which a stereoscopic video using parallax consisting of a left-eye video and a right-eye video is recorded,
Information recorded on the optical disc is
One of the left-eye video and the right-eye video is the main video, and the main video is an I picture that is data-compressed within a frame, and a P-picture that is data-compressed by adding motion compensation using the I picture in the forward direction in time. And a digital main video information block including a B picture compressed by applying motion compensation by the I picture or the P picture in the longitudinal direction in time,
Further, the sub-video that is the other of the main video is configured as a digital sub-video information block that is further differentially compressed using motion detection as sub-video information corresponding to a picture of the main video,
The main video information block and the further compressed sub video information block are alternately arranged in time for each video information block on the optical disc,
Further, the graphics data related to the main video information block and the sub video information block are collectively arranged as a combination with respect to the video data of the video information block,
In addition, the video ancillary related to the video information provided in two areas on the optical disc, which are different from the main video information block and the sub-video information block, and on the inner peripheral side and the outer peripheral side of the optical disc. In the information recording area, a graphics information display unit in which auxiliary information which is depth information for designating the pop-up or depth of graphics information displayed together with the stereoscopic video in the main video information block and the sub-video information block is displayed Recorded every time,
Further, in the video supplementary information recording area, a recording area of table information consisting of supplementary information related to the video information block in the video information block unit is secured, and the video information block is a main video information block or a 3D video. In the optical disc in which the 3D video supplementary information indicating the sub video information block is recorded in the table information area,
When playing back 2D video, when playing back the video data recording area, the sub-picture is jumped to play back the graphics data area where the main video information block and the graphics data are arranged. ,
When playing back a stereoscopic video, both the main video information block and the sub-video information block subjected to the differential compression and the graphics data area are played back, and the graphics information in the video ancillary information recording area is further reproduced. By reproducing auxiliary information including depth information specifying the pop-up or depth of the image and 3D video auxiliary information indicating whether the video information block stored in the table information is a main video information block or a sub-video information block,
The sub-picture information block in which the differential compression is performed in the video information block is decoded and separated into a main picture and a sub-picture that are stereoscopically represented by performing synchronous display,
A method for reproducing video information on an optical disc, wherein, in the separated video information, stereoscopic graphic information with parallax added by graphics depth information is displayed superimposed on the parallax video. One of the left-eye video and the right-eye video is the main video, and the main video is an I picture that is data-compressed within a frame, and a P-picture that is data-compressed by adding motion compensation using the I picture in the forward direction in time. And a digital main video information block including a B picture compressed by applying motion compensation by the I picture or the P picture in the longitudinal direction in time,
Further, the sub-video that is the other of the main video is configured as a digital sub-video information block that is further differentially compressed using motion detection as sub-video information corresponding to a picture of the main video,
The main video information block and the further compressed sub video information block are alternately arranged in time for each video information block on the optical disc,
For the graphics data related to the main video information block and the sub-video information block, record data is generated so as to be arranged together as a combination of video data of the video information block,
In addition, the video ancillary related to the video information provided in two areas on the optical disc, which are different from the main video information block and the sub-video information block, and on the inner peripheral side and the outer peripheral side of the optical disc. In each information recording area, each graphics information display unit in which auxiliary information including depth information for specifying the pop-up or depth of graphics information displayed together with the stereoscopic video in the main video information block and the sub-video information block is displayed. Recorded in
In addition, in the video incidental information recording area, a recording area for table information including auxiliary information related to the video information block in the video information block unit is secured, and the video information block is a main video information block or a 3D video. In an optical disc apparatus for reproducing an optical disc in which 3D video supplementary information indicating a sub-picture information block is recorded in the table information area,
An optical disk drive for reproducing recorded data from the optical disk;
System decoding means for extracting and separating the main video and the sub video alternately arranged from the optical disc based on the 3D video auxiliary information of the auxiliary information;
Video decoding means for decoding the extracted main video information block;
Second video decoding means for decoding, using the motion vector of the main video, the extracted and differentially compressed sub-video that is the other of the main video;
OSD generation means for generating graphics data;
OSD depth generation means for generating stereoscopic graphics data by giving a parallax to the graphics data based on the depth information in the auxiliary information reproduced from the optical disc drive in the generated graphics data;
And a blending unit for superimposing the stereoscopic graphics data on the stereoscopic video. A video information recording apparatus for recording a stereoscopic video using parallax consisting of a left-eye video and a right-eye video on an optical disc,
The video information recording device comprises:
Either the left-eye image or the right-eye image is used as the main image, and the I picture that is data-compressed within the frame in the main image and the P-picture that is data-compressed by adding motion compensation by the I picture in the forward direction in time And a video encoding means for generating a digital main video information block including a B picture compressed by applying motion compensation by the I picture or the P picture in the longitudinal direction in time,
The sub-picture, which is the other of the main pictures, is further subtracted using motion detection as sub-picture information corresponding to the picture of the main picture using motion detection, and compared with the main picture information block. A second video encoding means for generating a digital sub-picture information block of a sub-picture with a reduced amount of video data;
The video data of the main video information block and the sub video information block are alternately arranged so that the main video information block and the sub video information block are alternately arranged on the optical disc for each video information block. A format encoding unit for rearranging and generating data so that graphics data related to the main video information block and the sub video information block are also arranged as a combination of video data of the video information block And buffer memory means;
OSD encoding means for generating graphics information;
Having data modulation means and an optical head for writing data to the optical disc;
Further, the format encoding means includes
The video provided at the time of recording the stereoscopic video information, in the two areas on the inner side and the outer side of the optical disc that are different from the main video information block and the sub-video information block in the optical disc. With respect to the video incidental information recording area related to information, incidental information including depth information that specifies the pop-up or depth of graphics information displayed together with the stereoscopic video in the main video information block and the sub-video information block, Recording is performed for each graphic information display unit to be displayed, and a recording area for table information including auxiliary information regarding the video information block in the video information block unit is secured in the video additional information recording area, and the video 3 indicating whether the information block is a main video information block or a sub-video information block that is 3D video Has a function of recording video supplementary information,
A video information recording apparatus for recording stereoscopic video information including sub-video information with a reduced video data amount and stereoscopic graphics information on the optical disc. The information recorded on the optical disc
One of the left-eye video and the right-eye video is the main video, and the main video is an I picture that is data-compressed within a frame, and a P-picture that is data-compressed by adding motion compensation using the I picture in the forward direction in time. And a digital main video information block including a B picture compressed by applying motion compensation by the I picture or the P picture in the longitudinal direction in time,
Further, the sub-video that is the other of the main video is configured as a digital sub-video information block that is further differentially compressed using motion detection as sub-video information corresponding to a picture of the main video,
The main video information block and the further compressed sub video information block are alternately arranged in time for each video information block on the optical disc,
In addition, graphics data related to the main video information block and the sub-video information block are arranged in combination with video data of the video information block,
In addition, the video ancillary related to the video information provided in two areas on the optical disc, which are different from the main video information block and the sub-video information block, and on the inner peripheral side and the outer peripheral side of the optical disc. In the information recording area, in the main video information block and the sub-video information block, additional information which is depth information for designating the pop-up or depth of graphics information displayed together with the stereoscopic video is displayed. It is recorded for each unit,
Further, in the video supplementary information recording area, a recording area of table information consisting of supplementary information related to the video information block in the video information block unit is secured, and the video information block is a main video information block or a 3D video. An optical disc in which 3D video supplementary information indicating a sub video information block is recorded in the table information area.

Images