JP2004207772A - Stereoscopic image display apparatus, recording method, and transmission method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display an excellent stereoscopic image whose parasitic amount is adjusted with a little sense of incongruity. <P>SOLUTION: A stereoscopic image display apparatus A for displaying a stereoscopic image on the basis of a plurality of images corresponding to a plurality of view points includes: a parallax amount adjustment information calculation section 3 for calculating parallax amount adjustment information with respect to adjustment of the parallax amount on the basis of information related to change of the parallax amount; an image processing section 1 for generating a stereoscopic display image on the basis of the parallax amount adjustment information; and an image interpolation section 5 for interpolating a prescribed region in a plurality of the images. The image interpolation section 5 interpolates only a non pixel region by using other pixel values when the non pixel region without any pixel value exists in a display region of the stereoscopic display image. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for stereoscopically displaying a plurality of images corresponding to a plurality of viewpoints.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a stereoscopic image display method is known in which a stereoscopic image can be viewed by stereoscopically viewing a set of images having parallax. For example, a left-eye image and a right-eye image are alternately output to a display device, and the user reproduces the image through glasses that can switch shutters in synchronization with the switching timing of the display, thereby forming a stereoscopic image. Can be observed.
[0003]
As a method of reproducing a stereoscopic image without using special glasses or the like, there is a method called a parallax barrier method. Each of the image for the left eye and the image for the right eye is decomposed into strips in the vertical scanning direction of the image, and alternately arranged to form one image. The display device that displays the image has a strip-shaped slit similar to that obtained when the image is decomposed. The strip-shaped image data is observed on the display device through the slit. When the image for the left eye arranged in a strip shape by the polarizing plate is reproduced with the left eye of the user and the image for the right eye is reproduced with the right eye, a stereoscopic effect can be obtained in the image. There is also a method called a lenticular method using a lenticular lens instead of a slit.
[0004]
There is disclosed a technique for changing the stereoscopic effect of a reproduced image so that a user can view a better stereoscopic image. When a human observes an object three-dimensionally, different images are observed by the left and right eyes, and these images have a shift called parallax.
[0005]
Humans perceive a three-dimensional effect based on this parallax. The amount of parallax is called a parallax amount, and the stereoscopic effect is adjusted by adjusting the parallax amount. This document describes the technology of adjusting the amount of parallax when displaying and displaying a stereoscopic image in digital broadcasting, and displaying the image on a stereoscopic display. (For example, see Patent Document 1).
[Patent Document 1]
JP-A-2000-78615, FIG.
[0006]
[Problems to be solved by the invention]
By the way, when adjusting the amount of parallax, for example, in a method in which left and right images are alternately arranged in a strip shape for each line, such as a lenticular method or a parallax barrier method, the combination position of the strip data is usually determined. The parallax amount can be adjusted by changing it, but the accuracy of the adjustment is determined by the strip width. Therefore, the three-dimensional effect cannot always be adjusted according to the user's preference.
[0007]
Further, when the combination position of the strip-shaped data is changed, a portion where data for reproducing as a stereoscopic image does not exist occurs, and the portion cannot be displayed in a good stereoscopic view. Therefore, a portion where a stereoscopic image can be displayed well (a display region where the stereoscopic image can be observed) becomes small.
[0008]
On the other hand, even if all the image data is to be displayed in the image after the parallax adjustment, the width of the strip-shaped image data after the parallax adjustment is wider than the width of the display area by the parallax adjustment, and the entire image data is displayed. Cannot be displayed.
An object of the present invention is to display a good stereoscopic image with less discomfort in a stereoscopic image in which the amount of parallax has been adjusted.
[0009]
[Means for Solving the Problems]
According to one aspect of the present invention, there is provided a stereoscopic image display device that displays a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints, and includes a parallax adjustment method for adjusting a parallax amount based on information regarding a change in the parallax amount. A parallax amount adjustment information calculation unit that calculates amount adjustment information; an image processing unit that generates an image for stereoscopic display based on the parallax amount adjustment information; and an image interpolation unit that interpolates a predetermined area in the plurality of images. Is provided.
[0010]
It is preferable that, when there is a non-pixel value area where no pixel value exists in the display area of the image for stereoscopic display, the image interpolation unit interpolates only the non-pixel value area using another pixel value. .
According to the three-dimensional image display device, it is possible to interpolate the pixel value of the non-pixel value area in association with the change in the amount of parallax.
[0011]
According to another aspect of the present invention, there is provided a stereoscopic image display device that displays a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints, and adjusts a parallax amount based on information on a change in the parallax amount. A parallax adjustment information calculation unit that calculates parallax adjustment information relating to the image, an image processing unit that generates an image for stereoscopic display based on the parallax adjustment information, and a new image in a predetermined area in the plurality of images. And a three-dimensional image display device including an image generation unit that generates the image data.
[0012]
When there is a non-pixel area where no pixel value exists in the display area of the image for stereoscopic display, the image generation unit preferably generates a new image only in the non-pixel area.
According to the three-dimensional image display device, it is possible to generate a new image in the non-pixel value area in association with the change in the amount of parallax.
[0013]
BEST MODE FOR CARRYING OUT THE INVENTION
In this specification, each of the RGB data of the three primary colors is called a dot, and a group of the RGB data of the three primary colors is called a pixel. In the present embodiment, the image data includes a moving image and a still image. Further, the image data includes, for example, compressed image data using a still image compression technology such as JPEG or a moving image compression technology such as MPEG-4.
[0014]
Hereinafter, a stereoscopic image display technique according to the first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a functional block diagram illustrating a configuration example of the stereoscopic image display device according to the present embodiment. As shown in FIG. 1, the stereoscopic image display device A according to the present embodiment generates image data capable of performing stereoscopic display based on input image data Din (hereinafter, also referred to as “stereoscopic image data”). A stereoscopic image processing unit 1 for performing image processing, a user input unit 2 for inputting by a user, a parallax amount adjustment information calculation unit 3 for calculating parallax amount adjustment information based on a user's input, A display area determining unit 4 for determining an area to be actually displayed on the stereoscopic video display device from the stereoscopic video, an image interpolating unit 5 for generating interpolation data in a specific area, and a determined area of the generated image. It comprises a display unit 6 for displaying.
[0015]
First, the left-eye image data and the right-eye image data input to the stereoscopic image display device A are converted into data that can be stereoscopically displayed in the stereoscopic image processing unit 1. For example, in the case of a stereoscopic image display device using a parallax barrier method or a lenticular method, the stereoscopic image processing unit 1 outputs a stereoscopic image in which left-eye image data and right-eye image data are alternately arranged in a strip shape. Generate data.
[0016]
The user inputs the data for adjusting the amount of parallax of the stereoscopic image to be displayed using the user input unit 2. The input parallax adjustment data is converted into parallax adjustment information in the parallax adjustment information calculation unit 3 and output to the stereoscopic image processing unit 1. The stereoscopic image processing unit 1 generates stereoscopic image data using the parallax amount adjustment information when the stereoscopic image data is input when generating the stereoscopic image data. The generated stereoscopic image data is output to the display area determination unit 4.
[0017]
The display area determination unit 4 determines whether or not the generated stereoscopic image data can be favorably displayed on the display unit 6. The image width of the stereoscopic image data on which the parallax adjustment has been performed may not correspond to the image width that can be displayed by the display unit 6. In such a case, the pixel area of the display unit 6 does not match the pixel area of the stereoscopic image data, and an uncorresponding area occurs. In such an area, there is a case where the image data cannot be displayed in a good state without knowing how to display the image data.
[0018]
Therefore, the display area determination unit 4 determines whether there is an area that cannot be handled, and if the display area determination unit 4 determines that there is an area that cannot be handled, the image interpolation unit 5 cannot be handled. The information S1 relating to the area is output. When the display area determination unit 4 determines that there is no area that cannot be matched, the stereoscopic image data S2 is output to the display unit 6 as it is (through).
[0019]
The image interpolation unit 5 performs an interpolation process on the stereoscopic image data output from the display area determination unit 4 so as to provide pixel data to the area determined by the display area determination unit 4 by a predetermined method. In addition, the stereoscopic image data subjected to the interpolation processing is output to the display unit 6. The display unit 6 displays the stereoscopic image data S2 output from the display area determination unit 4 or the stereoscopic image data S3 output from the image interpolation unit 5.
[0020]
In the stereoscopic image display apparatus A according to the present embodiment, the format of the stereoscopic image data displayed on the display unit 6 is such that the left and right images typified by the parallax barrier method and the lenticular method are image data for one pixel. This is a method in which a stereoscopic view is performed by alternately arranging the images in a strip shape every time. The input image is image data including data that can be viewed stereoscopically, and the image data to be stereoscopically viewed on the display unit 6 is that in which data of an image for the left eye and an image for the right eye are arranged in a strip shape. . The input image data only needs to be able to generate strip data in the stereoscopic image processing unit 1. For example, the input image may be image data originally synthesized in a strip shape. In this case, the stereoscopic image processing unit 1 outputs the input data as it is without creating a strip image again.
[0021]
Alternatively, the image data for the left eye and the image data for the right eye may be separated image data. In this case, the stereoscopic image processing unit 1 creates a strip-shaped image from the left-eye image and the right-eye image as described in FIG. 2A below. FIG. 2A is a diagram illustrating an example of generating stereoscopic image data from input image data for the left eye and input image data for the right eye. From the image data for the left eye and the image data for the right eye in FIG. 2A, pixels (201, 202) reproduced by the same slit when stereoscopically viewing the strip-shaped data are obtained, and they are arranged. The data 200 for stereoscopic display is generated. By repeating this processing, it is possible to generate stereoscopic image data in which the width of the strip is one pixel.
[0022]
When the strip-shaped stereoscopic image data 200 is created by such a method, the width of the strip-shaped stereoscopic image data 200 is equal to the width of the input image data 201 for the left eye and the input image data 202 for the right eye. It becomes equal to the sum of the widths of the respective images. Therefore, in order to create data that can be displayed by the display unit 6, the input image data 201 for the left eye and the input image 202 for the right eye are formed by half of the generated strip-shaped three-dimensional display image data 200. It is necessary to keep the image width.
[0023]
When the image width of the input image data 201 for the left eye and the input image data 202 for the right eye is the same as the display width of the display unit 6, when generating the strip-shaped image data, the left eye Data is thinned out so that the image width of the input image data 201 for the right eye and the input image data 202 for the right eye is reduced to half, and then strip-shaped image data is generated.
[0024]
FIG. 2B shows an example of a method for creating strip-shaped data by thinning out data from input image data. As shown in FIG. 2B, one dot of G data 204 from input image data 201 for the left eye is taken from each pixel of the image for the left eye and the image for the right eye among the three primary colors of RGB, and A process for generating one RGB pattern 203 by taking only two dots of R data 205 and B data 206 from the input image data 202 of FIG. This processing is performed for each image alternately (a pattern that takes one data from the input image data 201 for the left eye and takes two data from the input image data 202 for the right eye, A pattern in which two pieces of data are taken from the data 201 and one piece of data is taken out of the input image data 202 for the right eye) is repeated to create the image data 200 for stereoscopic vision. The method of forming a strip image is not limited to the method described with reference to FIG. 2B. For example, the pixel denoted by reference numeral 203 in FIG. 2 is a left-eye G1 dot (204), It is composed of R1 dot (205) for right eye and B1 dot (206), but R1 dot (207) for left eye, B1 dot (208) and G1 dot for right eye (209) with reversed pattern. May be used to generate image data. The stereoscopic image processing unit 1 shown in FIG. 1 compares the number of pixels of each input image with the number of displayable pixels, determines whether to thin out the input image to generate a strip image, and Is generated.
[0025]
Next, processing when the user adjusts the amount of parallax will be described. The user adjusts the amount of parallax using the user input unit 2 in FIG. The user input unit 2 is not limited to an input device such as a keyboard or a mouse, or a remote controller of any shape and type. For example, a method of adjusting a parallax amount by an amount proportional to the number of times a key is pressed in the specific direction by pressing a key for designating a specific direction of the keyboard at least once is also conceivable. The data input by the user input unit 2 is converted into parallax adjustment information in the parallax adjustment information calculation unit 3. For example, it is converted into specific information such as “the right-eye image data is moved to the left by one pixel”. The parallax adjustment information calculation unit 3 outputs the calculated parallax adjustment information to the stereoscopic image processing unit 1. The stereoscopic image processing unit 1 generates strip image data using the parallax amount adjustment information.
[0026]
The progress of the parallax adjustment process will be described with reference to FIGS. In FIG. 3, a block denoted by a reference numeral 301 and the like indicates data for one pixel, and L and R are codes indicating a left-eye image and a right-eye image, respectively. The numbers attached below L or R indicate pixel numbers, which are assigned for convenience of explanation, and are assigned in order from the left end of the image data. Reference numeral 301 indicates the leftmost pixel of the left-eye image. Reference numeral 304 in FIG. 3 and reference numeral 403 in FIG. 4 indicate the width of the display area where the display unit displays each image data.
[0027]
Further, when observing a stereoscopic image using a lenticular lens, a parallax barrier, or the like, pixels observed through the same slit (for example, pixels indicated by reference numerals 1103 and 1104 in FIG. 11) are defined as a stereoscopic image. The pair to be reproduced is represented in the upper and lower directions as indicated by reference numerals 302 and 303 in FIG. Unless the parallax amount is adjusted by the user, there is data to be displayed for all the pixels of the display unit 6 in the generated strip-shaped image data. In the area determination, it is determined that there is no area to be interpolated.
FIG. 4 shows pixels and corresponding pixel data in the case where the user's adjustment by the parallax amount adjustment information calculation unit 3 moves, for example, the left-eye image by one pixel to the right.
[0028]
If the display position of the image data is moved and displayed as it is, there is no pixel data to be displayed like the position of the pixel 401 (shown by a broken line) in FIG. As described above, pixel data that cannot be displayed because it protrudes from the display area 403 occurs. At this time, the display area determination unit 4 determines that there is no image data to be displayed in the pixel 401 and that the pixel 402 is out of the display area 403, and outputs the information to the image interpolation unit 5.
[0029]
The movement of the image for adjusting the amount of parallax described above is described in the case of a unit of one pixel. In this case, the adjustment accuracy may be too coarse depending on the size of the image data and the size of the display unit 6.
[0030]
Human recognition of an image can be considered as recognition of RGB as a set of data when each dot of the RGB primary color data forming each pixel forms an image on the eye. Therefore, for example, even if the arrangement of RGB is changed to GRB, these may be recognized as a unit by the human eye. FIG. 5 illustrates the image data for the left eye in FIG. 3 expressed in RGB dot levels. In order to recognize the L3 image data (pixel 302 in FIG. 3), LR3 (dot 501 in FIG. 5), L-G3 (dot 502 in FIG. 5), and LB3 (dot 503 in FIG. 5) It is only necessary that the three dots can be recognized collectively.
[0031]
That is, it is possible to move at the dot level of the three primary color data RGB. FIG. 6 shows this state. FIG. 6 focuses on the image for the left eye in FIG. 5, and FIG. 6A shows the same state as FIG. Reference numeral 604 indicates a display area where the display unit displays the left-eye image. FIG. 6B shows a state (FIG. 4) in which one pixel has been moved. FIG. 6C shows a state in which the image is moved by one dot at the dot level. The shaded portion indicates that dot data is missing due to the movement of the left-eye image. FIG. 6C focuses on the dots of the R data (reference numeral 601 in FIG. 6A) of the image for the left eye, and L-R1 to L-R2, L-R2 to L-R3, and L-R2, respectively. It is a figure showing the state where R3 was moved to the position of L-R4.
[0032]
Even if the data is moved in this way, as shown in FIG. 6C, the original arrangement of one RGB pixel has not changed. The moving amount of the image by this moving method is, as shown by reference numeral 602 in FIG. 6C, a movement of 1/3 pixel, which is smaller than the moving of one pixel (603 in FIG. 6B). Become. Therefore, finer adjustment of the parallax amount is possible than when the image data is moved in data units of one pixel.
[0033]
Similarly, focusing on two dots of R data and G data, as shown in FIG. 7C, L-R1, L-G1 is L-R2, L-G2 is L-R2, and L-G2 is L-G2. By moving to -R3 and L-G3, respectively, the amount of adjustment can be reduced. This movement corresponds to a movement of 2/3 pixels (reference numeral 703 in FIG. 7). Even in this case, finer adjustment of the amount of parallax is possible than when the image is moved in data units of one pixel.
[0034]
In the present embodiment, an example of movement of R data and movement of R data and G data has been described. However, the present invention is not limited to only a combination of the data. Further, the image data may be moved by combining the movement in units of color data and the movement in units of one pixel.
[0035]
When adjusting only one of the left and right images when adjusting the amount of parallax by the method described above, the center of the stereoscopic image viewed by the user is also shifted in the direction in which the stereoscopic image is moved. Is difficult. Therefore, when calculating the parallax amount adjustment information, the parallax amount adjustment information calculation unit 3 may adjust the left-eye image and the right-eye image to move together in conjunction with each other. For example, if the adjustment of the amount of parallax by the user is four pixels, instead of moving the left-eye image by four pixels, the left-eye image is two pixels and the right-eye image is the left-eye image. A method of adjusting by moving two pixels in the opposite direction can also be used. As described above, by moving the left-eye image and the right-eye image in the opposite directions by the same amount, the amount of parallax can be adjusted without shifting the center of the stereoscopic display before and after the movement.
[0036]
When the adjustment amount of the parallax amount by the user is for even-numbered pixels, it is possible to adjust so that the center of the stereoscopic display is not shifted. However, when the adjustment amount is an odd-numbered pixel, the left-eye image and the right-eye image cannot be moved by the same amount, and the center of the stereoscopic display is shifted to the left or right.
[0037]
In such a case, the parallax amount adjustment information calculation unit 3 calculates the amount by which the left-eye image and the right-eye image are moved so that the shift of the center of the stereoscopic display is minimized. For example, when the user requests the parallax adjustment for five pixels, if the unit for moving the image is one pixel, the left-eye image is three pixels and the right-eye image is two pixels. Move in the opposite direction. As described above, if it is possible to move each dot of RGB, the image for the left eye is moved by 2+ (2/3) pixels, and the image for the right eye is moved by 2+ (1/3) pixels.
[0038]
As described above, even if the shift of the center of the stereoscopic display is minimized, the shift of the center becomes large when the adjustment amount is repeatedly moved by an odd number of pixels. Therefore, when the movement is repeated, the parallax amount adjustment information calculation unit 3 calculates and stores the information of the shift, and based on this storage, the image for the left eye and the right image are shifted so that the shift of the center of the stereoscopic display is reduced. The moving amount of the eye image is determined. Preferably, the adjustment is performed so that the shift of the center of the stereoscopic display is minimized.
[0039]
FIG. 8A shows an initial state of the left-eye image and the right-eye image. Reference numeral 801 denotes a display area for displaying each image on the display unit 6, and reference numeral 802 denotes the center of the display area 801. When the image data of FIG. 8A is stereoscopically viewed, the center of the stereoscopic display and the center (802) of the display area are the same. FIG. 8B shows a state in which the user first moves the left-eye image data by one pixel to the right from the state of FIG. 8A. The parallax adjustment information calculation unit 3 calculates the amount of movement as one pixel from the input of the parallax adjustment from the user, and notifies the stereoscopic image processing unit 1 of the amount of movement. For example, it is notified as vector information indicating that "the image for the left eye is moved rightward by one pixel" (reference numeral 803 in FIG. 8B).
[0040]
The stereoscopic image processing unit 1 to which the vector information is notified generates a moved image as shown in FIG. 8B. From this information, the parallax adjustment amount information calculation unit 3 calculates and stores the deviation of the center of the stereoscopic display from the center of the display area. In this example, since only one of the images is moved, the center of the stereoscopic display is the position indicated by the reference numeral 804 in FIG. 8B. Therefore, the displacement information calculated by the parallax amount adjustment information calculation unit 3 is an arrow (vector) indicated by reference numeral 805 in FIG. 8B.
[0041]
Next, when the user adjusts the amount of parallax again, the parallax amount adjustment information calculation unit 3 calculates the amount of parallax adjustment information so as to reduce the amount of deviation by using the deviation information at the time of the previous movement. It is preferable to make an adjustment so as to minimize the deviation. When it is calculated that the left-eye image data is further moved to the right by three pixels as the parallax amount adjustment information, the left-eye image is shifted so as to eliminate the shift of reference numeral 805 in FIG. To the stereoscopic image processing unit 1 so as to move the image to the right by one pixel and the image for the right eye to the left by one pixel. FIG. 8C shows a state of movement based on this notification. Arrows (vectors) indicated by reference numerals 806 and 807 in FIG. 8C correspond to the amounts of moving the respective images. The line indicated by reference numeral 808 in FIG. 8C is the center of the stereoscopic display after the movement, and it can be seen that the deviation from the center of the display area 801 has been eliminated.
[0042]
In the present embodiment, the method of adjusting the displacement based on the stored displacement information has been described as an example, but the method of reducing the displacement of the center of the stereoscopic display after the user adjusts the amount of parallax, and preferably the method of minimizing the displacement. If so, another method may be used. For example, the user uses a device that adjusts parallax on a pixel-by-pixel basis, stores the moved image and the direction, and, in a subsequent operation, moves an image that was not a moving target in the previous operation as a moving target and alternately moves the image. Alternatively, a method of eliminating the deviation may be used. The image data adjusted in the parallax amount by the stereoscopic image processing unit 1 in this manner is output to the display area determination unit 4.
[0043]
The input image is usually created in consideration of good display on the display unit 6 without adjusting the amount of parallax. Therefore, unless the amount of parallax is adjusted, the input image is generated from the input image. The rectangular image has the same size as the display area of the display unit 6. However, the strip-shaped image generated by adjusting the amount of parallax has a size larger than the display area of the display unit 6. Therefore, the display area determination unit 4 determines which area of the strip image is to be displayed on the display unit 6. As a criterion for determining this area, for example, a method of shifting the center of stereoscopic display from the center of the display as much as possible, and a method of extracting and displaying only a region that can be stereoscopically viewed can be considered.
[0044]
With reference to FIG. 9 and FIG. 10, a process when the input image data is moved will be described. FIG. 9A shows input image data. Reference numeral 901 in FIG. 9A indicates left-eye stereoscopic image data, and reference numeral 902 indicates right-eye stereoscopic image data. It is assumed that the size of the strip image generated from the input image data is the same as the display area of the display unit 6 in FIG. Reference numerals 903, 905, and 909 in FIGS. 9A to 9C are arrows indicating the width of the display area corresponding to each of the left-eye stereoscopic image data and the right-eye stereoscopic image data.
[0045]
FIG. 9B illustrates an example in which the stereoscopic image data for the right eye is moved by a vector indicated by reference numeral 906. Reference numeral 904 denotes an area where there is no image data to be displayed, reference numeral 911 denotes the center of the display area, reference numeral 912 denotes the center of stereoscopic display, and reference numeral 913 denotes a shift amount from the reference numeral 911 to the reference numeral 912. Is shown. As shown in FIG. 9 (B), the display area determination unit 4 (FIG. 1) changes the display area of the image data and extracts an area that does not change the center of the stereoscopic display, thereby providing a good stereoscopic image. Images can be played.
[0046]
For example, a range indicated by an arrow 1001 in FIG. 10 is set as a display area. As shown in FIG. 10, areas denoted by reference numerals 1004, 1005, 1006, and 1007 are not used for display. Of these, pixels are present in the areas denoted by reference numerals 1006 and 1005, but are cut off from the display. An area 1002 and an area 1003 are display areas, but have no image data, and thus are parts to which data is to be given by image interpolation processing described later. By selecting the display area in this way, the center of the display position and the center of the stereoscopic display can be set to coincide with or be close to each other.
[0047]
As shown in FIGS. 9 and 10, when the image is moved, a display area without image data is generated, such as an area 904 in FIG. 9A and areas 1002 and 1003 in FIG. . Therefore, in order to reproduce such an area better, interpolation processing of image data is performed. In the method according to the present embodiment, it is assumed that a display that can partially switch between the two-dimensional display mode and the three-dimensional display mode (stereoscopic display) is used for the display unit 6 (FIG. 1), and the display unit 6 (FIG. Interpolation processing that can be displayed is performed. This will be described with reference to FIG.
[0048]
Switching of the display mode is instructed by the reference numerals 1103 and 1104 reproduced by one eye, for example, by eliminating a part of the barrier (reference numerals 1101 and 1102) shown in FIG. This is realized when the pixels to be reproduced can be reproduced by both eyes as shown in FIG. However, if the display mode is partially switched, the switching means is not limited to the method shown in FIG.
[0049]
As described above, in the image data on which the parallax amount has been adjusted by the user, an area to be displayed is set by the display area determination unit 4, and an area without pixel data exists in the set display area. It is determined whether or not to do. Information on this area is output to the image interpolation unit 5 together with the image data. In order to obtain image data that is favorably reproduced in the two-dimensional display mode, the image interpolating unit 5 generates pixel data by performing an interpolation process based on data of peripheral pixels for pixels having no image data.
[0050]
With reference to FIG. 9, an example of processing in the image interpolation unit 5 will be described. As described above, the area indicated by reference numeral 904 in FIG. 9B is an area where interpolation processing is required due to movement. The display area determination unit 4 determines that there is no image data to be displayed in the area indicated by reference numeral 904, and instructs the display unit 6 to display the area 904 in the two-dimensional display mode. At the same time, it instructs the image interpolation unit 5 to generate interpolation data in this area 904.
[0051]
First, the image interpolation unit 5 copies the data of the corresponding part from the left-eye image to an area where data is missing due to the movement of the image and is also a display target. In the example shown in FIG. 9C, the data in the area 907 is copied to the area 908. Actually, in the state shown in FIG. 9C, the left-eye stereoscopic image data and the right-eye stereoscopic image data are arranged in a strip shape. FIG. 12A is an enlarged view of the vicinity of the region 910 in FIG. 9C. As shown in FIG. 12A, a tree portion is shown in black and a background portion is shown in white. The image data for stereoscopic vision converted into a strip shape including this portion is as shown in FIG.
[0052]
The shaded portion (the region indicated by reference numeral 1201) in FIG. 12B is a portion having no pixel data to be displayed because the right-eye stereoscopic image data has moved. FIG. 12C is a diagram illustrating a state in which pixel data is copied from the corresponding portion of the left-eye stereoscopic image data to the right-eye stereoscopic image data as described above. FIG. 12C shows that the image data of L1 is copied to R2 and the image data of L2 is copied to R3. By performing such a process, when the corresponding region is displayed, the region is better observed than in FIG. 12B, but the roughness is actually conspicuous. Therefore, the image interpolating unit 5 generates an interpolated image as shown in FIG. For example, a two-tap low-pass filter is applied in the horizontal direction. For example, the data of the R5 image area (reference numeral 1204) is calculated from L4 (reference numeral 1203) and L5 (reference numeral 1202) based on the formula (L4 + L5) / 2. For example, pixel data of another area where there is no pixel data including the pixel data of the column indicated by reference numeral 1205 is similarly calculated based on the adjacent left-eye image data.
[0053]
The interpolation method is not limited to the method using the average value as described above. For example, the interpolation may be performed using a low-pass filter having three or more taps. In addition, in the area where interpolation is performed, horizontal aliasing or blurring due to a filter may occur due to the interpolation processing. Therefore, a good display may be obtained by using a sharp filter, a noise removal filter, or the like. . The interpolated stereoscopic image data generated in this way is output to the display unit 6. By generating the interpolated image data using the low-pass filter as described above, a good image suitable for the two-dimensional display mode can be obtained on the display unit 6. In FIG. 10, display areas 1002 and 1003 in which no pixel data exists are generated. These areas may be interpolated by the same method as described above.
[0054]
As described above, the display area determination unit 4 in FIG. 1 determines the presence or absence of pixel data in the area used for actual display, and determines the area without pixel data as the area to be interpolated in the image interpolation unit 5. . By limiting the area to be interpolated to the display area in this way, the load of the interpolation processing can be reduced.
[0055]
Further, in the present embodiment, an example of moving an image in pixel units has been described. However, as described above, similar processing for area determination processing and interpolation processing can also be performed for an example of moving three primary color data in RGB data units. Needless to say, this is possible. In addition, although the technology for partially switching the interpolated area in the display device to the two-dimensional display mode has been described, the above method is applied to a three-dimensional display device having no function of partially switching to the two-dimensional display mode. It is also possible. In this case, the interpolated area is displayed in the stereoscopic display mode as before.
[0056]
As described above, according to the stereoscopic image display technique according to the first embodiment of the present invention, in a stereoscopic image in which the amount of parallax is adjusted, an interpolated image is created and displayed for an area where stereoscopic display is difficult. Thus, it is possible to display a good three-dimensional image with less discomfort without reducing the display area.
[0057]
Next, a stereoscopic image display technique according to a second embodiment of the present invention will be described. In the three-dimensional image display technology according to the first embodiment, a technology has been described in which interpolation image data is created for a display region where no pixel data exists due to adjustment of the amount of parallax by the user, and the region is displayed two-dimensionally. The three-dimensional image display technique according to the present embodiment is a technique for performing favorable three-dimensional display by displaying other image data in a display area where no pixel data exists.
[0058]
FIG. 13 is a block diagram illustrating a configuration example of the stereoscopic image display device according to the present embodiment. The difference from the stereoscopic image display device shown in FIG. 1 is that an image data generation unit 7 is provided instead of the image interpolation unit 5 (FIG. 1). For example, the image data generation unit 7 adds new image data to the region 1002 of the left-eye image data corresponding to the region 1002 and the region of the right-eye image data corresponding to the region 1003 and the region 1003 in FIG. insert.
[0059]
FIG. 14 is a diagram showing how the image data generation unit 7 (FIG. 13) inserts new image data. In FIG. 14A, reference numerals 1401 and 1402 denote portions where no pixel data exists, and reference numerals 1403 and 1404 denote portions where there is no paired data for stereoscopic display. The image data generator 7 combines these areas (the areas indicated by arrows 1406 and 1407 in FIG. 14A) with new images 1411a and 1411b as shown in FIG. 14B. Then, an image shown in FIG. The area indicated by the arrow 1405 indicates a display area.
As described above, by newly generating and displaying image data in a region where good stereoscopic vision cannot be performed, the user can reproduce only a portion of a good stereoscopic image.
[0060]
The new image used in the present embodiment may be an image for the two-dimensional display mode or an image for the three-dimensional display mode. When an image for the two-dimensional display mode is used, the display mode may be partially switched to the two-dimensional display mode. However, when there is no such function, reproduction is performed in the three-dimensional display mode. You may.
According to the stereoscopic image display technique according to the present embodiment, it is possible to display a good stereoscopic image with less discomfort by generating a new image in an area where stereoscopic display is difficult.
[0061]
Next, a stereoscopic image display technique according to a third embodiment of the present invention will be described with reference to the drawings. The stereoscopic image display technology according to the first and second embodiments is a stereoscopic image display process when the user adjusts the amount of parallax of the stereoscopic image and adjusts the amount of parallax. By the way, when a plurality of stereoscopic images are repeatedly displayed, if the operation of adjusting the amount of parallax is performed again each time, the processing becomes complicated for the user. Therefore, in order to avoid this, in the stereoscopic image display technique according to the present embodiment, the parallax amount adjusted by the user is stored for each stereoscopic image.
[0062]
Hereinafter, the parallax amount adjustment information is referred to as a shift vector. This shift vector is, for example, a vector indicated by reference numeral 906 in FIG. 9B. The content of the shift vector may be in any format as long as it can represent the adjusted amount of parallax. For example, a state in which a stereoscopic image is captured, that is, a state in which the parallax amount is not adjusted by the user is 0, a movement of a specific image in a specific direction is +, and a movement of the specific image in the opposite direction is-. The adjustment amount (the number of dots or the number of pixels) may be indicated. When the size of the shift vector is represented by the number of pixels, the amount of parallax changes when the size of the display for observing the stereoscopic image changes, so that the appearance (protruding distance or the like) changes. Therefore, the projection distance itself of the stereoscopic image may be recorded as a shift vector in units of, for example, cm so that the same state can be observed even when the size of the display is changed. The pop-out distance is represented, for example, with a state in which the amount of parallax is not adjusted as a reference (0). An example of calculating the pop-out distance will be described with reference to FIG.
[0063]
As shown in FIG. 25A, the distance between the left and right eyes of the observer is represented by e, and the distance between the observer and the display is represented by L. The state of FIG. 25A is a state in which the pixel observed by the right eye and the pixel observed by the left eye are observed through the same slit (P1). This pixel is then observed on the display.
[0064]
Next, the image observed by the right eye is moved by one pixel to the left in the drawing. This state is shown in FIG. The pixel existing at P1 moves to the position of P2. At this time, the pixel observed with the left eye and the pixel observed with the right eye are observed through different slits, forming an image at the position of S1, and a three-dimensional effect is generated. The distance d from the display at this time is the pop-out distance.
[0065]
Here, as shown in FIG. 25B, the moving distance from P1 to P2 is w. w is a value that depends on the device (display). Further, the distance L between the display and the observer also depends on the display in the case of the parallax barrier method or the lenticular method. Therefore, in order to calculate the pop-out amount, it is necessary to know the distance L and the moving distance w between the display and the observer. The distance e between the left and right eyes of the observer is considered to be substantially constant. Based on these assumptions, the pop-out distance d is obtained by the following equation.
[0066]
e: (Ld) = w: d (1)
Equation (2) is obtained from equation (1).
d = (w × L) / (e + w) (2)
[0067]
Here, the jumping distance d and the moving distance w take positive and negative values. When the pop-out distance d takes a positive value, it indicates that the user appears to jump out of the state before the adjustment of the parallax amount, and when the pop-out distance d takes a negative value, the user does not adjust the parallax amount before the adjustment. Indicates that you can see from the state to the back. The moving distance w has a positive value when the image observed with the right eye is moved to the left, and has a negative value when the image observed with the right eye is moved to the right. In FIG. 25B, since w and d both take positive values, they appear to jump out by the distance of d. Conversely, FIG. 25C is a diagram in which the image observed by the right eye is shifted to the right by one pixel from FIG. 25A. The pixel existing at P1 moves to the position of P3 and forms an image at the position of S2. At this time, since the moving distance w has a negative value, the pop-out distance d has a negative value according to the above equation (2). Therefore, in FIG. 25C, an image can be seen in the back as compared with FIG. 25A. When the image to be observed by the left eye is moved, the positive / negative value of the moving distance w is opposite to that when the image to be observed by the right eye is moved. In the stereoscopic image display technology according to the first embodiment of the present invention, the display area is automatically set. However, when the user sets the display area arbitrarily, the display area set in addition to the shift vector is set. The area information (display area information) is recorded and used for the next display. The display area information, for example, focuses on the image for the right eye, and indicates how much the left end of the image for the right eye has moved from the left end of the initial display area.
[0068]
In the stereoscopic image display technology according to the first embodiment of the present invention, the display area is automatically set. However, when the user sets the display area arbitrarily, the display area set in addition to the shift vector is set. The area information (display area information) is recorded and used for the next display. The display area information, for example, focuses on the image for the right eye, and indicates how much the left end of the image for the right eye has moved from the left end of the initial display area.
[0069]
The stereoscopic image display technique based on the above principle will be described more specifically. FIG. 15 is a functional block diagram illustrating a configuration example of the stereoscopic image display device according to the present embodiment. As illustrated in FIG. 15, the stereoscopic image display device according to the present embodiment includes a stereoscopic image processing unit 1 that performs image processing so that an input image can be stereoscopically displayed, similarly to the stereoscopic image display device illustrated in FIG. A user input unit 2 for inputting from a user, a parallax amount adjustment information calculation unit 3 for calculating parallax amount adjustment information from a user input, and an area to be actually displayed on a display from an image-processed stereoscopic image. It includes a display area determination unit 4 for determination, an image interpolation unit 5 for generating interpolation data in a specific area, and a display unit 6 for displaying a determined area in the generated image. The difference from FIG. 1 is that a stereoscopic information recording unit 8 is provided. The basic operations of the other parts are the same as those described in the embodiments.
[0070]
In FIG. 15, the disparity amount adjustment information (shift vector) calculated by the disparity amount adjustment information calculation unit 3 is output to the stereoscopic image processing unit 1 and also to the stereoscopic information recording unit 8 (reference numeral 1501). . The display area determination unit 4 determines the display area, further changes the display area when the user arbitrarily sets the display area, and outputs the changed display area information to the stereoscopic information recording unit 8 ( Reference numeral 1502). The stereoscopic information recording unit 8 records (stores) the shift vector, and also records the display area information as needed.
[0071]
Next, an example of a method of recording the shift vector and the display area information will be described. In the present embodiment, an area for recording a shift vector and display area information is provided in stereoscopic image data. Normally, image data is provided with an area for storing information for managing the size of the image, the reproduction time, and the like. FIGS. 16A and 16B are diagrams showing examples of the data structure of image data. As shown in FIG. 16A, the image data for stereoscopic vision includes, for example, a management information area for managing image information such as a reproduction time and an image size, and an image for recording image data for each of the left and right eyes. And a data area.
[0072]
As shown in FIG. 16A, the image information 1601 describes information about the entire image such as the size of the image and the reproduction time of the moving image in the case of a moving image. Describes information necessary for decoding each image data (for example, information that the MPEG-4 technology is used as an encoding technology). Further, as shown in FIG. 16B, an area 1604 for recording information for stereoscopic image data (stereoscopic information) is provided in this management area, and shift vectors and display area positions are recorded.
[0073]
At the time of recording, a header indicating the presence of stereoscopic information is necessary so that the device can correctly read such information. For this purpose, an area for recording the stereoscopic (3D) image identification information 1605 is provided at the head of the stereoscopic vision information 1604. The stereoscopic image identification information 1605 indicates the presence of stereoscopic information and indicates that the following image data is stereoscopic image data. The stereoscopic image identification information 1605 may be a flag encoded with a fixed-length or variable-length code, but may be a specific symbol string or character string as long as it can be identified.
[0074]
In the stereoscopic information 1604, there is information on the shift vector and the display area position in addition to the stereoscopic image identification information 1605. For example, long stereoscopic vision places a burden on the eyes, so the continuous viewing time is limited. In such a case, the viewable time may be recorded together. Such information is referred to as stereoscopic control information and is recorded in an area indicated by reference numeral 1606. FIG. 16B shows a configuration example of the stereoscopic image identification information and the stereoscopic vision control information. As shown in FIG. 16B, stereoscopic vision control information 1606 is recorded following the stereoscopic image identification information 1605.
[0075]
The stereoscopic vision control information will be specifically described with reference to FIGS. The shift vector indicates the number of pixels to be moved. As shown in FIGS. 17A and 17B, the right-eye image data is shifted by 4 pixels in a direction in which the parallax is widened (right direction) by focusing on the right-eye image data. Is 4. The display area 1703 has moved from the original display area by two pixels in the horizontal direction (indicated by an arrow 1701) with the left end of the image as a reference position. This is represented by a vector 1705. The components of this vector are 2 in the horizontal direction and 0 in the vertical direction (1702), and are represented by (2, 0) as display area information in FIG.
Although the case where the stereoscopic vision control information is recorded in the management information area accompanying the image data has been described as an example, the stereoscopic vision control information may be recorded in a predetermined storage unit included in the display device. .
[0076]
FIG. 18 shows an example of shift vector and display area information stored in a predetermined recording area in the parallax amount information recording unit 8. As shown in FIG. 18, the parallax amount information recording unit includes information for identifying the stereoscopic image data by attaching file names such as contents A, B, and C, and stereoscopic control information (shift vector and display area). Information) is associated with a file name and tabulated. For example, in the content A, the shift vector (the number of pixels) is 4, and the display area information (the number of pixels) is (2, 0). In FIG. 18, the shift vector is represented as a one-dimensional vector.
[0077]
In the stereoscopic image display technology according to the present embodiment, the shift vector and the display area information are described by taking an example in which an integer pixel is a unit, but the present invention is not limited to this, and is not an integer. The pixel unit can be used as information for displaying the shift vector and the display area information.
[0078]
In the stereoscopic image display technique according to the present embodiment, the shift vector may be the number of pixels for moving the right-eye image / left-eye image in the horizontal direction or the amount of change in the stereoscopic projection distance. Therefore, information or a flag indicating whether the shift vector is represented by the number of pixels to be moved or the projection distance of stereoscopic vision may be recorded in the stereoscopic vision control information. Also, an example has been described in which both the shift vector and the display area information are recorded as the stereoscopic vision control information. However, only the shift vector is recorded, and a predetermined area that is determined in advance or an area that is automatically obtained may be used as the display area. Good. Alternatively, only the display area information may be recorded. Further, other information for stereoscopic viewing may be recorded.
[0079]
Further, in the present embodiment, an example has been described in which the stereoscopic image identification information and the stereoscopic vision control information are recorded only at one position at the head of the image file. However, as long as extraction is possible, it may be recorded in any area of the image file, or may be recorded in the image data. For example, when an image is encoded by MPEG-4, the image data includes encoded data and information (header information) for decoding the encoded data. The header information has an area (user area) that can be freely used by the user, and stereoscopic image identification information and stereoscopic vision control information may be recorded in this user area. Further, a plurality of pieces of stereoscopic image identification information and stereoscopic vision control information may exist in the image data. Thereby, for example, the amount of parallax can be changed in the middle of the image data.
[0080]
Next, a procedure for reproducing image data by the stereoscopic image display technique according to the third embodiment will be described. Here, it is assumed that the stereoscopic image identification information and the stereoscopic vision control information shown in FIG. 17 are recorded in the image data.
[0081]
FIG. 19 is a functional block diagram illustrating a configuration example of a stereoscopic image display device that displays image data for stereoscopic vision to which stereoscopic vision control information is added according to the present embodiment. In addition to the configuration shown in FIG. 1, a stereoscopic information reading unit 9 is provided. The stereoscopic information reading unit 9 reads the stereoscopic image identification information included in the input image data. When the stereoscopic image identification information confirms that the image is a stereoscopic image, the stereoscopic information reading unit 9 reads stereoscopic control information (shift vector and display area information) and encodes the stereoscopic information. If there is, decrypt it. If the shift vector exists in the stereoscopic vision control information, the shift vector is output to the parallax amount adjustment information calculation unit 3, and if the shift vector exists, it is output to the display area determination unit 4.
[0082]
The parallax adjustment information calculating unit 3 calculates parallax adjustment information using the shift vector input from the stereoscopic information reading unit 9 and the data for adjusting the parallax input from the user input unit 2. If there is no parallax adjustment by user input, parallax adjustment information is calculated from only the shift vector. The stereoscopic image processing unit 1 generates stereoscopic display image data using the parallax amount adjustment information.
[0083]
Further, the display area determination unit 4 determines a new display area according to the display area information and the parallax adjustment information input from the stereoscopic information reading unit 9. If there is no parallax adjustment by user input, the parallax adjustment information from the stereoscopic information reading unit 9 is used as it is.
[0084]
As described above, when the user repeatedly reproduces the same stereoscopic image data, the adjustment performed by the user during the previous reproduction is stored as the stereoscopic control information, and the stereoscopic control information is used during the reproduction. Thus, it is possible to perform the reproduction in the same manner as the previous reproduction without having to perform the adjustment of the parallax amount each time the reproduction is performed.
[0085]
As described above, according to the third embodiment of the present invention, by recording the shift vector related to the image data, that is, the content, it is not necessary to adjust the amount of parallax again when viewing the same content. An appropriate shift vector can be recorded every time.
[0086]
Next, a stereoscopic image display technique according to a fourth embodiment of the present invention will be described with reference to the drawings. Appropriate adjustment of the amount of parallax is effective for the user because reproduction can be performed in a better state. However, if the adjustment method is erroneous, stereoscopic vision may not be properly performed. In the present embodiment, in order to avoid this, the maximum value of the shift vector (maximum shift vector) is set, and the amount of parallax that can be adjusted by the user is limited. The maximum shift vector may be included in the image data management area (stereoscopic control information in FIG. 16B) or may be recorded in a predetermined recording area of the display device according to the present embodiment. Good.
[0087]
Description examples of the maximum shift vector are shown in FIGS. 20A and 20B, the maximum shift vector is described in the same manner as the above-described shift vector. That is, it is expressed as the number of dots to be moved or the pop-out distance. FIG. 20A shows an example in which image data is recorded, and the maximum shift vector is set to 10. FIG. 20B is an example in which the maximum shift vector value is determined for each content in a table format in a predetermined storage unit of the display device. For example, the maximum shift vector is 10 for the content A. The examples of FIGS. 20A and 20B show an example including not only the maximum shift vector but also information for other stereoscopic views, but it is not always necessary to include the information in the same region as those. Absent.
[0088]
FIG. 21 shows a configuration example of a stereoscopic image data reproducing apparatus according to the present embodiment. As shown in FIG. 21, the configuration has a configuration in which the stereoscopic information reading unit 9 shown in FIG. 19 is replaced with a maximum shift vector reading unit 10. The maximum shift vector reading unit 10 reads the maximum shift vector from the stereoscopic control information. If the stereoscopic control information exists in the same file as the input image data, the information of the maximum shift vector is extracted from the file. When the shift vector is recorded in the storage unit independent of the file, the shift vector corresponding to the input image data is read from the storage unit, and the maximum shift vector information is extracted.
[0089]
The extracted maximum shift vector is output to the parallax amount adjustment information calculation unit 3. The parallax amount adjustment information calculation unit 3 determines whether the shift vector calculated from the user input is within the range of the maximum shift vector, and if it is within the range, sends the calculated shift vector to the stereoscopic image processing unit 1 as it is. If the shift vector is larger than the maximum shift vector, the maximum shift vector is output as a shift vector. Accordingly, display using a large shift vector that makes it difficult to perform stereoscopic viewing is not performed, so that favorable stereoscopic display can be performed. The maximum shift vector can define both a positive value and a negative value, and the absolute value of the positive value and the absolute value of the negative value do not necessarily have to be equal. For example, when, for example, (−7) is defined together with (+10) as the maximum shift vector (one-dimensional vector in this example) for the content A in FIG. 20B, the shift vector calculated from the user input is The value is limited to a value between (−7) and (+10).
[0090]
If the shift vector calculated from the user input falls outside the range of the maximum shift vector, the shift vector may not be used. For example, when the shift vector by the user input is calculated as 20 at the time of reproducing the content A of FIG. 20B, the shift vector used for display is kept at the shift vector value “4”. If the display mode can be switched on the display unit 6 and the shift vector by the user input exceeds the maximum shift vector, the entire image may be displayed in the two-dimensional display mode.
[0091]
In the present embodiment, the maximum shift vector reading unit 10 is provided to extract the maximum shift vector. However, a function capable of executing the same processing is added to the stereoscopic information reading unit 9 shown in FIG. It is good.
[0092]
According to the fourth embodiment of the present invention, by setting the maximum shift vector, it is possible to prohibit the adjustment of the amount of parallax to the extent that stereoscopic viewing becomes difficult, which may impair the intention of the content creator. Therefore, a good stereoscopic image can be generated.
[0093]
Next, a three-dimensional image display device according to a fifth embodiment of the present invention will be described with reference to the drawings. When reproducing an image using stereoscopic image data in which a shift vector and a display area are set, or when adjusting the amount of parallax of a stereoscopic image during image reproduction, image processing such as image enlargement / reduction When the stereoscopic image data is reproduced by using both, the shift vector may be different from the number of pixels to be actually shifted. In such a case, it is preferable to perform the enlargement / reduction processing on the shift vector together with the image enlargement / reduction processing. This situation will be described with reference to FIGS.
[0094]
The image data shown in FIG. 22A is used as input image data, and the image data shown in FIG. 22B is used as image data that has been subjected to parallax adjustment (one-pixel shift). On the other hand, the image data shown in FIG. 22C is an image obtained by subjecting the input image data to enlargement processing (two-fold enlargement in the horizontal direction), and the image data shown in FIG. This is a parallax amount adjusted (one pixel shift) of the image data shown in (C). When the shift vector is expressed in units of pixels, the adjustment as shown in FIG. 22D is performed on the image after the enlargement as described above. Will be different. Therefore, the technology according to the present embodiment also enlarges / reduces stereoscopic control information such as a shift vector in accordance with enlargement / reduction of an image.
[0095]
FIG. 23 is a functional block diagram showing a configuration example of the stereoscopic image display device according to the present embodiment. FIG. 23 includes an enlargement / reduction processing unit 11 for performing image enlargement / reduction processing and a stereoscopic control information conversion unit 12 in addition to the configuration shown in FIG. 19. When the enlargement / reduction is performed, the enlargement / reduction processing unit 11 performs enlargement / reduction processing of the input image data, and the enlarged / reduced image data is output to the image processing unit 1. At that time, the enlargement / reduction processing unit 11 outputs the enlargement / reduction ratio to the stereoscopic vision control information conversion unit 12.
[0096]
The stereoscopic vision control information accompanying the image data is read by the stereoscopic vision information reading unit 9. The stereoscopic information reading unit 9 in the present embodiment reads the maximum shift vector in addition to the shift vector and the display area information. The read stereoscopic control information is output to the parallax adjustment information calculation unit 3 and the display area determination unit 4.
[0097]
The parallax amount adjustment information calculated by the parallax amount adjustment information calculation unit 3 is converted by the stereoscopic vision control information conversion unit 12 by multiplying the enlargement / reduction ratio of the input image data. For example, when the parallax adjustment information is a one-dimensional vector (−3) and the enlargement / reduction ratio is 2, the parallax adjustment information is converted to (−6). Similarly, the display area information generated by the display area determination unit 4 is converted by the stereoscopic control information conversion unit 12 according to the enlargement / reduction ratio.
[0098]
When the stereoscopic image display technology according to the present embodiment is used, the parallax amount adjustment according to the enlargement / reduction ratio is performed as shown in FIG. 22E instead of FIG. Can be observed.
[0099]
According to the fifth embodiment of the present invention, when the stereoscopic image is enlarged or reduced, the shift vector for adjusting the amount of parallax is corrected in accordance with the enlargement or reduction ratio. Even when the stereoscopic image data to be adjusted has been subjected to enlargement / reduction processing, a good stereoscopic image can be generated.
[0100]
Next, a stereoscopic image display technique according to a sixth embodiment of the present invention will be described with reference to the drawings. The technology in each of the above embodiments is a technology for storing stereoscopic vision information for stereoscopic image data stored in a file. When transmitting image data for stereoscopic viewing as broadcast content such as BS broadcasts and terrestrial digital broadcasts, it is necessary to store and transmit stereoscopic information in a method suitable for broadcasting. For example, it is necessary to be able to acquire such stereoscopic information even when the user changes channels and starts receiving new broadcast content.
[0101]
In the case of BS broadcasting, terrestrial digital broadcasting, and the like, program arrangement information for managing what kind of program is being broadcast is multiplexed with the content as shown in FIG. . The broadcast contents include data of a plurality of contents, and information indicating these contents is program arrangement information. The program arrangement information includes information (PMT: Program Map Table) for separating / identifying video signals and audio signals constituting each content from broadcast contents and program guide information (EIT: Event) describing the content of each content. Information Table).
[0102]
The program arrangement information is repeatedly multiplexed and transmitted in the broadcast content so that the channel switching in the receiver can be performed at any time. In the stereoscopic image display technique according to the present embodiment, stereoscopic information is included in the program arrangement information.
[0103]
FIG. 24B shows an example in which stereoscopic information is incorporated into program arrangement information. Here, it is assumed that the stereoscopic information includes the stereoscopic image identification information and the stereoscopic control information as described in the third embodiment of the present invention. That is, the stereoscopic image identification information indicates that the content is a stereoscopic image, and the stereoscopic vision control information includes information such as shift vector, display area information, and maximum shift vector.
[0104]
The receiver uses the stereoscopic image identification information from the broadcast content as described above, determines that the content is a stereoscopic image, and extracts stereoscopic vision control information. Then, as in the other embodiments described above, a shift vector or the like is obtained from the stereoscopic vision control information, and stereoscopic image data is generated and displayed.
[0105]
Although the information such as the shift vector and the area display position in the stereoscopic vision control information has been described as recording data when the user changes the stereoscopic effect of the image, the broadcast side may set these information. . For example, at the start of broadcasting, the value of the shift vector may be set to 0, and the broadcasting station may set the shift vector to a value other than 0 on the broadcasting side later, either independently or at the request of the user. It is. Also, according to the content creator's intention, the maximum shift vector can be set on the broadcast side and included in the stereoscopic view control information, which can be used to limit parallax adjustment at the receiver.
[0106]
In addition, the stereoscopic image identification information and the stereoscopic vision control information have been described as being included in the program arrangement information. However, such information may be included not only in the program arrangement information but also in image data. For example, when the broadcast content is encoded by MPEG-4, the stereoscopic image identification information and the stereoscopic vision control information may be included in the user area as described above. Even in such a case, in order to allow the content to be viewed from the middle of the program, the stereoscopic image identification information and the stereoscopic vision control information must be periodically included in the image data. The information for decoding may include stereoscopic image identification information and stereoscopic vision control information.
[0107]
According to the sixth embodiment of the present invention, by including stereoscopic information in program arrangement information such as BS broadcast and terrestrial digital broadcast, even when broadcasting the content of a stereoscopic image, adjustment and display of the amount of parallax can be performed. It is possible to change the position and limit the adjustment of the amount of parallax.
[0108]
Next, a stereoscopic image display technique according to a seventh embodiment of the present invention will be described with reference to the drawings. In the third embodiment, an example is given in which the number of pixels to be moved in the horizontal direction is specified as a shift vector that is information for adjusting the amount of parallax. An example of specifying a distance, an example of specifying a moving direction and a distance of an image, and an example of specifying a parallax angle of a stereoscopic image are described below.
[0109]
As shown in FIG. 25, with respect to the image observed with the right eye, when the pixel located at P1 moves so as to be located at P2 or P3, the stereoscopic image looks like P1 on the display surface before the movement. After moving, the pixel that forms an image at a position S1 or S2 that is separated from the display surface by a distance | d | Here, | d | is a quantity that can be expressed in units of cm equal length. d itself has positive and negative values. For example, as shown in FIG. 25 (B), the image has a positive value if the image is farther from the display surface, and as shown in FIG. 25 (C), the image is deeper than the display surface. If they are far apart, a negative value is used.
[0110]
d is an amount measured or defined under a standard environment in which the characteristics and viewing conditions of the display are known, that is, a “standard viewing environment”. In the stereoscopic image display device, an adjustment value of the actually required parallax amount is calculated based on the d. By adopting such a configuration, the viewer's impression and influence received from the stereoscopic image displayed as a result of the parallax adjustment may be affected by a plurality of stereoscopic image display devices having various display unit characteristics and viewing conditions. Thus, it is possible to perform control so as to be substantially the same. Hereinafter, an example of a method of calculating the adjustment value of the parallax amount based on the d will be described.
[0111]
The state shown in FIG. 25B will be described as an example. As described above, when the right-eye image is horizontally moved so that the right-eye pixel P1 on the display surface comes to a position P2 which is horizontally separated by a distance w, the image is moved to the position P1 on the display surface before the movement. Form an image at a position S1 that is separated by a distance d from the display surface. At this time, d is given as in the aforementioned equation (2).
[0112]
Here, all the variables d, L, w, and e used in equation (2) are values in the standard observation environment. Similarly, in an arbitrary stereoscopic image display device, the relationship of the above equation (2) holds. However, d, L, and w, excluding e, usually have different values from the standard observation environment. Here, these variables in an arbitrary stereoscopic image display device are represented as d ′, L ′, w ′. That is, the following equation (2 ′) holds.
d ′ = (w ′ × L ′) / (e + w ′) (2 ′)
[0113]
e is constant regardless of observation conditions. Solving d ′ from equations (2) and (2 ′) yields the following equation (3). Here, W and W ′ represent the image display width W on the display in the standard observation environment and the image display width W ′ on the display in the arbitrary stereoscopic image display device, respectively, and these are (W: W ′ = w: w ′).
d ′ = L ′ × W ′ × d / (L × W + (W′−W) × d) (3)
[0114]
If d ′ is obtained as in the above equation (3), w ′ is obtained from the relationship of the above equation (2 ′). Since w ′ represents the moving distance of the image on the display surface, h ′ is obtained from the following equation (4), where h ′ is the number of pixels that need to be actually moved. Here, p ′ is a horizontal pixel pitch (distance between adjacent pixels) of the display.
h ′ = w ′ / p ′ (4)
[0115]
As described above, if the parameters such as L, W, and d in the standard observation environment are known, it is possible to calculate the adjustment value of the amount of parallax required in an arbitrary stereoscopic image display device.
[0116]
Among the parameters, L and W are parameters whose values are fixed if the standard observation environment is defined, and therefore need not be specified for each stereoscopic image data, and are stored in advance in each stereoscopic image display device. Just do it. Therefore, in order to adjust the amount of parallax, the parameter corresponding to d may be recorded or transmitted in association with the stereoscopic image data. When a parameter corresponding to d is recorded or transmitted, the parameter may be composed of two parameters, a parameter indicating the positive and negative directions of d and a parameter indicating the absolute value of the distance, or may include both parameters. It may be composed of only one parameter having positive and negative values.
[0117]
FIGS. 26A and 26B show examples of the configuration of the stereoscopic information including a parameter corresponding to d. In the example of FIG. 26A, d is represented by two parameters of the pop-out direction 2601 and the pop-out distance 2602, and in the example of FIG. ing. The parameter indicating d may be a parameter indicating a protruding position (or a retracted position) from the display surface as described above, or a direction perpendicular to the display surface of a pixel at an arbitrary reference position distant from the display surface. May be a parameter representing the amount of movement of.
[0118]
Note that there are two cases where d is a value for specifying the amount of protrusion to be adjusted and a value for specifying the allowable maximum value of the amount of protrusion. Therefore, when recording or transmitting the parameter corresponding to d, whether it is determined in advance, or in the stereoscopic vision control information, separately include a parameter indicating any of these values It is good. Alternatively, both may be included in the stereoscopic vision control information.
As another example of the information for adjusting the amount of parallax, an example in which a direction and a distance in which an image is moved is specified will be described below.
[0119]
As described above, assuming that (W: W '= w: w'), if the value of W is known in advance as the condition of the standard observation environment, then if w is given, then W is given. It is possible to calculate w 'in a stereoscopic image display device satisfying the condition'. Therefore, a parameter indicating w may be specified as information for adjusting the amount of parallax. Similar to d, the parameter indicating w may be composed of two parameters, a value indicating the positive and negative directions of w and a value indicating the absolute value of the distance. It may be composed of only one parameter. The direction of w at this time is positive when the image for the right eye is moved rightward, and negative when the image is moved leftward.
As yet another example of information for adjusting the amount of parallax, an example in which the angle of parallax of a stereoscopic image is specified will be described below.
[0120]
As shown in FIG. 25, when the pixel located at P1 moves so as to be located at P2 or P3, the angle which is the parallax θ with respect to the position at which the image is formed changes to θ1, θ2. If these angles are defined as values in the standard observation environment, it is possible to calculate the parallax angle θ ′ in an arbitrary stereoscopic image display device from the relationship between L and L ′, the relationship between W and W ′, and the like. . Therefore, a parameter indicating θn may be specified as information for adjusting the amount of parallax. The parameter indicating θn may be constituted by a parameter indicating the absolute value of the angle, such as θ1 or θ2, or θ, which is in a state without parallax as shown in FIG. It may be constituted by a parameter indicating a difference from these θ1 and θ2 (that is, a value representing θ1−θ and θ2−θ).
[0121]
Next, how the display area is determined as a result of the parallax adjustment will be described. As described above, when the image is moved to adjust the amount of parallax, the width of the image that can be displayed as a stereoscopic image changes, so that the display area is not uniquely determined. In the third embodiment, an example is described in which the display area is arbitrarily set by the user and recorded as display area information. An example in which flat display image selection information is used instead of the display area information will be described below.
[0122]
The plane display image selection information is information indicating which of a left-eye image and a right-eye image is used when displaying stereoscopic image data as a plane image. Since the stereoscopic image data includes the left-eye image and the right-eye image as described above, any of the images can be displayed as it is as a planar image. Of course, the display may be performed after the interpolation processing and the correction processing are added.
[0123]
The plane display image selection information is included in the stereoscopic vision control information as shown in FIG. The plane display image selection information may record a result set by the user similarly to the display area information, or may be included in the stereoscopic information in advance as described in the sixth embodiment. Is also good.
[0124]
In the three-dimensional image display device, the two-dimensional display image selection information is interpreted by the display area determination unit 4 and is used to determine the display area of the three-dimensional image on which the parallax amount has been adjusted. That is, the display area determination unit 4 adjusts the amount of parallax so as to display the image specified by the planar display image selection information in the stereoscopic vision control information, that is, the entire left-eye image or right-eye image. The display area of the displayed stereoscopic image is determined. This operation is applicable to the three-dimensional image display devices (FIGS. 1, 13, 15, 15, 19, 21, and 23) having the configurations described in all the embodiments.
[0125]
FIG. 27 shows a display state on the display unit 6 when the display area is determined as described above. FIG. 27A shows a state where the parallax amount adjustment is not performed, and FIGS. 27B and 27C show a state where the right-eye image is shifted rightward with respect to the left-eye image. is there. Due to the shift, the horizontal positions of the left-eye image and the right-eye image do not match as shown in the figure. An area where both the left-eye image and the right-eye image (binocular image) are present is an area where stereoscopic display can be performed as it is. An area in which only one of the left-eye image and the right-eye image (one-eye image) exists is an area in which stereoscopic display cannot be performed as it is. The one-eye image area may be a two-dimensional image by showing the same image to both eyes of the observer, or may be subjected to some image processing such as generating or interpolating the other one-eye image based on the one-eye image. May be a three-dimensional image. At this time, the display area is determined so as to match the position of the image specified by the flat display image selection information. FIG. 27 (b) shows the relationship between the position of the left and right images and the display area in the example where “left-eye image” is specified in the plane display image selection information, and FIG. 27 (c) shows the plane display image selection. The relationship between the positions of the left and right images and the display area in an example in which “image for the right eye” is specified in the information is shown.
[0126]
The following effects are obtained by determining the display area as described above. That is, as described above, when a two-dimensional image is displayed on the three-dimensional image display device, display is performed using only the left-eye image or the right-eye image in accordance with the two-dimensional image selection information. For example, when “left-eye image” is specified in the plane display image selection information, the entire left-eye image is displayed. At this time, even if the display mode is switched to the stereoscopic image display, the display area is similarly selected so that the entire left-eye image is displayed. The position of the image for the left eye appears to be fixed, and as a result, it is possible to reduce the discomfort of switching the display mode.
[0127]
In the three-dimensional image display device capable of switching between the three-dimensional display and the two-dimensional display as described above, the display unit 6 has a three-dimensional display mode and a two-dimensional display mode, and is a display device that can switch between the two modes. Switching between the three-dimensional display mode and the two-dimensional display mode may be performed by a user of the three-dimensional image display device by operating a button or the like, or may be such that stereoscopic information is repeatedly transmitted as described in the sixth embodiment. In the case, the stereoscopic control information may include a parameter for switching between stereoscopic display and planar display. In addition, the configuration is such that the stereoscopic display and the planar display are switched depending on the presence or absence of the stereoscopic image identification information, that is, the stereoscopic information itself. In addition, the display mode may be switched for the entire display screen, or may be switched in arbitrary display range units.
[0128]
Next, a stereoscopic image recording technique according to an eighth embodiment of the present invention will be described. In the third embodiment or the seventh embodiment of the present invention, an example in which information for adjusting the amount of parallax is specified is described. The stereoscopic image recording technology according to the present embodiment is one form of a recording method including such stereoscopic information, and is a recording method for recording a stereoscopic image and stereoscopic information on a digital video tape, and a recording apparatus therefor. .
[0129]
First, a track format of a digital video tape recorded by the stereoscopic image recording technique according to the present embodiment will be described. 2. Description of the Related Art A generally known digital VTR employs a method called helical scan. As shown in FIG. 28, in this method, data is recorded on discontinuous tracks on a tape. FIG. 28 shows this state. A plurality of tracks 2801 are formed on the tape 2800, and one stereoscopic image is also divided into a plurality of tracks 2801 and recorded. The running direction of the tape is from right to left (the direction of the arrow) in FIG. 28. The leftmost track is scanned upward from below, and then the track adjacent to the right is scanned upward from below.
[0130]
FIG. 29 is a diagram illustrating a configuration example of one track 2801, and is a diagram illustrating a configuration example of a track format of a digital VTR recorded by the stereoscopic image recording technique according to the present embodiment. The track 2801 includes an ITI (Insert and Track Information) area 2901 for ensuring post-recording, an audio recording area 2902 for recording audio-related data, an image recording area 2903 for recording image-related data, and a time code. And a subcode recording area 2904 in which the accompanying information is recorded. In the image recording area 2903, not only the stereoscopic image itself but also accompanying information related to the stereoscopic image can be recorded. Similarly, in the audio recording area 2902, not only the audio itself but also accompanying information related to the audio can be recorded. In addition, apart from these two, as described above, additional information can be recorded in the subcode recording area 2904 as well. In addition, there is a margin between the respective areas, and dubbing can be performed individually.
[0131]
FIG. 30 is an enlarged view of the image recording area 2903. An image recording area 2903 includes a preamble 3001 in which a synchronization pattern and the like are recorded, VAUX (Video AUXiary data) α3002 and VAUXβ3004 in which accompanying information about an image is recorded, and an image encoded data recording area in which image encoded data is recorded. 3003, an error correction code 3005, and a postamble 3006 having a function for gaining a margin. In the three-dimensional image recording technique according to the present embodiment, the area in which the accompanying information about the image is recorded is divided into two parts, VAUXα3002 and VAUXβ3004. These two areas are hereinafter collectively referred to as a VAUX area.
Although not shown, an AAUX (Audio AUXiary data) area is also provided in the audio recording area 2902 as an area for recording accompanying information on audio.
[0132]
Subsequently, the recording apparatus according to the present embodiment will be described with reference to FIG. FIG. 31 is a block diagram showing the configuration of the recording apparatus of the present embodiment. As shown in FIG. 31, the present recording device includes a stereoscopic image encoding unit 3101, an audio encoding unit 3102, an accompanying information encoding unit 3103, a multiplexing unit 3104, and a tape recording unit 3105.
[0133]
The stereoscopic image encoding unit 3101 receives stereoscopic image data as input. The stereoscopic image data is image data generated based on a plurality of images and capable of stereoscopic display, as described in the first embodiment and the like. The stereoscopic image encoding unit 3101 encodes the stereoscopic image data by a predetermined method, and outputs encoded stereoscopic image data.
[0134]
Speech encoding section 3102 receives speech data as input, encodes the speech data, and outputs encoded speech data.
The accompanying information encoding unit 3103 performs stereoscopic information on the stereoscopic image data, that is, stereoscopic image identification information for indicating a stereoscopic image, a shift vector for adjusting the amount of parallax, or a pop-out direction and distance, and display. Ancillary information including plane display image selection information for determining an area is encoded, and the associated information encoded data is output. Examples of the encoding method include conversion into a fixed-length numerical value corresponding to each piece of information.
[0135]
The multiplexing unit 3104 receives the stereoscopic image encoded data, the audio encoded data, and the accompanying information encoded data, multiplexes them into a format that can be recorded on a tape, and outputs tape recording data.
The tape recording unit 3105 records the data for tape recording on a tape as a recording medium according to the format described above.
[0136]
Subsequently, the multiplexing unit 3104 will be described in more detail with reference to FIG. As shown in FIG. 32, the multiplexing unit 3104 includes an additional information coded data distribution unit 3205, an image recording area data synthesizing unit 3201, an audio recording area data synthesizing unit 3202, and a subcode recording area data synthesizing unit. Unit 3203 and a track synthesizing unit 3204.
[0137]
The associated information coded data distribution unit 3205 receives the associated information coded data as input, determines which of the VAUX area, the AAUX area, and the subcode area to record, and distributes the data. In the present embodiment, the encoded data relating to the stereoscopic image identification information and the plane display image selection information is allocated to the VAUX area, and the encoded data relating to the shift vector and the pop-out direction / distance are allocated to the subcode area.
[0138]
The image recording area data combining section 3201 receives as input the stereoscopic image encoded data output from the stereoscopic image encoding section 3101 and the VAUX area associated information encoded data output from the associated information encoded data distribution section 3205. The encoded information and the stereoscopic image encoded data are combined so as to have the format shown in FIG. 30, and the image recording area data is output.
[0139]
The audio recording area data synthesizing unit 3202 receives as input the audio encoded data output from the audio image encoding unit 3102 and the AAUX area incidental information encoded data output from the incidental information encoded data distribution unit 3205, These are synthesized so as to have a predetermined format, and the data for the audio recording area is output.
[0140]
The sub-code recording area data synthesizing unit 3203 receives the sub-code area incidental information encoded data output from the incidental information encoded data distribution unit 3205, synthesizes them into a predetermined format, and records the sub-code. Outputs area data.
[0141]
The track synthesizing unit 3204 receives the data for the image recording area, the data for the audio recording area, and the data for the subcode recording area, and synthesizes them into the format shown in FIG. The recording data is output with a margin between each area added.
[0142]
In the present embodiment, the audio recording area, the image recording area, and the subcode recording area are recorded simultaneously. However, these need not always be recorded at the same time. Can be recorded first, and the subcode recording area can be recorded by post-recording. Alternatively, even if all are recorded at the same time, each area can be individually updated by post-recording.
[0143]
Information such as the shift vector and the pop-out direction / distance for adjusting the amount of parallax may not only be determined at the time of shooting but also determined at the editing stage after shooting according to the final content. There is. For example, there is a case where a default value is recorded at the time of shooting, and after the finish is confirmed at the editing stage, information indicating the shift vector and the pop-out direction / distance is dubbed. As the default value of the shift vector and the pop-out direction / distance, for example, all 0s, that is, the parallax amount is specified not to be adjusted. As an adjustment method at the editing stage, for example, similarly to the stereoscopic image display device described in the third embodiment, the stereoscopic image recording device performs image processing so that the input image can be further stereoscopically viewed. Means, means for receiving input from a user, means for calculating adjustment information of the amount of parallax from the input of the user, and an area to be actually displayed on a display from a stereoscopic image image-processed using the adjustment information of the amount of parallax And a means for displaying the determined area. In the three-dimensional image recording apparatus, the user may adjust the amount of parallax while checking the display of the three-dimensional image. According to the recording method and the recording apparatus according to the present embodiment, since the information indicating the shift vector and the pop-out direction / distance is recorded in the sub-code area where the post-recording is easy, it is possible to easily change even in the editing stage. is there.
[0144]
Further, in the present embodiment, the information indicating the shift vector and the pop-out direction / distance is recorded in the sub-code area. However, from the viewpoint that these are also the accompanying information relating to the image, all of them are recorded together in the VAUX area. is there. For this purpose, the operation of the additional information coded data distribution unit 3205 in FIG. 32 is changed so that all the coded data of the above information is output to the VAUX area. In this case, the ease of post-recording is lost, but there is an advantage that handling is simplified because the accompanying information on the image is collected in one place. For example, when making a copy on a medium having another recording format, if only a copy of the image recording area is made, all information relating to the image can be obtained, and there is no need to handle the subcode area. Further, in order to avoid overwriting by after-recording, a method of recording in both the subcode area and the VAUX area is also possible.
[0145]
Alternatively, if the information cannot be stored in the subcode area and the VAUX area due to the size limitation of the area, the information that cannot be stored in the information related to the stereoscopic image is recorded in the AAUX area. Is also possible.
[0146]
The configuration of the present embodiment also conforms to a digital VTR system widely used for home use, except for a part specific to a stereoscopic image. For this reason, among the accompanying information recorded in the present embodiment, the accompanying information unique to the stereoscopic image, that is, the stereoscopic image identification information, the plane display image selection information, the shift vector, the information on the pop-out direction and the distance, and the like are stored in the home digital VTR. If the recording is performed in the extension area where the extension is permitted in the format of, the two-dimensional image and the three-dimensional image can be recorded on the same tape.
[0147]
Further, in the description of the stereoscopic image recording method of the present invention, recording is performed on a digital video tape as a recording medium based on the configuration of FIGS. 28 and 31, but generally, a recording area provided in an image processing apparatus or a terminal, or It is also possible to record on a recording area of an IC memory mounted on a cassette of a digital video tape. In this case, similarly to the above description, the recording area includes an audio recording area, an image recording area, a subcode recording area, and the like, and it is possible to record accompanying information such as stereoscopic information in these areas. .
Although the embodiments have been described above, the present invention is not limited to these examples, and it goes without saying that various modifications are possible.
[0148]
【The invention's effect】
As described above, according to the present invention, it is possible to display a good stereoscopic image with less discomfort in a stereoscopic image in which the amount of parallax is adjusted.
In addition, when displaying a stereoscopic image in which the amount of parallax is adjusted, the impression and influence received by the observer such as the degree of popping out of the image and the degree of fatigue when watching for a long time are compared between displays having different screen sizes and the like. Can not be so different.
[0149]
When displaying a stereoscopic image in which the amount of parallax has been adjusted, the display mode is switched between stereoscopic display and planar display by selecting the display area so that the entire left-eye image or right-eye image is displayed. In such a case, the ranges of the images displayed in both modes do not need to be shifted, and the display mode can be switched without discomfort.
[0150]
According to the present invention, it is possible to improve the convenience of editing by recording information relating to the adjustment of the amount of parallax in a subcode area in which post-recording is easy.
According to the present invention, it is possible to simplify the handling by collectively recording the stereoscopic vision control information including the information on the adjustment of the parallax amount in the image recording area.
[0151]
According to the present invention, it is possible to prevent data loss due to overwriting during post-recording by recording stereoscopic control information including information on adjustment of the amount of parallax in both the subcode area and the image recording area. Become.
[Brief description of the drawings]
FIG. 1 is a functional block diagram illustrating a configuration example of a stereoscopic image display device according to a first embodiment of the present invention.
FIGS. 2A and 2B generate stereoscopic image data from left-eye image data and right-eye image data in the stereoscopic image display device according to the first embodiment of the present invention; It is a figure showing an example.
FIG. 3 is a diagram illustrating an example of corresponding pixels of left-eye image data and right-eye image data when performing stereoscopic viewing in the stereoscopic image display device according to the first embodiment of the present invention.
FIG. 4 is a diagram illustrating an example of image movement when the amount of parallax is adjusted in the stereoscopic image display device according to the first embodiment of the present invention.
FIG. 5 is a diagram showing an example of correspondence between dot data of three primary colors of RGB and pixels in the stereoscopic image display device according to the first embodiment of the present invention.
FIG. 6 is a diagram illustrating an example in which one dot of dot data of three primary colors of RGB is moved when moving an image in the stereoscopic image display device according to the first embodiment of the present invention.
FIG. 7 is a diagram illustrating an example in which two dots of dot data of three primary colors of RGB are moved when moving an image in the stereoscopic image display device according to the first embodiment of the present invention.
FIGS. 8A to 8C are diagrams illustrating an example in which an image is moved without shifting the center when adjusting the amount of parallax in the stereoscopic image display device according to the first embodiment of the present invention. It is.
FIGS. 9A to 9C are diagrams illustrating an example in which image data is moved to adjust the amount of parallax in the stereoscopic image display device according to the first embodiment of the present invention. is there.
FIG. 10 is a diagram illustrating an interpolation target area when moving image data for a parallax amount request in the stereoscopic image display device according to the first embodiment of the present invention and determining an area so that the center after the movement is not shifted; It is a figure showing the example of.
FIGS. 11A and 11B are diagrams illustrating an example of a method of switching a display between a three-dimensional display and a two-dimensional display in the stereoscopic image display device according to the first embodiment of the present invention. .
FIGS. 12A to 12D are diagrams illustrating an example of interpolating image data in the stereoscopic image display device according to the first embodiment of the present invention.
FIG. 13 is a functional block diagram illustrating a configuration example of a stereoscopic image display device according to a second embodiment of the present invention.
FIGS. 14A to 14C illustrate an example in which parallax adjustment is performed according to the second embodiment of the present invention, and when no pixel value exists in a display area of an image, an image is displayed in that area. FIG. 9 is a diagram illustrating an example of generation.
FIG. 15 is a functional block diagram illustrating a configuration example of a stereoscopic image display device according to a third embodiment of the present invention.
FIGS. 16A and 16B are diagrams illustrating an example of a method of storing stereoscopic vision control information for performing stereoscopic vision in a stereoscopic image display device according to a third embodiment of the present invention. .
FIGS. 17A and 17B are descriptions for storing stereoscopic vision control information for performing stereoscopic vision in the stereoscopic image display device according to the third embodiment of the present invention in image data; It is a figure showing an example of a method.
FIG. 18 is a diagram illustrating an example of a description method when recording stereoscopic control information for performing stereoscopic vision in the stereoscopic image display device according to the third embodiment of the present invention in the device.
FIG. 19 is a functional block diagram illustrating a configuration example of a stereoscopic image display device that displays stereoscopic image data accompanied by stereoscopic control information.
FIGS. 20A and 20B are diagrams illustrating an example in which the maximum shift vector in the stereoscopic image display device according to the fourth embodiment of the present invention is recorded while being included in stereoscopic vision control information.
FIG. 21 is a functional block diagram illustrating a configuration example of a stereoscopic image display device that displays stereoscopic image data having information on a maximum shift vector in a stereoscopic image display device according to a fourth embodiment of the present invention.
FIGS. 22 (A) to (E) show a method for adjusting the amount of parallax when displaying stereoscopic image data in a stereoscopic image display device according to a fifth embodiment of the present invention in an enlarged or reduced manner. FIG. 9 is a diagram illustrating an example of moving image data when performing the operation.
FIG. 23 is a functional block diagram illustrating a configuration example of a stereoscopic image display device according to a fifth embodiment of the present invention.
FIGS. 24A and 24B are diagrams illustrating storage examples when transmitting stereoscopic control information by a transmission medium in a stereoscopic image display device according to a sixth embodiment of the present invention.
FIGS. 25A to 25C are diagrams illustrating calculation examples for calculating a pop-out distance in the stereoscopic image display device according to the third embodiment of the present invention.
FIG. 26 is a diagram illustrating a configuration example of stereoscopic information according to a seventh embodiment of the present invention.
FIG. 27 is a diagram showing an example of selecting an image display area according to the seventh embodiment of the present invention.
FIG. 28 is a diagram showing a recording state of a track on a digital video tape.
FIG. 29 is a diagram showing a track format of a digital VTR.
FIG. 30 is a diagram showing a data configuration in an image recording area of each track.
FIG. 31 is a diagram illustrating a configuration of a stereoscopic image recording device according to an eighth embodiment of the present invention.
FIG. 32 is a diagram illustrating a configuration of a multiplexing unit of a stereoscopic image recording device according to an eighth embodiment of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Stereo image processing part, 2 ... User input part, 3 ... Parallax amount adjustment information calculation part, 4 ... Display area determination part, 5 ... Image interpolation part, 6 ... Display part, 7 ... Image data generation part, 8 ... Solid Visual information recording unit, 9: stereoscopic information reading unit, 10: maximum shift vector reading unit, 11: enlargement / reduction processing unit, 2601: pop-out direction, 2602: pop-out distance, 2603: pop-out vector, 2604: flat display image selection Information 3002 VAUX α, 3003 Image encoded data recording area, 3004 VAUX β, 3103 Associated information encoding section, 3104 Multiplex section, 3205 Associated information encoded data distribution section.

Claims (36)

A stereoscopic image display device that displays a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints,
Based on the information about the change in the amount of parallax, a parallax amount adjustment information calculation unit that calculates parallax amount adjustment information about adjustment of the amount of parallax,
A stereoscopic image display device comprising: an image processing unit that generates an image for stereoscopic display based on the parallax amount adjustment information. An image interpolating unit that interpolates a predetermined area in the plurality of images,
The image interpolating unit, when there is a non-pixel value area having no pixel value in a display area of the image for stereoscopic display, interpolates only the non-pixel value area using another pixel value. The three-dimensional image display device according to claim 1. A stereoscopic image display device that displays a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints,
Based on the information about the change in the amount of parallax, a parallax amount adjustment information calculation unit that calculates parallax amount adjustment information about adjustment of the amount of parallax,
An image processing unit that generates an image for stereoscopic display based on the parallax amount adjustment information,
A stereoscopic image display device comprising: an image generation unit configured to generate a new image in a predetermined area in the plurality of images. The method according to claim 3, wherein the image generation unit generates a new image only in the non-pixel region when there is a non-pixel region having no pixel value in a display region of the image for stereoscopic display. 3. The stereoscopic image display device according to claim 1. The stereoscopic image display device according to claim 1, further comprising an input unit configured to input information regarding a change in the amount of parallax. The parallax amount adjustment information calculation unit may adjust the parallax amount such that a distance between a first reference position of stereoscopic display and a second reference position corresponding to the first reference position of the image for stereoscopic display is reduced. The three-dimensional image display device according to claim 1, wherein information is calculated. Further, a display area determination unit that determines a display area for performing stereoscopic display, wherein a distance between a first reference position of stereoscopic display and the first reference position of the image for stereoscopic display is reduced. The three-dimensional image display device according to any one of claims 1 to 6, further comprising a display area determination unit that determines the display area. The stereoscopic image display device according to claim 1, wherein the parallax amount adjustment information calculation unit has a function of limiting a change amount of the parallax amount. Further, a stereoscopic image information recording unit that records stereoscopic image information including at least the stereoscopic image identification information indicating that the plurality of images are stereoscopic images and information indicating a change in the amount of parallax is provided. The stereoscopic image display device according to any one of claims 1 to 8, wherein: The image processing apparatus further includes a stereoscopic information recording unit that records stereoscopic information including at least stereoscopic image identification information indicating that the plurality of images are stereoscopic images and information on the display area. Item 9. The stereoscopic image display device according to any one of Items 1 to 8. A stereoscopic information recording unit that records stereoscopic information including at least the stereoscopic image identification information indicating that the plurality of images are stereoscopic images and information indicating a range of change in the amount of parallax. 9. The three-dimensional image display device according to claim 1, wherein: A recording method for recording a plurality of images corresponding to a plurality of viewpoints,
Recording stereoscopic image identification information indicating that the plurality of images are stereoscopic images, and stereoscopic control information for stereoscopically displaying the plurality of images, wherein the stereoscopic control information includes at least parallax. A recording method comprising: any one of information indicating a change in the amount, information indicating a range of the change in the parallax amount, and information indicating a display area of the plurality of images. A transmission method for transmitting a plurality of images corresponding to a plurality of viewpoints, the stereoscopic image identification information indicating that the plurality of images are stereoscopic images, and a stereoscopic image for stereoscopically viewing the plurality of images. Transmitting stereoscopic control information, wherein the stereoscopic control information is at least one of information indicating a change in the amount of parallax, information indicating a range of change in the amount of parallax, and information indicating a display area of the plurality of images. A transmission method characterized by including the following information: Furthermore, an enlargement / reduction processing unit that enlarges or reduces the plurality of images, and a stereoscopic control information conversion unit that converts stereoscopic control information is provided.
The stereoscopic vision control information conversion unit converts information indicating a change in the amount of parallax based on an enlargement / reduction rate at which the plurality of images have been enlarged / reduced by the enlargement / reduction processing unit. The stereoscopic image display device according to any one of claims 1 to 11. Furthermore, an enlargement / reduction processing unit that enlarges or reduces the plurality of images, and a stereoscopic control information conversion unit that converts stereoscopic control information is provided.
The stereoscopic control information conversion unit converts the information of the display area based on an enlargement / reduction ratio obtained by enlarging / reducing the plurality of images in the enlargement / reduction processing unit. 12. The stereoscopic image display device according to any one of items 11 to 11. Furthermore, an enlargement / reduction processing unit that enlarges or reduces the plurality of images, and a stereoscopic control information conversion unit that converts stereoscopic control information,
The stereoscopic vision control information conversion unit converts information indicating a range of change in the amount of parallax based on an enlargement / reduction rate at which the plurality of images are enlarged / reduced by the enlargement / reduction processing unit. The stereoscopic image display device according to any one of claims 1 to 11, wherein A recording method for recording a plurality of images corresponding to a plurality of viewpoints,
Recording a stereoscopic control information for stereoscopically displaying the plurality of images,
The stereoscopic vision control information includes at least information indicating a change in the amount of parallax,
The recording method according to claim 1, wherein the information indicating the change in the amount of parallax includes parameters determined under a predetermined observation condition. 18. The recording method according to claim 17, wherein the parameter is a parameter indicating a pop-out direction and a distance of the stereoscopic image. 18. The recording method according to claim 17, wherein the parameter is a parameter indicating a direction and a distance in which at least one of the plurality of images is moved. 18. The recording method according to claim 17, wherein the information indicating the change in the amount of parallax includes a parameter indicating a parallax angle of the stereoscopic image. 21. The stereoscopic viewing control information according to claim 17, wherein the stereoscopic vision control information includes at least planar display image selection information indicating which one of the plurality of images is to be used when displaying a stereoscopic image in a plane. 2. The recording method according to item 1. A transmission method for transmitting a plurality of images corresponding to a plurality of viewpoints,
Comprising transmitting stereoscopic control information for stereoscopically displaying the plurality of images,
The stereoscopic vision control information includes at least information indicating a change in the amount of parallax,
The transmission method according to claim 1, wherein the information indicating the change in the amount of parallax includes parameters determined under predetermined observation conditions. The transmission method according to claim 22, wherein the parameter is a parameter indicating a pop-out direction and a distance of the stereoscopic image. The transmission method according to claim 22, wherein the parameter is a parameter indicating a direction and a distance in which at least one of the plurality of images is moved. 23. The transmission method according to claim 22, wherein the information indicating the change in the amount of parallax includes a parameter indicating a parallax angle of a stereoscopic image. 26. The stereoscopic viewing control information according to claim 22, wherein the stereoscopic vision control information includes at least planar display image selection information indicating which one of the plurality of images is to be used when displaying the stereoscopic image in a plane. The transmission method according to claim 1. A stereoscopic image display device that displays a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints,
Based on the information about the change in the amount of parallax, a parallax amount adjustment information calculation unit that calculates parallax amount adjustment information about adjustment of the amount of parallax,
An image processing unit that generates an image for stereoscopic display based on the parallax amount adjustment information,
A stereoscopic image display device comprising: a display area determination unit that determines a display area based on image selection information related to planar display. 28. The parallax amount adjustment information calculation unit calculates parallax amount adjustment information from parameters indicating a pop-out direction and a distance of a stereoscopic image, according to any one of claims 1 to 11, or 27. 3. The stereoscopic image display device according to 1. The parallax amount adjustment information calculation unit calculates parallax amount adjustment information from a parameter indicating a direction and a distance in which at least one image of the plurality of images is moved, Alternatively, the stereoscopic image display device according to claim 27. The parallax amount adjustment information calculation unit calculates parallax amount adjustment information from a parameter indicating a parallax angle of a stereoscopic image, wherein the parallax amount adjustment information calculation unit calculates parallax amount adjustment information based on a parameter indicating a parallax angle of the stereoscopic image. 3. The stereoscopic image display device according to claim 1. A stereoscopic image recording method for recording a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,
A stereoscopic image recording method, comprising: recording parallax amount adjustment information relating to adjustment of a parallax amount of a stereoscopic image in an image recording area provided in the recording area and for recording the stereoscopic image. A stereoscopic image recording method for recording a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,
A stereoscopic image recording method, comprising: recording parallax amount adjustment information relating to adjustment of a stereoscopic image parallax amount in an audio recording area provided in the recording area for recording audio. A stereoscopic image recording method for recording a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,
A stereoscopic image recording method, comprising the step of recording parallax amount adjustment information relating to adjustment of the parallax amount of a stereoscopic image in a subcode area provided in the recording area and for recording accompanying information. A stereoscopic image recording device that records a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,
A stereoscopic image recording apparatus, comprising: means for recording parallax amount adjustment information relating to adjustment of a parallax amount of a stereoscopic image in an image recording area provided in the recording area for recording the stereoscopic image. A stereoscopic image recording device that records a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,
A stereoscopic image recording apparatus, comprising: means for recording parallax amount adjustment information relating to adjustment of a parallax amount of a stereoscopic image in an audio recording area provided in the recording area for recording audio. A stereoscopic image recording device that records a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,
A stereoscopic image recording apparatus, comprising: means for recording parallax amount adjustment information relating to adjustment of a parallax amount of a stereoscopic image in a sub-code area for recording accompanying information, the stereoscopic image recording apparatus.

Images