WO2004030377A1

WO2004030377A1 - Three-dimensional image display device, recording method, and transmission method

Info

Publication number: WO2004030377A1
Application number: PCT/JP2003/012302
Authority: WO
Inventors: Tadashi Uchiumi; Hiroyuki Katata; Toshio Nomura; Yoshiaki Ogisawa; Norio Ito; Hiroaki Yabe; Ryuji Kitaura; Masatoshi Tsujimoto; Motohiro Ito
Original assignee: Sharp Kabushiki Kaisha
Priority date: 2002-09-27
Filing date: 2003-09-26
Publication date: 2004-04-08
Also published as: AU2003268677A1

Abstract

A three-dimensional image display device (A) for displaying a 3-dimensional image according to a plurality of images corresponding to a plurality of viewpoints. The device includes: a parallax amount adjustment information calculation section (3) for calculating parallax amount adjustment information on the parallax amount adjustment according to information on parallax amount modification; an image processing section (1) for generating an image for three-dimensional display according to the parallax amount adjustment information; and an image interpolation section (5) for interpolating a predetermined region in a plurality of images. When a non-pixel value region having no pixel value is present in the image display region for three-dimensional display, the image interpolation section (5) interpolates only the non-pixel value region by using other pixel value, so that a preferable three-dimensional image is displayed with little uncomfortable feeling in a three-dimensional image in which the parallax amount is adjusted.

Description

3D image display device, recording method, and transmission method

The present invention relates to a technology for stereoscopically displaying a plurality of images corresponding to a plurality of viewing points. Background art

Conventionally, there is known a stereoscopic image display method capable of viewing an image having a stereoscopic effect by stereoscopically viewing a pair of images having parallax. For example, images for the left eye and right eye are alternately output to the display device, and the user can switch the shutter in synchronization with the display switching timing through the glasses. By playing back the image, it is possible to observe a stereoscopic image.

In addition, there is a method called parallax barrier method as a method of reproducing a stereoscopic image without using special glasses or the like. The image for the left eye and the image for the right eye are each broken into strips in the vertical scanning direction of the image, and alternately arranged to form a single image. The display device that displays the image has strip strips similar to those obtained when the image is disassembled.

The strip-like image data is observed through the slit by the display. When the left-eye image arranged in a strip shape by the polarizer is reproduced by the user's left eye and the right-eye image is reproduced by the right eye, it is possible to obtain a three-dimensional effect on the image. There is also a method called a lenticular lens method that uses a lenticular lens instead of a slit.

In order for the user to see a better stereoscopic image, change the stereoscopic effect of the playback image Further techniques are disclosed. When a human observes an object in three dimensions, the left and right eyes observe different images, and these images have a shift called parallax.

A human recognizes a stereoscopic effect by this parallax. The amount of parallax is called the amount of parallax, and adjusting the amount of parallax adjusts the stereoscopic effect. It describes the technology to display on a stereoscopic display by adjusting the amount of parallax when receiving and displaying a stereoscopic video in digital broadcasting, in accordance with the environment in which the user views the stereoscopic video, etc. Thus, it is possible to observe a stereoscopic image (see, for example, Japanese Patent Application Laid-Open No. 2000-76815, Fig. 1). Disclosure of the invention

However, when adjusting the amount of parallax, for example, in the method of alternately arranging the left and right images in a strip shape such as a lenticule method, a parallax variation method, etc., the combination position of the strip data is usually used. Although the amount of parallax can be adjusted by changing, the accuracy of the adjustment is defined by the strip width. Therefore, it is not always possible to adjust the three-dimensional effect according to the user's preference.

In addition, when the combination position of the strip data is changed, a portion where there is no data to be reproduced as a three-dimensional image is generated, and the portion can not perform a good stereoscopic display. Therefore, the part where the stereoscopic image can be displayed well (display area where the stereoscopic image can be observed) becomes smaller.

On the other hand, in the image after parallax amount adjustment, the width of the strip-like image data after parallax amount adjustment is wider than the width of the display area by the parallax amount adjustment, even if all image data are to be displayed. It is a record that can display all image data.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a stereoscopic image in which the amount of parallax is adjusted Is to display a good 3D image.

According to one aspect of the present invention, there is provided a stereoscopic image display device that displays a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints, and a readout unit that reads parallax amount change information indicating a change in parallax amount. A stereoscopic image display apparatus is provided, comprising: an image processing unit that generates an image for stereoscopic display based on the parallax amount change information; and an image capturing unit that interpolates a predetermined region in the plurality of images. <When the non-pixel value area where the pixel value is not present exists in the display area of the image for stereoscopic display, the image interpolation unit captures only the non-pixel value area using another pixel value. Is preferred.

According to the stereoscopic image display device, it is possible to interpolate the pixel value of the non-pixel value area in relation to the change of the parallax amount.

According to another aspect of the present invention, there is provided a three-dimensional image display device for displaying a three-dimensional image based on a plurality of images corresponding to a plurality of viewpoints. An image processing unit that generates an image for stereoscopic display based on the parallax amount change information; and an image generation unit that generates a new image in a predetermined region of the plurality of images. An image display device is provided.

The image generation unit preferably generates a new image only in the non-pixel area, when there is a non-pixel area in which no pixel value exists in the display area of the image for stereoscopic display.

According to the stereoscopic image display device, a new image can be generated in the non-pixel value area in association with the change of the parallax amount.

According to another aspect of the present invention, there is provided a transmission method for transmitting a plurality of images corresponding to a plurality of viewpoints, wherein stereoscopic control information including information for adjusting the amount of parallax related to stereoscopic vision is transmitted. A transmission method is provided.

Furthermore, according to another aspect of the present invention, a plurality of points corresponding to a plurality of viewpoints is provided. Provided are a recording method and a recording apparatus for recording an image, and a recording method and a recording apparatus for recording stereoscopic vision control information including information for adjusting a parallax amount related to stereoscopic vision.

Brief description of the drawings

FIG. 1 is a functional block diagram showing an example of the configuration of a stereoscopic image display apparatus according to a first embodiment of the present invention. '

Figs. 2 (A) and 2 (B) show an example of generating stereoscopic image data from the image data for the left eye and the image data for the right eye in the stereoscopic image display apparatus according to the first embodiment of the present invention. FIG.

FIG. 3 is a view showing an example of corresponding pixels of image data for the left eye and image data for the right eye when viewed stereoscopically in the stereoscopic image display device according to the first embodiment of the present invention.

FIG. 4 is a view showing an example of movement of an image at the time of parallax amount adjustment in the stereoscopic image display apparatus according to the first embodiment of the present invention.

FIG. 5 is a diagram showing an example of correspondence between dot data of the RGB three primary colors and pixels in the stereoscopic image display device according to the first embodiment of the present invention.

FIG. 6 is a view showing an example of moving one dot of dot data of RGB three primary colors when moving an image in the stereoscopic image display apparatus according to the first embodiment of the present invention. is there.

FIG. 7 is a diagram showing an example of moving two dots of dot data of the R G B three primary colors when moving an image in the stereoscopic image display apparatus according to the first embodiment of the present invention.

8 (A) to 8 (C) show the structure of the first embodiment of the present invention. FIG. 17 is a diagram showing an example of moving an image without shifting the center when adjusting the amount of parallax in the body image display device.

Figures 9 (A) to (C) are diagrams showing an example when moving image data for adjusting the amount of parallax in the stereoscopic image display apparatus according to the first embodiment of the present invention. is there.

FIG. 10 shows an area for moving the image data for the parallax amount request in the stereoscopic image display apparatus according to the first embodiment of the present invention, and not shifting the center after movement. It is a figure which shows the example of the interpolation object area | region at the time of determining. Figures 1 (a) and (b) illustrate an example of how to switch between three-dimensional display and two-dimensional display in the stereoscopic image display apparatus according to the first embodiment of the present invention. It is.

12 (A) to (D) are diagrams showing an example of capturing image data in the stereoscopic image display apparatus according to the first embodiment of the present invention. FIG. 13 is a functional block diagram showing a configuration example of a stereoscopic image display apparatus according to a second embodiment of the present invention.

14 (A) to (C), the parallax amount adjustment is performed in the second embodiment of the present invention, and when there is no pixel value in the display area of the image, the image is displayed in that area. Is a diagram illustrating an example of generating

FIG. 15 is a functional block diagram showing a configuration example of a stereoscopic image display apparatus according to a third embodiment of the present invention.

16 (A) and 16 (B) are diagrams showing an example of a storage method of stereoscopic vision control information for stereoscopic vision in the stereoscopic image display apparatus according to the third embodiment of the present invention.

17 (A) and (B) are description methods for storing, in image data, stereoscopic vision control information for stereoscopic vision in a stereoscopic image display apparatus according to the third embodiment of the present invention. Is a diagram illustrating an example of FIG. 18 is a view showing an example of a description method when recording stereoscopic vision control information for stereoscopic vision in the stereoscopic image display apparatus according to the third embodiment of the present invention. is there.

FIG. 19 is a functional block diagram showing a configuration example of a stereoscopic image display device for displaying stereoscopic image data accompanied by stereoscopic control information.

FIGS. 20A and 20B show an example of recording the maximum shift vector in the stereoscopic image display apparatus according to the fourth embodiment of the present invention in the stereoscopic control information. It is.

FIG. 21 shows the configuration of a stereoscopic image display device for displaying stereoscopic image data having information on the maximum shift vector in the stereoscopic image display device according to the fourth embodiment of the present invention. It is a functional block diagram showing an example.

Figures 22 (A) to (E) illustrate parallax adjustment when the image data for stereoscopic viewing in the stereoscopic image display apparatus according to the fifth embodiment of the present invention is displayed in an enlarged scale or reduced size. It is a figure which shows the movement example of the image data of. FIG. 23 is a functional block diagram showing a configuration example of a stereoscopic image display apparatus according to a fifth embodiment of the present invention.

24 (A) and (B) are diagrams showing storage examples when transmitting stereoscopic control information in a transmission medium in the stereoscopic image display apparatus according to the sixth embodiment of the present invention.

FIGS. 25 (A) to (C) are diagrams showing calculation examples for calculating the projection distance in the stereoscopic image display apparatus according to the third embodiment of the present invention.

FIG. 26 is a diagram showing an example of a configuration of stereoscopic information according to a seventh embodiment of the present invention.

FIG. 27 is a view showing an example of selection of an image display area according to a seventh embodiment of the present invention. FIG. 28 is a diagram showing an example of the configuration of stereoscopic information according to the eighth embodiment of the present invention.

Fig. 29 is a diagram showing the recording state of the track on the digital video tape.

FIG. 30 is a diagram showing a track format of digital VTR. FIG. 31 shows the data structure in the image recording area of each track.

FIG. 32 is a diagram showing the configuration of a stereoscopic image recording apparatus according to a ninth embodiment of the present invention.

FIG. 33 is a diagram showing the configuration of a multiplexing unit of a stereoscopic image recording apparatus according to a ninth embodiment of the present invention. BEST MODE FOR CARRYING OUT THE INVENTION

In the present specification, each of RGB data of three primary colors is referred to as a dot, and a group of RGB data of three primary colors is referred to as a pixel. Also, in the present embodiment, the image data includes moving images and still images. Furthermore, the image data includes, for example, still image compression technology such as JPEG, and compressed image data using moving image compression technology such as MPEG-4.

Hereinafter, a stereoscopic image display technique according to the first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a functional block diagram showing a configuration example of a stereoscopic image display apparatus according to the present embodiment. As shown in FIG. 1, a stereoscopic image display apparatus A according to the present embodiment is image data capable of stereoscopic display based on input image data D in (hereinafter referred to as “image data for stereoscopic vision” as well). Image processing unit 1 to perform image processing to generate, user input unit 2 to which the user inputs, and parallax amount adjustment information calculation to calculate parallax amount adjustment information based on the user's input. Part 3 and 3D image processed The display area determination unit 4 that determines the area to be actually displayed on the stereoscopic video display device from the image, the image interpolation unit 5 that generates interpolation data in a specific area, and the determined area among the generated images are displayed. It consists of the display unit 6.

The input image data D in is a plurality of images corresponding to a plurality of viewpoints, and in the following description, when it is described as an image for the left eye or an image for the right eye, Represents one of the images. The image data for the left eye and the image data for the right eye input to the stereoscopic image display device A are first converted to data that can be stereoscopically displayed in the stereoscopic image processing unit 1. For example, in the case of a stereoscopic image display apparatus based on the parallax barrier method or lenticule method, the stereoscopic image processing unit 1 displays stereoscopic image in which left-eye image data and right-eye image data are alternately arranged in a strip shape. Generate image data for

The user inputs data for adjusting the amount of parallax of the stereoscopic image to be displayed by the user input unit 2. The input data for parallax amount adjustment is converted into parallax amount adjustment information in the parallax amount adjustment information calculation unit 3, and is output to the stereoscopic image processing unit 1. The stereoscopic image processing unit 1 generates stereoscopic image data using the input of parallax amount adjustment information when generating stereoscopic image data. The generated stereoscopic image data is output to the display area determination unit 4.

The display area determination unit 4 determines whether or not the generated stereoscopic image data can be favorably displayed on the display unit 6. The image width of the stereoscopic image data for which the parallax amount adjustment has been performed may not correspond to the image width that can be displayed by the display unit 6. In such a case, the pixel area of the display unit 6 and the pixel area of the stereoscopic image data do not coincide with each other, and an unmatched area is generated. In such areas, it may not be possible to display image data in good condition without knowing how it is displayed. Therefore, if the display area determination unit 4 determines the presence or absence of an area for which the correspondence can not be obtained, and if the display area determination unit 4 determines that there is an area for which the correspondence can not be obtained, the image interpolation unit 5 can not correspond. Image data S 1 for stereoscopic vision including information on the area is output. If the display area determination unit 4 determines that there is no area for which the correspondence can not be obtained, the stereoscopic image data S 2 is output as it is (through) to the display unit 6.

In the image interpolation unit 5, the stereoscopic image data output from the display area determination unit 4 is subjected to interpolation processing so as to apply pixel data to the area determined by the display area determination unit 4 according to a predetermined method. I do. Further, the display unit 6 outputs the stereoscopic image data subjected to the interpolation processing. The display unit 6 displays the stereoscopic image data S 2 output from the display area determination unit 4 or the stereoscopic image data S 3 output from the image capturing unit 5.

In the stereoscopic image display device A according to the present embodiment, the format of the stereoscopic image data displayed on the display unit 6 is the left and right images represented by the parallax barrier method and the lenticule method. In this method, stereoscopic viewing is performed by alternately arranging strips of image data for one pixel. The input image is image data including data that can be viewed stereoscopically, and the image data viewed stereoscopically in the display unit 6 handles data in which the data of the left-eye image and the right-eye image are arranged in a strip shape Do. As long as the input image data can generate strip-like data in the stereoscopic image processing unit 1, it is sufficient. For example, the input image may be surface image data originally combined into a strip shape. In this case, the three-dimensional image processing unit 1 outputs the input data as it is without creating a strip-like image again.

Alternatively, the image data of the left eye and the right eye may be separate image data. In this case, as described in FIG. 2 (A) below, the stereoscopic image processing unit 1 creates a strip-like image from the image for the left eye and the image for the right eye. Figure 2 (A) shows the input image data for the left eye FIG. 18 is a diagram showing an example of generating stereoscopic image data from the image of FIG. From the image data for the left eye and the image data for the right eye in FIG. 2 (A), pixels reproduced by the same slit when stereoscopically viewing strip-like data (2 0 1, 2 0 2 ) Get the data, arrange them and data for stereoscopic display 2 0

Generate 0. By repeating this process, it is possible to generate stereoscopic image data having a strip-like width of one pixel.

When strip-like stereoscopic image data 2 0 0 is created by this method, the width of the strip-like stereoscopic image data 2 0 0 is the width of the input image data 2 0 1 for the left eye. And the input image data 2 0 2 for the right eye are equal to the combined width of the respective images. Therefore, in order to create data that can be displayed by the display unit 6, the input image data 2 0 1 for the left eye and the input image 2 0 2 for the right eye The image width needs to be half of 200.

If the image width of the input image data 2 0 1 for the left eye and the input image data 2 0 2 for the right eye is the same width as the display width of the display unit 6, strip-shaped image data is generated At the same time, strip out the data so that the image width of the input image data 2 0 1 for the left eye and the input image data 2 0 2 for the right eye is halved, and then generate strip-like image data. .

Fig. 2 (B) shows an example of a method of creating strip data by thinning out the input image data. As shown in FIG. 2 (B), from each pixel of the image for the left eye and for the right eye, of the RGB three primary colors, the input image data for the left eye 2 0 1 to the G data 2 0 4 1 RGB dot 2 0 3 is generated by taking 2 dots of R image 2 0 5 and B data 2 0 6 from the input image data 2 0 2 for the right eye. Do the process. This process is alternately performed for each image (input image data 2 0 1 for the left eye is taken one data, and input image data 2 0 2 for the right eye is obtained. And two sets of input image data for left eye 2 0 1 and a set of input image data for right eye 2 0 2 and a single set of data) are repeated. , Create 3 0 0 image data for stereoscopic viewing. The method of producing the strip-like image is not limited to the method described with reference to FIG. 2 (B). For example, the pixel indicated by reference numeral 2 0 3 in FIG. Dot (2 0 4), R 1 dot (2 0 5) for the right eye, B 1 dot (2 0 6), but the R 1 dot for the left eye with reversed pattern Image data may be generated from the image (2 0 7), the B 1 dot (2 0 8), and the right eye G 1 dot (2 0 9). The three-dimensional image processing unit 1 shown in FIG. 1 compares the number of pixels of each input image with the number of displayable pixels, and determines whether to strip the input image to generate a strip-like image. Perform and generate strip-shaped images. Next, processing when the user adjusts the amount of parallax will be described. The user adjusts the amount of parallax with the user input unit 2 shown in Fig.1. The user input unit 2 may have any shape and method, such as an input device such as a keyboard or a mouse, or a remote control.

For example, it is conceivable to adjust the amount of parallax by an amount proportional to the number of times the key is pressed in the specific direction by pressing the key for designating the specific direction of the keyboard one or more times. The data input by the user input unit 2 is converted into parallax amount adjustment information in the parallax amount adjustment information calculation unit 3. For example, it is converted into specific information such as “move the image data for the right eye by one pixel in the left direction”. The parallax amount adjustment information calculation unit 3 outputs the calculated parallax amount adjustment information to the stereoscopic image processing unit 1. The stereoscopic image processing unit 1 generates strip-like image data using the parallax amount adjustment information.

The process of adjusting the amount of parallax will be described with reference to FIGS. 3 and 4. In FIG. 3, a block indicated by reference numeral 301 indicates the data for one pixel, and L and R indicate an image for the left eye and an image for the right eye, respectively. It is a code that indicates and. Also, the numbers given below L or R indicate the pixel numbers, which are given for the convenience of description, and the pixel numbers are given in order from the left end of the image data. The reference numeral 301 indicates the leftmost pixel of the left-eye image. The reference numerals 3 0 4 in FIG. 3 and 4 0 3 in FIG. 4 indicate the width of the display area in which the display unit displays each image data.

In addition, when observing a three-dimensional image using a lenticular lens, a parallax parallax, or the like, pixels observed through the same slit (for example, reference numeral 1 1 0 3 in FIG. 11). The pixels denoted by reference numerals 1 1 0 4 4) are shown as pairs corresponding to 3 0 2 and 3 0 3 in FIG. 3 corresponding to the upper and lower sides as a pair reproduced as a stereoscopic image. Since the strip-like image data generated here has data to be displayed for all the pixels possessed by the display unit 6 if the parallax amount adjustment is not performed by the user, the display area determination is performed. It is determined that there is no area to be captured in the display area determination in Part 4.

It is FIG. 4 that the user's adjustment by the parallax amount adjustment information calculation unit 3 shows, for example, a pixel when moving the image for the left eye by one pixel to the right, and the corresponding pixel data.

If the display position of the image data is moved and displayed as it is, as in the position of the pixel 401 in FIG. 4 (indicated by a broken line), there is no pixel data to be displayed, and FIG. 4 As with the pixel indicated by the symbol of 402, pixel data that can not be displayed because of being out of the display area 4 0 3 is generated. At this time, the display area judgment unit 4 judges that there is no image data to be displayed at the pixel 401, and that the pixel 402 is out of the display area 403, and the information is Output to the image interpolation unit 5.

The movement of the image for adjusting the amount of parallax described above has been described for the case of one pixel unit. In this case, the adjustment accuracy may be too coarse depending on the size of the image data and the size of the display unit 6. Humans recognize images by recognizing RGB as a group of data when each dot of the RGB three-primary-color data that composes each pixel forms an image on the eye. You can think of it as something. Therefore, for example, even if the RGB alignment is changed to GRB, they should be recognized as a whole by the human eye. Fig. 5 shows the image data for the left eye in Fig. 3 expressed at the RGB dot level. In order to recognize the L 3 image data (pixel 3 0 2 in FIG. 3), L—R 3 (dot 5 0 1 in FIG. 5) and L-G 3 (dot 5 in FIG. 5) It is only necessary to recognize 3 dots of 0 2) and L-B 3 (dot 5 0 3 in Fig. 5) as one group.

That is, it is possible to move at the dot level of the three primary color data RGB. The situation at this time is shown in Fig.6. Fig. 6 focuses on the image for the left eye in Fig. 5, and Fig. 6 (A) is in the same state as Fig. 5. The code 6 0 4 indicates a display area in which the display unit displays the image for the left eye. Fig. 6 (B) shows the state moved by one pixel (Fig. 4). Figure 6 (C) shows the situation where the dot level has been moved by one dot.

The shaded area indicates that the dot data is missing due to the movement of the left-eye image. Fig. 6 (C) focuses on the dots of R data of the image for the left eye (symbol 6 0 1 in Fig. 6 (A)), and L 1 R 1 is L 1 R 2 L 1 R 2 is L -It is a figure which shows the state which moved R data to the position of L-R 4-R 3 L-R 3.

As shown in Fig. 6 (C), the original arrangement of RGB 1 pixels has not changed even if the data is moved in this way. The movement amount of the image by this movement method is smaller than the movement of one pixel unit (6 0 3 in FIG. 6 (B)), as shown by the code 62 2 in FIG. 6 (C), 1 The movement is 3 pixels. Therefore, it is possible to adjust the amount of parallax finer than when moving in data units of one pixel. Similarly, focusing on 2 dots of R data and G data, as shown in Fig. 7 (C), L-R l, L 1 G 1 to L 1 R 2, L 1 G 2 L 1 By moving R 2 and L 1 G 2 to L 1 R 3 and L 1 G 3 respectively, it is possible to make the adjustment amount finer. This movement corresponds to movement of 2/3 pixels (symbol 7 0 3 in Fig. 7). Even in this case, the amount of parallax can be adjusted more finely than moving in data units of one pixel.

In the present embodiment, an example of the movement of R data and the movement of R data and G data has been described, but the present invention is not limited to the combination of the corresponding data. Also, the image data may be moved by combining the movement of color data unit and the movement of one pixel unit.

When adjusting the amount of parallax according to the method described above, if only one of the left and right images is moved, the center of the stereoscopic image observed by the user is also shifted in the moved direction, which is favorable. Stereoscopic vision is difficult. Therefore, when calculating the parallax amount adjustment information, the parallax amount adjustment information calculation unit 3 may adjust so that the left eye image and the right eye image are interlocked and moved at the same time. . For example, if the user adjusts the amount of parallax by four pixels, the left eye image is not moved by four pixels, but the left eye image by two pixels and the right eye image by the left eye image. It is also possible to use an adjustment method by moving two pixels in the reverse direction. In this way, by moving the image for the left eye and the image for the right eye in the opposite directions by the same amount, it is possible to adjust the amount of parallax without shifting the center of the stereoscopic display before and after the movement. It is possible.

When the amount of parallax adjustment by the user is an even number of pixels, it is possible to adjust so as not to shift the center of the stereoscopic display. However, if the adjustment amount is an odd number of pixels, the left-eye image and the right-eye image can not be moved by the same amount, and the center of the stereoscopic display is shifted to the left or right. In such a case, the parallax amount adjustment information calculation unit The amount of movement of the left-eye image and the right-eye image is calculated so as to minimize the deviation. For example, when the user requests a parallax amount adjustment of 5 pixels, if the unit for moving the image is 1 pixel unit, the left eye image is 3 pixels and the right eye image is 2 pixels. Move in the reverse direction. As described above, if it is possible to move for each dot of RGB, the image for the left eye is moved by 2 + (2 × 3) pixels, and the image for the right eye is moved by 2 + (1/3) pixels Let As described above, even if the center shift of the stereoscopic display is minimized, the shift of the center becomes large as the adjustment amount repeatedly moves by the odd number of pixels. Therefore, when the parallax amount adjustment information calculation unit 3 repeatedly moves, the deviation information is calculated and stored, and based on this storage, the deviation of the center of the stereoscopic display is reduced. The amount of movement of the left-eye image and the right-eye image is determined. Preferably, the center of the stereoscopic display should be adjusted to minimize deviation.

FIG. 8 (A) shows the initial state of the image for the left eye and the image for the right eye. The numeral 8 0 1 denotes a display area for displaying each image in the display unit 6, and the numeral 8 0 2 denotes a center of the display area 8 0 1. When the image data in Fig. 8 (A) is viewed stereoscopically, the center of the stereoscopic display and the center (80 2) of the display area are the same. Figure 8 (B) shows a state in which the user first moves the image data for the left eye by one pixel to the right from the state in Figure 8 (A). The parallax amount adjustment information calculation unit 3 calculates the movement amount as one pixel, from the input of the parallax amount adjustment from the user, and notifies the stereoscopic image processing unit 1 of it. For example, it is notified as vector information indicating “move the image for the left eye one pixel to the right” (code 8 0 3 in FIG. 8 (B)).

The three-dimensional image processing unit 1 to which this vector information has been notified generates a moved image as shown in FIG. 8 (B). The parallax adjustment amount information calculation unit 3 calculates and stores the deviation from the center of the display area at the center of the stereoscopic display from this information. In this example, because only one image is moved, The center is located at position 8 0 4 in Fig. 8 (B). Therefore, the shift information calculated by the parallax amount adjustment information calculation unit 3 is an arrow (vector) indicated by a symbol 800 in FIG. 8 (B).

Next, when the user adjusts the parallax amount again, the parallax amount adjustment information calculation unit 3 uses the deviation information at the time of the previous movement to calculate the amount of deviation adjustment information so that the deviation becomes smaller. Do. It is preferable to adjust it so that the deviation is minimized. When it is calculated that “move the image data for the left eye to the right by 3 pixels further” as parallax amount adjustment information, the deviation of the code 8 0 5 in FIG. 8 (B) is eliminated. Thus, the left-eye image is moved by one plane to the right, and the right-eye image is notified to the three-dimensional image processing unit 1 so as to move two pixels to the left.

Figure 8 (C) shows how to move based on this notification. Arrows (vectors) shown by symbols 8 0 6 and 8 0 7 in Fig. 8 (C) correspond to the amounts of movement of the respective images. It can be seen that the line indicated by reference numeral 8 0 8 in FIG. 8 (C) becomes the center of the stereoscopic display after movement, and the deviation from the center of the display area 8 0 1 is eliminated.

In the present embodiment, the method of adjusting the displacement based on the stored displacement information has been described as an example, but a method in which the displacement of the center of the stereoscopic display becomes smaller after the user adjusts the amount of parallax, preferably or minimally Other methods may be used as long as For example, the user stores the moved image and direction using the device that adjusts the parallax for each pixel, moves the image that is not the moving object in the previous operation as the moving object in the subsequent operation, and moves alternately It is also possible to use a method to eliminate the gap by In this manner, the image data whose parallax amount has been adjusted by the stereoscopic image processing unit 1 is output to the display area determination unit 4.

Because the input image is usually created in consideration of the fact that it is displayed well on the display unit 6 without adjusting the amount of parallax, the adjustment of the amount of parallax should not be performed. Then, the strip-shaped image generated from the input image has the same size as the display area of the display unit 6. However, the strip-shaped image generated by adjusting the amount of parallax is larger than the display area of the display unit 6. Therefore, the display area determination unit 4 determines which area of the strip-like image is to be displayed on the display unit 6. As a determination criterion of this area, for example, a method of shifting the center of the stereoscopic display as far as possible from the center of the display as possible, or a method of extracting and displaying only the stereoscopically visible area, etc. Is considered.

The process when moving the input image data will be described with reference to FIGS. 9 and 10. FIG. 9 (A) shows input image data. In FIG. 9 (A), reference numeral 901 is left-eye stereo image data, and reference numeral 902 is right-eye stereo image data. It is assumed that the size of the strip-shaped image generated from these input image data is the same size as the display area of the display unit 6 in FIG. Fig. 9 (A) The reference numerals 符号 0 3 符号 C C C (C) まで 0 0 3 0 0 0 0 0 0 0 0 0 0 0 表示表示表示表示表示表示表示. It is an arrow showing the width of.

Fig. 9 (B) shows an example of moving the stereo image data for the right eye by the vector indicated by the symbol 9 0 6. A symbol 9 0 4 is an area where there is no image data to be displayed, a symbol 9 1 1 indicates the center of the display area, a symbol 9 1 2 indicates the center of the stereoscopic display, and a symbol 9 1 3 indicates Indicates the shift amount from 9 1 1 to 9 1 2. As shown in FIG. 9 (B), the display area judgment unit 4 (FIG. 1) changes the display area of the image data and extracts an area which does not change the center of the stereoscopic display. This makes it possible to reproduce good stereoscopic images.

For example, the range indicated by the arrow of reference numeral 1 0 0 1 in FIG. As shown in Fig. 10, the area of the sign 1 0 0 4, the sign 1 0 0 5, the sign 1 0 0 6 and the sign 1 0 0 7 is not used for display. Among these, sign 1 0 0 6 and sign In the area shown by 1 0 0 5, there is a pixel, but it is a part that is discarded from the display. In addition, since the area 1002 and the area 1003 are display areas but there is no image data, they are portions to which data are given by image interpolation processing described later. In this way, by selecting the display area, it is possible to set the center of the display position and the center of the stereoscopic display in agreement or in the vicinity.

As shown in FIG. 9 and FIG. 10, when the image is moved, the area of reference numeral 9 0 4 in FIG. 9 (A) and the reference numeral 1 0 0 2 and reference numeral 1 0 0 in FIG. As in the third area, a display area without image data is generated. Therefore, in order to reproduce such a region better, interpolation processing of image data is performed. In the method according to the present embodiment, it is assumed that a display capable of partially switching between the two-dimensional display mode and the three-dimensional display mode (three-dimensional display) is used for the display unit 6 (FIG. 1). , Make an interpolation process that can be displayed well on the display. Description will be made with reference to FIG.

The display mode can be switched, for example, by eliminating part of the barriers (symbols 1 1 0 1 and 1 1 0 2) shown in FIG. 11 (A). By the fact that the pixel indicated by the code 1 1 0 3 and the code 1 1 0 4 4 reproduced by one eye is reproducible by both eyes as shown in FIG. 1 1 (B). Be realized. However, if the display mode is partially switched, the switching means is not limited to the method shown in FIG. 11 (B). As described above, for the image data for which the amount of parallax has been adjusted by the user, the display area determination unit 4 sets the area to be displayed in the display area, and the pixel data is set in the set display area. Determine if there is a missing area. Information on this area is output to the image interpolation unit 5 together with the image data. In order to obtain image data reproduced satisfactorily in the two-dimensional display mode, the image interpolation unit 5 uses data of peripheral pixels for pixels in which image data does not exist. Next Pixel data is generated by interpolation processing.

An example of the process in the image interpolation unit 5 will be described with reference to FIG. As described above, the area indicated by reference numeral 9 0 4 in FIG. 9 (B) is an area where interpolation processing is required due to movement. The display area determination unit 4 determines that there is no image data to be displayed in the area indicated by the reference numeral 9 0 4 and displays the display area so as to display this area 9 0 4 in the two-dimensional display mode. Instruct 6 At the same time, it instructs the image interpolation unit 5 to generate interpolation data in this area 90 4.

The image interpolation unit 5 first copies the data of the corresponding part from the image for the left eye to the area where the data is missing due to the movement of the image and is also the display target. In the example shown in Fig. 9 (C), the data of area 9 0 7 is copied to area 9 0 8. Actually, in the state shown in FIG. 9 (C), left-eye stereo image data and right-eye stereo image data are arranged in a strip shape. Figure 12 (A) shows an enlarged view of the area 911 in Figure 9 (C). As shown in Fig. 12 (A), the tree part is shown in black and the background part in white. The stereoscopic image data converted to a strip including this portion is as shown in FIG. 12 (B).

The shaded portion (such as the region indicated by reference numeral 1201) in Fig. 12 (B) is a portion where there is no pixel data to be displayed because the right-eye stereo image data has moved. It is. FIG. 12 (C) is a view showing a state where pixel data is copied from the corresponding part of the left-eye stereoscopic image data to the right-eye stereoscopic image data as described above. Figure 12 (C) shows that the image data of L 1 is copied to the image data of R 2 L 2 to R 3. By performing this kind of treatment, when the corresponding area is displayed, it is observed better than Fig. 12 (B), but the roughness is actually noticeable. Therefore, the image capturing area 5 generates the capturing image as shown in FIG. 12 (D) during the capturing process. Do. For example, apply 2 taps low pass filter horizontally. For example, the data of the image area (code 1 2 0 4) of R 5 is L 4 (code 1 2 0 3) and L 5 (code 1 2 0 2), etc. (L 4 + L 5) 2 Calculated based on the equation of. For example, the pixel data of another region where there is no pixel data including the pixel data of the column indicated by reference numeral 1 2 0 5 is similarly calculated based on the adjacent left-eye image data.

The interpolation method is not limited to the method using the average value as described above. For example, capture may be performed using a low pass filter of three or more taps. In addition, since the region where interpolation has been performed may cause blurring due to the horizontal aliasing filter due to the capture interval processing, a sharpening filter, a noise removal filter, etc. may be used. It may be possible to obtain a good display. The interpolated stereoscopic image data generated in this manner is output to the display unit 6. As described above, by generating the interpolation image data using the low pass filter, a good image suitable for the two-dimensional display mode can be obtained in the display unit 6. In FIG. 10, display areas 1 0 0 2 and 1 0 0 3 in which no pixel data exist are generated, but these areas may be captured in the same manner as described above.

As described above, the display area determination unit 4 of FIG. 1 determines the presence or absence of pixel data in the area used for actual display, and the area without pixel data is to be interpolated by the image interpolation unit 5. It is determined that In this way, by limiting the capturing area to the display area, the burden of interpolation processing can be reduced.

Further, in the present embodiment, the movement of the image in pixel units has been described as an example, but as described above, the same region is obtained also in the example of movement of the three primary color data in RGB data units. It goes without saying that judgment processing and catch-up processing are possible. In addition, the area interpolated on the display device can be partially Although the technology for switching to a hard disk drive has been described, the above method can also be applied to a stereoscopic display device that does not have a function to switch partially to a two-dimensional display mode. In this case, the captured area is displayed in the conventional display mode.

As described above, according to the stereoscopic image display technology according to the first embodiment of the present invention, in a stereoscopic image in which the amount of parallax is adjusted, a capture image is created and displayed for a region where stereoscopic display is difficult. Thus, it is possible to display a good stereoscopic image with less discomfort without reducing the display area. Next, a stereoscopic image display technique according to a second embodiment of the present invention will be described.

In the stereoscopic image display technology according to the first embodiment, a description is given of a technology for creating inter-capture image data for a display area in which no pixel data is present by adjusting the amount of parallax for the user and two-dimensionally displaying the corresponding area. did. The stereoscopic image display technology according to the present embodiment is a technology for performing good stereoscopic display by displaying other image data in a display area in which no pixel data exists.

FIG. 13 is a block diagram showing a configuration example of a stereoscopic image display apparatus according to the present embodiment. The difference from the stereoscopic image display device shown in FIG. 1 is that an image data generation unit 7 is provided instead of the image acquisition unit 5 (FIG. 1). For example, the image data generation unit 7 includes an area for image data for the left eye corresponding to the area 1 0 0 2 and the area 1 0 0 2 in FIG. 1 0 and an area 1 0 0 3 and an area 1 0 0 3 New image data is inserted in the area of the corresponding right-eye image data. FIG. 14 is a diagram showing how the image data generation unit 7 (FIG. 13) inserts new image data. In FIG. 14 (A), reference numerals 1 4 0 1 and 1 4 0 2 denote portions where there is no pixel data, and reference numerals 1 4 0 3 and 1 4 0 4 denote pairs for stereoscopic display. Data that does not exist. The image data generation unit 7 has these areas (FIG. 14 (A). The new images 1 4 1 1 a and 1 4 1 lb are synthesized as shown in FIG. 1 4 (B) with respect to the region (indicated by the arrows 1 4 0 6 and 1 4 0 7). , Obtain the image shown in Figure 14 (C). The arrow area indicated by reference numeral 1 4 0 5 indicates a display area.

In this way, by newly generating and displaying image data in an area where good stereoscopic viewing can not be performed, the user can reproduce only the location of a good 3D surface image.

The new image used in the present embodiment may be an image for the two-dimensional display mode or an image for the three-dimensional display mode. When an image for 2D display mode is used, the display mode may be partially switched to 2D display mode, but if there is no such function, 3D display is possible. You may play in the mode.

According to the stereoscopic image display technology according to the present embodiment, it is possible to display a good stereoscopic image with less discomfort by generating a new image in a region where stereoscopic display is difficult. Ru.

Next, a stereoscopic image display technique according to a third embodiment of the present invention will be described with reference to the drawings. The stereoscopic image display technology according to the first and second embodiments is display processing of a stereoscopic image when the parallax amount adjustment and the parallax amount adjustment of the stereoscopic image are performed by the user. However, when displaying a plurality of stereoscopic images repeatedly, if the operation of adjusting the amount of parallax is performed again each time, processing becomes complicated for the user. Therefore, to avoid this, in the stereoscopic image display technology according to the present embodiment, the parallax amount adjusted by the user is stored for each stereoscopic image.

Hereinafter, the parallax amount adjustment information is referred to as shift vector. This shift is, for example, a vector indicated by symbol 9 0 6 in FIG. 9 (B). If the contents of the shift vector can represent the adjusted amount of parallax, The format may be For example, a state in which a three-dimensional surface image is captured, that is, a state in which the parallax amount adjustment by the user is not performed is 0, and a movement of a specific image in a specific direction is +, the opposite direction It is also possible to set the amount of adjustment (the number of dots or the number of pixels) as one. In addition, when the size of the shift table is expressed by the number of pixels, if the size of the display for observing a stereoscopic image changes, the amount of parallax also changes, so the appearance (e.g., projection distance) changes. . Therefore, as a shift vector, the projection distance itself of the stereoscopic image is recorded in units of cm, for example, so that the same state can be observed even if the size of the display is changed. It is also good. The projection distance is expressed, for example, with reference to (0) when the amount of parallax is not adjusted. An example of calculation of the projection distance will be described with reference to FIG.

As shown in Fig. 25 (A), let e be the distance between the left and right eyes of the observer and L be the distance between the observer and the display. The state of Fig. 25 (A) is a state in which the pixel observed with the right eye and the pixel observed with the left eye are observed in the same slit (P 1). At this time, this pixel is observed on the display.

Next, move the image observed with the right eye by one pixel in the left direction of the drawing. This state is shown in Fig. 25 (B). The pixel present in P 1 moves to the position of P 2. At this time, pixels observed with the left eye and pixels observed with the right eye are observed with different slits, and an image is formed at the position of S 1 to produce a three-dimensional effect. The distance d from the display at this time is the jumping distance.

Here, as shown in Fig. 25 (B), let w be the movement distance to P1 and P2. w is a value dependent on the device (display). In addition, the distance L between the display and the observer also depends on the display in the case of the parallax barrier method or the lenticule method. Therefore, in order to calculate the projection amount, it is necessary to know the distance L between the display and the observer and the movement distance w. It is important. The distance e between the left and right eyes of the observer is considered to be substantially constant. Assuming these, the distance of projection d can be obtained by the following equation.

e: (L-d) = w: d (1)

Equation (2) is obtained from equation (1).

d = (w X L) / (e + w) (2)

Here, the projection distance d and the movement distance w take positive and negative values. When the projection distance d takes a positive value, it indicates that the user seems to pop out from the state before adjusting the amount of parallax, and when it takes a negative value, the user adjusts the amount of parallax Show what you can see from the state you did before. Also, the moving distance w takes a positive value when moving the image observed with the right eye to the left, and takes a negative value when moving the image observed with the right eye to the right. In Fig. 25 (B), both w and d have positive values, so they appear as if they were jumping by a distance d. On the other hand, Fig. 25 (C) shows the image of Fig. 25 (A) in which the image observed with the right eye is shifted by one pixel to the right. The pixel present at P 1 moves to the position of P 3 and forms an image at the position of S 2. At this time, the movement distance w takes a negative value, so the projection distance d becomes a negative value according to the above equation (2). Therefore, compared to Fig. 25 (A), the image can be seen in the back in Fig. 25 (C). Also, when moving the image observed with the left eye, the positive and negative values of the moving distance w are the reverse when moving the image observed with the right eye. In the stereoscopic image display technology according to the first embodiment of the present invention, the display area is automatically set. However, when the user arbitrarily sets the display area, the setting is made in addition to the shift vector. Record the information of the displayed area (display area information) and use it for the next display. The display area information indicates, for example, by focusing on the image for the right eye, the amount of movement of the left end of the image for the right eye from the left end of the initial display area.

In the stereoscopic image display technology according to the first embodiment of the present invention, the display area is If the display area has been set automatically, but the user arbitrarily sets the display area, the information on the display area (display area information) set in addition to the shift vector is recorded, and it will be displayed at the next display. Use The display area information is indicated, for example, by focusing on the image for the right eye, and how far the left end of the image for the right eye has moved from the left end of the initial display area.

The stereoscopic image display technology based on the above principle will be described more specifically. FIG. 15 is a functional block diagram showing a configuration example of a stereoscopic image display apparatus according to the present embodiment. As shown in FIG. 15, the stereoscopic image display apparatus according to the present embodiment is, like the stereoscopic image display apparatus shown in FIG. 1, image processing so that the input image can be displayed stereoscopically. A stereoscopic image processing unit 1 for performing the process, a user input unit 2 for performing an input from the user, a parallax amount adjustment information calculating unit 3 for calculating adjustment information of the parallax amount from the user's input; A display area determination unit 4 that determines an area to be actually displayed on a display from an image, an image interpolation unit 5 that generates interpolation data in a specific area, and a display unit that displays the determined area of the generated image 6 Includes and. The difference with Fig. 1 is that a stereoscopic information recording unit 8 is provided. The basic operations of the other parts are the same as those described in the above embodiments.

In FIG. 15, the parallax amount adjustment information (shift vector) calculated by the parallax amount adjustment information calculation unit 3 is output to the stereoscopic image processing unit 1 and the stereoscopic information recording unit 8 is also output. (Sign 1 5 0 1). The display area determination unit 4 determines the display area, and further, changes the display area when the user arbitrarily sets the display area, and changes the display area information after the change to the stereoscopic information recording unit 8. Output (sign 1 5 0 2). The stereoscopic information recording unit 8 records (stores) shift vectors, and also records display area information as necessary.

The following is an example of how to record shift vectors and display area information. In the present embodiment, shift vectors and display area information are recorded in the stereoscopic image data. Provide an area for In general, image data is provided with an area for storing information for managing the size, playback time, etc. of the image. Figures 16 (A) and (B) show examples of the data structure of image data. As shown in FIG. 16 (A), the stereoscopic image data is, for example, a management information area for managing image information such as reproduction time-image size, and images for left eye and right eye. image data area and a include _c first 6 is configured to view recording data Remind as (a), the in the image information 1 6 0 1, if the image size and video, such as the reproduction time Information on the entire image is described, and right-eye 'left-eye image data information 1 6 0 2 · 1 6 0 3 contains information necessary for decoding each image data (for example, as a coding technique) Information such as MPEG-4 technology is used. Further, as shown in FIG. 16 (B), an area 1604 for recording information (stereoscopic information) for stereoscopic image data is provided in this management area, and shift vectors and Record the display area position.

When recording, a header is required to indicate the presence of stereoscopic information so that the device can read these pieces of information correctly. For this reason, stereoscopic information 1

At the beginning of 6 0 4, set up an area for recording 3D (3 D) image identification information 1 0 5. The stereoscopic image identification information 1 6 0 5 indicates the presence of stereoscopic vision information and also indicates that the image data to follow is stereoscopic vision image data. The stereoscopic image identification information 1 6 0 5 may be a flag encoded by a fixed-length or variable-length code, but may be, for example, a specific symbol string or character string as long as it can be identified.

In the stereoscopic information 1 6 0 4, there are information on shift vectors and display area positions in addition to the stereoscopic image identification information 1 6 0 5. For example, for long-term standing vision, the eye In order to impose a burden, if you limit the continuous viewing time, you may record the available viewing time together. These information in three dimensions W

It is called visual control information and is recorded in the area indicated by reference numeral 1 6 0 6. A configuration example of stereo image identification information and stereo vision control information is shown in Fig. 16 (B). As shown in FIG. 1 6 (B), stereoscopic vision control information 1 6 0 6 is recorded following stereo image identification information 1 6 0 5.

The stereoscopic control information will be specifically described with reference to Figs. (A) and (B).

The shift vector represents the number of pixels to be moved. As shown in Figs. 1 (A) and (B), focusing on the right-eye image data, the right-eye image data is moved by 4 pixels in the direction in which the parallax spreads (right direction). , Shift vector 1 7 0 4 is 4. The display area 1 7 0 3 moves from the original display area by 2 pixels in the horizontal direction (indicated by the arrow of 1 7 0 1) with the left end of the image as the reference position. This is represented by vector 1 7 0 5. The components of this vector are 2 in the horizontal direction and 0 in the vertical direction (1702), and are represented by (2, 0) as display area information in FIG. 17 (A). .

In the above, the case of recording the stereoscopic control information in the management information area attached to the image data has been described as an example, but the stereoscopic control information may be recorded in a predetermined storage unit of the display device. .

An example of shift vectors and display area information stored in a predetermined recording area by the parallax amount information recording unit 8 is shown in FIG. As shown in FIG. 18, the information storage unit for parallax amount information has a file name such as content A, B and C, and information for identifying stereoscopic image data together with the stereoscopic image. Visual control information (shift vector and display area information) is tabulated in association with file names. For example, the content A has shift vector (number of pixels) of 4 and display area information (number of pixels) of (2, 0). In Fig. 18, shift vectors are expressed as one-dimensional vectors.

In the stereoscopic image display technology according to the present embodiment, shift vectors and Although the display area information is described by taking an integer pixel as an example, the present invention is not limited to this and displays a shift solid and display area information on a non-integer pixel unit. It can also be used as information. Further, in the stereoscopic image display technology according to the present embodiment, the shift vector may be the number of pixels for moving the right-eye image / left-eye image in the horizontal direction or the amount of change in the projection distance of stereoscopic vision. . Therefore, information or a flag indicating whether the shift vector is expressed in the form of the number of pixels to be moved or the projection distance of stereoscopic vision may be recorded in the stereoscopic control information. Also, although an example has been described in which both the shift vector and the display area information are recorded as stereoscopic control information, only the shift vector is recorded, and a predetermined area or automatic area is automatically recorded. The area to be determined may be a display area. Alternatively, only display area information may be recorded. Also, other information for stereoscopic vision may be recorded.

Further, in the present embodiment, an example has been shown in which the stereoscopic image identification information and the stereoscopic control information are recorded at only one place at the beginning of the image file. However, as long as extraction is possible, it may be recorded in any area of the image file, or may be recorded in the image data. For example, when an image is encoded according to M PEG-4, the image data includes encoded data and information (header information) for decoding the encoded data. This header information is provided with an area (user's area) that can be freely used by the user, and stereoscopic image identification information and stereoscopic control information may be recorded in this user area. Also, a plurality of stereoscopic image identification information and stereoscopic control information may be present in image data. This makes it possible, for example, to change the amount of parallax in the middle of image data.

Next, a procedure for reproducing image data by the stereoscopic image display technology according to the third embodiment will be described. Here, in the image data, Fig. 1 7 It is assumed that the stereoscopic image identification information shown in and the stereoscopic control information are recorded.

FIG. 19 is a functional block diagram showing a configuration example of a stereoscopic image display device for displaying stereoscopic image data to which stereoscopic control information is added according to the present embodiment. In addition to the configuration shown in FIG. 1, a stereoscopic information reading unit 9 is provided. The stereoscopic information reading unit 9 reads out the stereoscopic image identification information included in the input image data. When the stereoscopic image is recognized by the confirmation of the stereoscopic image identification information, the stereoscopic information reading unit 9 reads the stereoscopic control information (shift vector and display area information), and the stereoscopic image is read. Decode if visual information is encoded. If there is shift vector in the stereoscopic control information, it is output to the parallax amount adjustment information calculation unit 3, and if there is a region display position, it is output to the display region determination unit 4.

The parallax amount adjustment information calculation unit 3 uses the shift vector input from the stereoscopic information reading unit 9 and the data for parallax amount adjustment input from the user input unit 2 to generate the parallax amount adjustment information. calculate. If there is no adjustment of the amount of parallax by user input, the amount of parallax adjustment information is calculated from shift only. The stereoscopic image processing unit 1 generates stereoscopic display image data using the parallax amount adjustment information.

Further, the display area determination unit 4 determines a new display area in accordance with the display area information and the parallax amount adjustment information input from the stereoscopic information reading unit 9. When there is no parallax amount adjustment by user input, the parallax amount adjustment information from the stereoscopic information reading unit 9 is used as it is.

As described above, when the user repeatedly reproduces the same stereoscopic image data, the adjustment made by the user at the time of the previous reproduction is stored as stereoscopic control information, and the stereoscopic control is performed at the time of reproduction. By using the information, it is not necessary to adjust the amount of parallax each time playback is performed, as in the previous playback. It becomes possible to reproduce in the same way.

As described above, according to the third embodiment of the present invention, the shift amount is adjusted again when viewing the same content by recording image data, that is, a shift vector related to the content. This saves time and makes it possible to record the appropriate shift vector for each content.

Next, a stereoscopic image display technique according to a fourth embodiment of the present invention will be described with reference to the drawings. Adjustment of the appropriate amount of parallax is effective because it can be reproduced in better condition for the user. However, if the adjustment method is incorrect, it may not be possible to view stereoscopically well. In the present embodiment, in order to avoid this, the shift vector maximum value (maximum shift vector) is set, and the amount of parallax that can be adjusted by the user is limited. The maximum shift vector may be included in the management area of the image data (stereoscopic control information of FIG. 16 (B)), or the predetermined recording area of the display device according to the present embodiment. It may be recorded in

An example of the description of the maximum shift vector is shown in Fig. 20 (A) and (B). In Fig. 20 (A) and (B), the maximum shift vector is described in the same description method as the shift vector described above. In other words, it is expressed as the number of dots to move or the distance to fly. Fig. 20 (A) is an example of recording in image data, and the maximum shift vector is set to 10. Fig. 20 (B) is an example of recording in a table format in a predetermined storage section of the display unit, and the maximum shift vector value is determined for each content. For example, in content A, the maximum shift value is determined. The topic vector is 1 0. Although the example of FIGS. 20 (A;) and (B) shows an example including not only the maximum shift vector but also information for other stereoscopic vision, the same area is not necessarily the same. It does not have to be included in. FIG. 21 shows an example of the configuration of an apparatus for reproducing stereoscopic image data according to the present embodiment. As shown in FIG. 2 1, the stereoscopic information readout unit 9 shown in FIG. Is replaced with the maximum shift reading unit 10. The maximum shift vector readout unit 10 reads the maximum shift vector from the stereoscopic control information. If stereoscopic control information exists in the same file as the input image data, the information on the largest shift vector is extracted from the file. If the file is stored in a storage unit independent of the file, the shift vector corresponding to the input image data is read out from the storage unit, and maximum shift vector information is extracted.

The extracted maximum shift vector is output to the parallax amount adjustment information calculation unit 3. The parallax amount adjustment information calculation unit 3 determines whether or not the shift vector calculated from the user input is within the range of the maximum shift vector, and if it is within the range, the calculated shift vector Toru is sent to stereo image processing unit 1 as it is. If the shift vector is larger than the maximum shift vector, the maximum shift vector is output as the shift vector. As a result, display using a large shift vector that would make stereoscopic vision difficult is not performed, and good stereoscopic display is possible. Note that the maximum shift vector can define both positive and negative values, and the absolute value of the positive value does not have to be equal to the absolute value of the negative value. For example, the maximum shift vector (1 dimension vector in this example) for the content A in FIG. 20 (B)

For example, if (1 7) is defined together with (+ 1 0), shift vectors calculated from user input are limited to values between (1 7) force and (+ 1 0). It will be

If the shift vector calculated from the user input falls outside the range of the above-mentioned maximum shift vector, that shift vector may not be used. For example, if the shift vector by user input is calculated to be 20 at the time of reproduction of content A in FIG. 20 (B), the shift vector used for display is the shift vector value “4”. I will leave. Also, display 6 If the display mode can be switched, if the shift vector input by the user exceeds the maximum shift vector, display the entire image in 2D display mode. You may

In this embodiment, in order to extract the maximum shift vector, the maximum shift vector readout unit 10 is provided, but the function capable of executing the same processing is shown in FIG. According to the fourth embodiment of the present invention, setting may be made such that stereoscopic vision becomes difficult by setting the maximum shift vector. It is possible to prohibit the adjustment of the amount of parallax, and it is possible to generate a good stereoscopic image without damaging the content creator's intention.

Next, a stereoscopic image display apparatus according to a fifth embodiment of the present invention will be described with reference to the drawings. When performing image playback using stereoscopic image data in which a shift vector and display area are set, or when adjusting the amount of parallax of a stereoscopic image during image playback, image enlargement / reduction When reproducing stereoscopic image data by using image processing such as, etc., the shift vector may differ from the number of pixels that should actually be shifted. In such a case, it is better to execute enlargement / reduction processing for the shift vector as well as the image enlargement / reduction processing. This situation will be described with reference to FIGS. 22 (A) to 22 (E).

Let the image data shown in Figure 22 (A) be the input image data, and let the image data shown in Figure 22 (B) be the image data for which the amount of parallax adjustment (1 pixel shift) has been performed. On the other hand, the image data shown in Figure 22 (C) is an image obtained by performing enlargement processing (double enlargement in the horizontal direction) of the input image data, and the image shown in Figure 22 (D) The data is the image data shown in Fig. 22 (C) with the amount of parallax adjusted (1 pixel shift). When the shift vector is expressed in pixel units, the image after enlargement in this way is also shown in Fig. 22 (D). You will have to make some adjustments, but this is different from the adjustment the user intended. Therefore, the technology according to the present embodiment also performs enlargement / reduction of stereoscopic control information such as shift vectors in accordance with the enlargement / reduction of the image. FIG. 23 is a functional block diagram showing a configuration example of a stereoscopic image display apparatus according to the present embodiment. In FIG. 23, in addition to the configuration shown in FIG. 19, an enlargement / reduction processing unit 1 1 for performing enlargement / reduction processing of an image and a stereoscopic control information conversion unit 1 2 are provided. When enlargement and reduction are performed, the enlargement and reduction processing unit 11 performs enlargement and reduction processing of the input image data, and the enlarged and reduced image data is output to the image processing unit 1. At that time, the enlargement-reduction processing unit 1 1 outputs the enlargement / reduction ratio to the stereoscopic control information conversion unit 1 2.

The stereoscopic control information accompanying the image data is read out by the stereoscopic information reading unit 9. The stereoscopic information reading unit 9 according to the present embodiment reads the maximum shift vector in addition to the shift vector and the display area information. The stereoscopic control information read out is output to the parallax amount adjustment information calculation unit 3 and the display area determination unit 4.

The parallax amount adjustment information calculated by the parallax amount adjustment information calculation unit 3 is converted by multiplying the enlargement / reduction ratio of the input image data in the stereoscopic control information conversion unit 1 2. For example, when the amount of parallax adjustment information is one-dimensional vector (13) and the enlargement / reduction ratio is 2, the amount of parallax adjustment information is converted to (16). Similarly, the display area information generated by the display area determination unit 4 is converted in accordance with the enlargement / reduction ratio in the stereoscopic control information conversion unit 1 2.

When the stereoscopic image display technology according to the present embodiment is used, the parallax amount adjustment is performed according to the enlargement / reduction ratio as shown in FIG. 22 (E) instead of FIG. 22 (D). In addition, the user can observe the appropriate image. According to the fifth embodiment of the present invention, shift vector adjustment for parallax amount adjustment When the stereo image is scaled up or down, correction is performed according to the scaling up / down rate, so even when the 3D surface image data for which the amount of parallax is adjusted is scaled up / down, It is possible to generate a good stereoscopic image. Next, a stereoscopic image display technique according to a sixth embodiment of the present invention will be described with reference to the drawings. The technology in each of the above-described embodiments is a technology for storing stereoscopic vision information for stereoscopic image data stored in a file. However, when transmitting image data for stereoscopic viewing as broadcast content such as BS broadcasting and terrestrial digital broadcasting, it is necessary to store and transmit stereoscopic information in a method suitable for broadcasting. For example, even when the user changes channels and starts receiving new broadcast content, it is necessary to be able to obtain such stereo information.

In the case of BS broadcasting or terrestrial digital broadcasting, program arrangement information for managing what kind of program is being broadcast is, for example, multiplexed with content as shown in FIG. 24 (A). Has been broadcast. Broadcast content contains data of multiple contents, and the information indicating the contents is program arrangement information. Program arrangement information describes information for separating / identifying video signals and audio signals constituting each content from broadcast content (PMT: Program Mapable) and the content of each content. Program guide information (EIT: EV Information T ab 1 e) and so on.

The program arrangement information is repeatedly multiplexed and transmitted in the broadcast content so that channel switching in the receiver can be coped with at any time. In the stereoscopic image display technology according to the present embodiment, the program arrangement information includes stereoscopic information.

Figure 24 (B) shows an example in which stereoscopic information is incorporated into program arrangement information. Here, the stereoscopic information is described in the third embodiment of the present invention. In addition, it is assumed that stereoscopic image identification information and stereoscopic control information are included. That is, the stereoscopic image identification information indicates that the content is a stereoscopic image, and the stereoscopic control information includes information such as shift vectors, display area information, and maximum shift vectors.

The receiver uses stereoscopic image identification information from the above broadcast content to determine that the content is a stereoscopic image, and extracts stereoscopic control information. Then, as in the other embodiments described above, shift vectors and the like are acquired from stereoscopic control information, and stereoscopic image data is generated and displayed.

Although information such as shift vectors and area display positions in the stereoscopic control information has been described as recording data when the user changes the stereoscopic effect of an image, these information May be set by the broadcaster. For example, when the broadcast starts, the shift vector value is set to 0, and then the broadcast station sets the shift vector value to a value other than 0 on the broadcast side independently or at the request of the user. Broadcast is also possible. Also, according to the content creator's intention, it is possible to set the maximum shift vector on the broadcast side and include it in the stereoscopic control information, and use it for the restriction when adjusting the amount of parallax at the receiver. .

Further, although the stereoscopic image identification information and the stereoscopic control information are described as being included in the program arrangement information, these information may be included in image data as well as the program arrangement information. For example, when broadcast content is encoded in MPEG-4, stereoscopic image identification information and stereoscopic control information may be included in the user area as described above. In this case as well, in order to enable viewing of content from the middle of a program, stereoscopic image identification information and stereoscopic control information must be periodically included in the image data. To include stereo image identification information and stereo vision control information in the information for this decoding. May.

According to the sixth embodiment of the present invention, the adjustment of the parallax amount is performed even when the content of the stereoscopic image is broadcasted by including the stereoscopic information in the program arrangement information such as the BS broadcast and the terrestrial digital broadcast. You can change the display position, and adjust the amount of parallax adjustment.

Next, a stereoscopic image display technique according to a seventh embodiment of the present invention will be described with reference to the drawings. In the third embodiment, an example in which the number of pixels to be moved in the horizontal direction is specified as a shift vector, which is information for adjusting the amount of parallax, is given as another example. An example of specifying the direction and distance of the stereoscopic image, an example of specifying the direction and distance to move the image, and an example of specifying the parallax angle of the stereoscopic image are shown below.

As shown in FIG. 2 5, regarding the image observed with the right eye, when the pixel positioned at P 1 is moved so as to be positioned at P 2 or P 3, the appearance of the stereoscopic image is Is that pixels connected to the image at the position P 1 on the display surface before movement, and images to the positions S 1 and S 2 separated by the distance I d I from the display surface after movement become. Here, I d I is an amount expressed in cm equal-length units. d itself takes positive and negative values, for example, as shown in FIG. 25 (B), if the image is far in front of the display surface, as shown in FIG. If the image is far from the display surface, the negative value is taken.

d shall be the quantity which is known under the standard environment, that is, the “standard observation environment” where the display characteristics and observation conditions are known. In the stereoscopic image display apparatus, the adjustment value of the parallax amount actually required is calculated based on this d. By adopting such a configuration, the viewer's impression and influence received from the stereoscopic image displayed as a result of the parallax amount adjustment, the characteristics of the display (display) and the viewing conditions are variously different. It becomes possible to control the same degree among multiple stereoscopic image display devices. The following is based on the above d An example of a method of calculating the adjustment value of the parallax amount will be described.

The state shown in Fig. 25 (B) will be described as an example. As described above, if the right-eye image is horizontally moved so that the right-eye pixel P 1 on the display surface is at a position P 2 horizontally separated by a distance w, the display is performed before the movement. The image connected at position P 1 on the surface is connected at position S 1 at a distance d from the display surface. At this time, d is given as in the above equation (2).

Here, the variables d, L, w and e used in the equation (2) are all values in the above-mentioned standard observation environment. Similarly, in any three-dimensional image display device, the relationship of the above equation (2) is established. However, d, L and w excluding e are usually different from the standard observation environment. Here, these variables in any three-dimensional image display device are denoted as d ′, L ′, w ′. That is, the following equation (2 ') holds.

d '= (w' X L ') No (e + w,) ... (2') e is constant regardless of the observation conditions. Solving for d 'from Eq. (2) and Eq. (2') gives the relationship of Eq. (3) below. Here, W and W ′ represent the image display width W on the display in the standard observation environment and the image display width W ′ on the display in any of the above-described three-dimensional image display devices, respectively. It is assumed that (W: W '= w: w').

d '= L' XW 'X d / (LXW + (W-W) X d) ... (3) If d' is obtained as in the above equation (3), the equation (2 ') You get w 'from the relationship. Since w 'represents the movement distance of the image on the display surface, h' can be obtained from the following equation (4), where h 'is the number of pixels actually required to be moved. Here, p 'is the horizontal pixel pitch (distance between adjacent pixels) of the display. h '= w' / p '· · · (4)

As described above, if parameters such as L, W, d, etc. in the standard observation environment are known, it is possible to calculate the adjustment value of the amount of parallax necessary for any three-dimensional image display device. is there.

Among the parameters, L and W are parameters whose values are fixed if a standard observation environment is defined. Therefore, it is not necessary to specify for each stereoscopic image data, and stored in advance in each stereoscopic image display device. Just do it. Therefore, in order to adjust the amount of parallax, the parameter corresponding to d may be associated with the stereoscopic image data and recorded or transmitted. When recording or transmitting a parameter corresponding to d, the parameter may be composed of two parameters: a parameter indicating the positive or negative direction of d and a parameter indicating the absolute value of the distance. It may be composed of only one parameter that has both positive and negative values.

Fig. 26 (a), (b) shows a configuration example of the stereoscopic vision information including the parameter corresponding to the d. In the example of FIG. 2 6 (a), d is represented by two parameters of popping direction 2 6 0 1 and popping distance 2 6 0 2, and in the example of FIG. 2 6 (b), d is It is expressed by only the jumping vector 2 6 0 3 including the direction and distance. The parameter indicating d may be a parameter indicating the projection position (or retraction position) from the display surface as described above, or the pixel at an arbitrary reference position away from the display surface. It may be a parameter that represents the amount of movement in the direction orthogonal to the display surface.

The above d can be considered as either a value for specifying the pop-out amount to be adjusted or a value for specifying the allowable maximum value of the pop-out amount. Therefore, when recording or transmitting a parameter corresponding to d, which one is determined in advance In addition, it may be necessary to separately include parameters indicating which of these values. Alternatively, both may be included in the stereoscopic control information.

As another example of information for adjusting the amount of parallax, an example of specifying the direction and distance for moving the image is shown below.

As described above, assuming that the relationship of (W: W '= w: w') is satisfied, if the value of W is known in advance as the condition of the standard observation environment, then w is If given, it is possible to calculate w 'in a stereoscopic image display device with the condition of W'. Therefore, a parameter indicating w may be specified as information for adjusting the amount of parallax. As a parameter indicating w, similarly to d, it may be composed of two parameters of a value indicating the positive or negative direction of w and a value indicating the absolute value of the distance, or both of them may be positive or negative It may consist of only one parameter with a value. The direction of w at this time is, for example, positive when moving the image for the right eye to the right, and negative when moving the image for the left. FIG. 26 (c) shows a configuration example of the stereoscopic information including the parameter indicating w. Here, the translation vector 2 6 0 5 corresponds to the parameter indicating w.

As another example of information for adjusting the amount of parallax, an example of specifying the angle of parallax of a stereoscopic image is shown below.

As shown in FIG. 2 5, when the pixel located at P 1 moves so as to be located at P 2 or P 3, the angle at which the parallax is 0 with respect to the position where the image is formed is 0 1 , Θ 2 change.

If these angles are defined as values in a standard viewing environment, it is possible to calculate the parallax angle 0 ′ in any stereoscopic image display device from the relationship of “S” and the relationship between W and W ′. It is possible. Therefore, a parameter indicating 0 n may be specified as information for adjusting the amount of parallax. As a parameter indicating Θ n, it indicates the absolute value of the angle as represented by 0 1 or 0 2 It may be composed of parameters, as shown in FIG. 25 (A). A parameter indicating the difference between 0 with no view and these 0 1 and 0 2 (ie, θ 1 1 0 Or 0 2 — a value representing 0). FIGS. 2D (d) and (e) show examples of the configuration of the stereoscopic information including the parameter indicating 0 n or the parameter indicating 0 1 − 0 or 0 2 − 0. Here, the viewing angle 2 6 0 6 and the parallax angle displacement 2 6 0 7 correspond to the parameter representing the S n or the parameter representing the 0 1-0 or 0 2-0.

Next, how to determine the display area as a result of the above parallax amount adjustment will be described. As described above, if the image is moved to adjust the amount of parallax, the width of the image that can be displayed as a stereoscopic image changes, so the display area can not be uniquely determined. In the third embodiment, an example in which the user arbitrarily sets this display area and records it as display area information is given. An example in which flat display image selection information is used instead of the display area information will be described below.

The flat display image selection information is information indicating which of the left eye image and the right eye image is used when displaying the stereoscopic image data as a flat image. Since the image data for stereoscopic vision includes the image for the left eye and the image for the right eye as described above, it is possible to display any image as it is as a plane image. Of course, it may be displayed after adding capture processing or capture processing instead.

Planar display image selection information is included in stereoscopic control information as shown in FIG. The flat display image selection information may record the result set by the user in the same manner as the display area information, or, as described in the sixth embodiment, is included in the stereoscopic information in advance. May be

The flat display image selection information is interpreted by the display area determination unit 4 in the stereoscopic image display device, and the display of the stereoscopic image in which the parallax amount adjustment is performed. Used to determine the area. That is, the display area determination unit 4 displays the parallax so as to display the entire image designated by the planar display image selection information in the stereoscopic vision control information, that is, the image for the left eye or the image for the right eye. Determine the display area of the volume adjusted stereo image. Note that this operation is not limited to the stereoscopic image display apparatus configured as described in all the above-described embodiments (FIG. 1, FIG. 13. FIG. 13, FIG. 15. FIG. 19, FIG. 19. FIG. Applicable for).

The display state on the display section 6 when the display area is determined as described above is shown in FIG. 27. Figure 2 7 shows that (a) does not adjust the amount of parallax, and (b) and (c) both shift the right-eye image to the right with respect to the left-eye image. It is in a state of Because of the shift, as shown in the figure, the horizontal positions of the left eye image and the right eye image do not match. An area in which both the left-eye image and the right-eye image (binocular image) exist is an area in which stereoscopic display can be performed as it is. An area in which only one of the left-eye image and the right-eye image (one-eye image) exists is an area in which stereoscopic display can not be performed as it is. The area of the one-eye image may be a plane image by showing the same image to both eyes of the observer, or generating or capturing the other one-eye image based on the one-eye image. Image processing of may be applied to make a stereoscopic image. At this time, the display area is determined so as to coincide with the position of the image specified by the flat display image selection information. Fig. 27 (b) shows the relationship between the position of the left and right images and the display area in the example in which the "image for the left eye" is specified in the planar display image selection information, and Fig. 27 (c) shows the relation It shows the relationship between the position of the left and right images and the display area in the example in which “image for right eye” is specified in the planar display image selection information.

The following effects can be obtained by determining the display territory as described above. That is, as described above, in the case of planar display in a stereoscopic image display device, according to the planar display image selection information, an image for the left eye or an image for the right eye Display using only the image. For example, when “image for left eye” is specified in the flat display image selection information, the entire image for left eye is displayed. At this time, even if the display mode is switched to stereoscopic image display, the display area is selected so that the entire left-eye image is displayed as well, so the difference between planar display and Z stereoscopic display is However, the position of the image for the left eye appears to be fixed to the observer, and as a result, it is possible to reduce the discomfort of the display mode switching.

In addition, in the stereoscopic image display apparatus capable of switching between stereoscopic display and planar display as described above, the display unit 6 has a stereoscopic display mode and a planar display mode, and it is possible to switch between the two. Is a possible display device. Switching between the stereoscopic display mode and the planar display mode may be switched by the user of the stereoscopic image display apparatus by an operation such as a button, or the stereoscopic display as described in the sixth embodiment. Information will be transmitted repeatedly

In the above case, parameters for switching between stereoscopic display and planar display may be included in the stereoscopic control information. In addition, it is configured to switch between stereoscopic display and planar display also according to the presence or absence of stereoscopic image identification information itself, that is, stereoscopic information itself. The display mode may be switched over the entire display screen or may be switched in any display range unit.

Next, a stereoscopic image display technique according to an eighth embodiment of the present invention will be described with reference to the drawings. As described above, in order to change or adjust the amount of parallax at the time of stereoscopic display, it is necessary to move the display position of at least one of the plurality of images. In each of the embodiments described above, it is not specified as a parameter whether to move the display position or an image of an object not to be moved, but the stereoscopic image display technology of this embodiment designates this. It is characterized by this.

The stereoscopic control information of this embodiment is, as shown in FIG. In addition to the parameters that specify the adjustment amount, that is, the projection direction and distance, the projection vector, the translation vector, the parallax angle, and the parallax angle displacement, include the parallax amount adjustment reference image 2 7 0 1 Is a feature. The parallax amount adjustment reference image 2 7 0 1 is a parameter for specifying an image used as a reference when adjusting the parallax amount, that is, an image whose display position is fixed and is not moved. Using this parameter, a user who makes or adjusts a 3D image can adaptively select an image to be the reference of the display range at the time of adjusting the amount of difference. As a means of specification, it is sufficient to use viewpoint numbers corresponding to a plurality of images. For example, when recording or transmitting stereoscopic image data composed of two viewpoints of an image for the left eye and an image for the right eye, the adjustment amount of the parallax amount is designated by the parameters such as the projection direction and distance. At the same time, either the image for the left eye or the image for the right eye is specified by the parallax amount adjustment reference image parameter. In the case of two viewpoints, as the parallax amount adjustment reference image parameter, for example, binary 0/1 can be used to specify an image for the left eye and an image for the right eye.

In addition, it is possible to leave the selection of the reference image to the display device side by specifying a value that means “indefinite”.

In the stereoscopic image display device, the parallax amount is adjusted by moving the display positions of the other images while the display position of the image specified by the parallax amount adjustment reference image parameter is fixed. Specifically, the parallax amount adjustment information calculation unit 3 calculates the parallax amount adjustment information based on the stereoscopic control information including the parallax amount adjustment reference image, and the stereoscopic image processing unit 1 adjusts the parallax amount accordingly. Generate an image. This operation is a stereoscopic image display apparatus configured as described in all the above embodiments (FIG. 1, FIG. 13; FIG. 15. FIG. 15. FIG. 19. FIG. 19. FIG. 21). It is applicable about. By generating a stereoscopic image in this manner, an image displayed as a stereoscopic image when the parallax amount adjustment is performed between a plurality of different stereoscopic image display devices of one stereoscopic image data. It becomes possible to make the areas of the image identical to each other, and it is possible to appropriately reflect the creator's intention of the stereo image data regarding the range of the image display.

As another means for obtaining the same effect, instead of using the parallax amount adjustment reference image parameter, a method of designating the movement direction and the movement amount of each of a plurality of images can be considered. That is, Fig. 26 (c) and Fig. 28

The parameters corresponding to the translation vectors 2 6 0 5 and 2 7 0 5 in (c) should be provided for each of the plurality of images. Then, if the translation vector corresponding to a specific image is specified as 0, that image will not be moved, so it is the same as specifying a parallax adjustment reference image. The effect is obtained. For example, in the case of a stereoscopic image with two viewpoints, the left eye image can be used as a reference image by designating 0 as the translation vector of the left eye image.

The stereoscopic control information according to the present embodiment may further include the flat display image selection information 2 6 0 4 shown in the seventh embodiment. In this case, the meaning of the flat display image selection information is different from the meaning in the seventh embodiment. That is, in the seventh embodiment, the display area of the stereoscopic image is also determined in conjunction with the flat display image selection information. However, in the present embodiment, the display area in the flat display mode and the stereoscopic display mode are determined. Each display area in can be specified and controlled independently.

Next, a stereoscopic image recording technique according to a ninth embodiment of the present invention will be described. In the third embodiment or the seventh embodiment of the present invention, an example in which information for adjusting the amount of difference is specified is given. The stereoscopic image recording technology according to the present embodiment is one form of a recording method including such stereoscopic vision information, and a recording method for recording a stereoscopic image and stereoscopic vision information on a digital video tape, and It is a recording device for that. · First, the track format of the digital video tape recorded by the stereoscopic image recording technology according to the present embodiment will be described. A commonly used digital VTR employs a method called “helical scan”. As shown in Figure 29, in this method data is recorded for discrete tracks on the tape. This is shown in Fig.29. A plurality of tracks 2800 is formed on the tape 2800, and a single stereoscopic image is also a plurality of tracks 2 It is divided into 8 0 1 and recorded. The running direction of the tape is from right to left (arrow direction) in Fig. 29. The leftmost track is scanned from the bottom to the top, and then the next track to the right is scanned. It is scanned from the bottom to the top.

FIG. 30 is a diagram showing an example of the configuration of one track 2801, and the track format of the digital VTR recorded by the stereoscopic image recording technology according to the present embodiment. 1 shows an example of one configuration. The track 2 8 0 1 is an ITI (Insert and Trap Information) area 2 9 0 1 for securing an afreco, an audio recording area 2 9 0 2 in which data relating to audio is recorded, and an image And the sub code recording area 2 9 0 4 where accompanying information such as the time code is recorded. In the image recording area 2903, not only the stereoscopic image itself but also the accompanying information related to the stereoscopic image can be recorded. Similarly, not only the audio itself but also the accompanying information related to the audio can be recorded in the audio recording area 2 9 0 2. In addition to the above two, incidental information can also be recorded in the subcode recording area 2904 as described above. In addition, there is a margin between each area, and it is possible to individually make an Aleco.

FIG. 31 is an enlarged view of the image recording area 2 9 0 3. The image recording area 2 9 0 3 is a preamble 3 0 0 1 in which synchronization patterns and the like are recorded. The additional information about the image is recorded VAUX (V ideo AUX i 1 iarydata) a 3 0 0 2 及 V AU X j 3 3 0 0 4 and the image coded data recording area where the image coded data is recorded 3 0 It contains 0 3, an error correction code 3 0 0 5, and a boss amble 3 0 0 6 with a function to obtain a margin. In the stereoscopic image recording technology according to the present embodiment, the area in which the incidental information on the image is recorded is divided into two areas of VAUX 3002 and V AU X: 8300 4. Hereafter, these two areas are collectively called V AU X area.

Also, although not shown, an A AU X (A u d o i o A u x i 1 i a i r y d a a a) area is prepared in the voice recording area 2 0 0 0 2 as an area for recording additional information related to voice.

Subsequently, a recording apparatus according to the present embodiment will be described with reference to FIG. FIG. 32 is a block diagram showing the configuration of the recording apparatus of the present embodiment. As shown in FIG. 32, the present recording apparatus comprises a stereo image coding unit 3 1 0 1, a speech coding unit 3 1 0 2 2, an accompanying information coding unit 3 1 0 3, a multiplexing unit 3 1 0 4 , Includes tape recording unit 3 1 0 5.

The stereoscopic image coding unit 3 1 0 1 receives stereoscopic image data as input. The stereoscopic image data is image data capable of stereoscopic display generated based on a plurality of images, as described in the first embodiment and the like. The stereoscopic image encoding unit 3101 encodes this stereoscopic image data by a predetermined method, and outputs stereoscopic image encoded data.

The speech coding unit 3 1 0 2 takes speech data as input, encodes it, and outputs speech coded data.

The incidental information coding unit 3 1 0 3 is used for stereoscopic vision information relating to the stereoscopic vision image data, that is, stereoscopic image identification information for indicating that it is a stereoscopic image, shift vector for adjusting the amount of visual difference. Or direction and distance, display Encodes accompanying information including plane display image selection information and the like for determining a region, and outputs accompanying information encoded data. As an encoding method here, conversion into fixed-length numerical values corresponding to each information can be mentioned. Multiplexing section 3101 receives stereo image encoded data, audio encoded data, and accompanying information encoded data as input, multiplexes these into a format that can be recorded on tape, and outputs data for tape recording. .

The tape recording unit 3105 records the data for tape recording on the tape which is the recording medium in accordance with the format shown above.

Subsequently, the multiplexing unit 3104 will be described in more detail with reference to FIG.

As shown in FIG. 33, the multiplexing unit 3104 has an accompanying information coded data distribution unit 3205, an image recording area data synthesis unit 3201, and a voice. It includes a recording area data synthesis unit 3202, a subcode recording area data synthesis unit 3203, and a track synthesis unit 3204.

The incidental information encoded data distribution unit 3205 takes the incidental information encoded data as an input, and discriminates whether the information should be recorded in the VA U X area, A A U X area, or the sub code area. In the present embodiment, the encoded data relating to the stereoscopic image identification information and the flat display image selection information is distributed to the VAUX region, and the encoded data relating to the shift vector and the projection direction distance to the subcode region.

The image recording area data synthesis unit 3 2 0 1 outputs 3D image encoded data output from the 3D image encoding unit 3 1 0 1 and VAUX output from the incidental information encoded data distribution unit 3 2 0 5. The incidental information encoded data for the area is input, and the incidental information encoded data and the stereo image encoded data are synthesized so that the format shown in Fig. 3 1 is obtained, and the image recording S area data is synthesized. Output. The voice recording area data synthesis unit 3 202 2 is a voice coded data output from the voice image coding unit 3 10 2 and an AAUX area output from the associated information coded data distribution unit 3 20 5. It takes the incidental information coded data as input, synthesizes them into a predetermined format, and outputs data for voice recording area.

The data synthesizing unit for sub code recording area 3 2 0 3 receives the incidental information encoded data for the sub code area outputted from the incidental information coding data distribution unit 3 2 0 5 as an input, and outputs these as a predetermined former Synthesize as it is and output data for subcode recording area.

The track synthesizing unit 3200 receives the data for the image recording area, the data for the audio recording area, and the data for the subcode recording area, and inputs them into the format shown in FIG. Synthesize so as to output data for recording by adding ITI information 2 0 0 1 and a margin between each area.

Although the audio recording area, the image recording area, and the sub code recording area are simultaneously recorded in the present embodiment, these are not necessarily required to be recorded simultaneously, and some of them, for example, the audio recording area and the image recording area. It is also possible to record only the area first, and record the subcode recording area with AFRECO. Alternatively, even if all are recorded at the same time, it is possible to update each area individually by Aleco.

Information such as shift vector for adjusting the amount of parallax and direction / distance of projection is not only determined at the time of shooting, but also shooting ends according to the final composition of the content. It may be determined at a later editing stage. For example, when shooting, default values may be recorded, and finishing may be confirmed at the editing stage, and then information may be added that indicates shift vectors or popping direction / distance. As a default value of shift vector and popping direction Z distance, for example, it is specified that all 0, ie, not adjust the amount of parallax. Also, As a method of adjustment at the editing stage, for example, as in the stereoscopic image display apparatus shown in the third embodiment, the stereoscopic image recording apparatus can further stereoscopically view an input image. A means for performing image processing, a means for receiving input from the user, a means for calculating adjustment information of the parallax amount from the user's input, and a stereoscopic image actually processed from image processing using the adjustment information of the parallax amount A means for determining an area to be displayed on the display, and a means for displaying the determined area, in the stereoscopic image recording device, if the user adjusts the amount of parallax while confirming the display of the stereoscopic image good. According to the recording method and apparatus according to the present embodiment, information indicating shift vector and jump direction distance is recorded in an easy subcode area of AF frame, so even at editing stage. It can be easily changed.

Further, in the present embodiment, information indicating shift vector and popping direction / distance is recorded in the sub code area, but all of them are collectively taken from the viewpoint of being additional information on the image. There is also a method of recording in the VAUX area. In order to do this, the operation of the incidental information encoded data distribution section 3 2 0 5 in FIG. 33 is changed to output all the encoded data of the above information to the V A U X area. In this case, the ease of Afreko is lost, but there is the advantage that the handling is simplified by the additional information related to the image being collected in one place. For example, when copying on a medium having another recording format, if only copying of the image recording area is made, all information on the image can be acquired, and it is necessary to handle the subcode area. Will be lost. It is also possible to record in both the subcode area and the V A U X area to avoid overwriting by Afreko.

Alternatively, if the sub-code area and the VAUX area can not be stored in these areas due to the size limitation of the sub-code area and the VAUX area, the information that can not be stored among the information related to the stereoscopic image is recorded in the AAUX area. It is said that Is also possible.

Note that the configuration of the present embodiment is also based on the digital V T R method widely used for home use, except for a portion specific to a stereoscopic image. Therefore, among the incidental information recorded in the present embodiment, incidental information specific to the stereoscopic image, that is, stereoscopic image identification information, flat display image selection information, shift vector, information on shift direction and distance, etc. It is possible to record a two-dimensional image and a three-dimensional image on the same tape if recording is performed in the expansion area where expansion is permitted in the format of the digital VTR.

Further, in the description of the stereoscopic image recording method of the present invention, although recording is performed on a digital video tape as a recording medium based on the configurations of FIG. 2 9 and FIG. It is also possible to record in the recording area, or in the recording area of the IC memory mounted on a cassette of digital video tape. In this case, as in the above description, the recording area includes an audio recording area, an image recording area, a subcode recording area, and the like, and additional information such as stereoscopic vision information may be recorded in these areas. It is possible. As described above, according to the present embodiment, the present invention is not limited to these examples, and it goes without saying that various modifications are possible. Industrial applicability

As described above, according to the present invention, it is possible to display a favorable three-dimensional image with less discomfort, in a three-dimensional image whose parallax amount has been adjusted. In addition, when displaying a stereoscopic image with an adjusted amount of parallax, the viewer's impressions and effects such as the degree of image popping out and the degree of fatigue when watching for a long time are compared between displays with different screen sizes etc. You can make sure that they do not differ a lot.

In addition, when displaying a stereoscopic image in which the amount of parallax has been adjusted, The range of the image displayed in both modes when the display mode is switched between 3D display and 2D display by selecting the image or the entire right eye image to be displayed. It is possible to switch the display mode without giving a sense of incongruity.

In addition, by enabling specification of images that become the basis of parallax amount adjustment, it becomes possible for data producers of stereoscopic images to adaptively select the display area of the images, and stereoscopic images different from each other Since the display range of the stereoscopic image between the display devices is the same, it is possible to more appropriately reflect the data producer's intention regarding the stereoscopic display.

According to the present invention, it is possible to enhance the convenience of editing by recording information related to the adjustment of the parallax amount in the subcode area easy to use in the AF.

According to the present invention, handling can be simplified by collectively recording stereoscopic control information including information on adjustment of the parallax amount in the image recording area.

Further, according to the present invention, by recording stereoscopic control information including information on the adjustment of the parallax amount in both the subcode area and the image recording area, overwriting due to overwriting at the time of AF is possible. It becomes possible to prevent data loss.

Claims

The scope of the claims

1. A stereoscopic image display apparatus for displaying a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints,

A disparity amount adjustment information calculation unit that calculates disparity amount adjustment information on disparity amount adjustment based on information on change in disparity amount;

An image processing unit that generates an image for stereoscopic display based on the parallax amount adjustment information;

A stereoscopic image display apparatus having:

2. The image interpolation unit further includes an image interpolation unit for capturing a predetermined region in the plurality of images, and the image interpolation unit includes a non-pixel value region having no pixel value in a display region of the image for stereoscopic display. The stereoscopic image display apparatus according to claim 1, wherein, in a certain case, only the non-pixel value area is interpolated using another surface element value.

3. A stereoscopic image display apparatus for displaying a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints,

An image generation unit configured to generate a new image in a predetermined area of the plurality of images.

4. The image generation unit is characterized in that, when there is a non-pixel area where no pixel value exists in the display area of the image for stereoscopic display, a new image is generated only in the non-pixel area. The three-dimensional image display apparatus of Claim 3.

5. In addition, it has an input unit to input information about the change in parallax amount A stereoscopic image display apparatus according to any one of claims 1 to 4,

6. The parallax amount adjustment information calculation unit is configured to reduce the distance between the first reference position of stereoscopic display and the second reference position corresponding to the first reference position of the image for stereoscopic display. The stereoscopic image display device according to any one of claims 1 to 5, wherein a range of a request characterized by calculating the parallax amount adjustment information.

7. A display area determination unit that determines a display area in which stereoscopic display is to be performed, and the distance between the first reference position of the stereoscopic display and the first reference position of the image for stereoscopic display is The stereoscopic image display apparatus according to any one of claims 1 to 6, characterized in that it has a display area judgment unit that judges the display area so as to be smaller. .

8. The stereoscopic image display device according to any one of claims 1 to 7, wherein the parallax amount adjustment information calculation unit has a function of limiting a change amount of the parallax amount. .

9. A stereoscopic view for recording stereoscopic information including: stereoscopic image identification information indicating that at least the plurality of images are stereoscopic images; and information indicating the change in the amount of parallax. The three-dimensional image display device according to any one of claims 1 to 8, characterized by having an information recording unit.

Furthermore, a stereoscopic information recording system for recording stereoscopic information including at least stereoscopic image identification information indicating that the plurality of images are images for stereoscopic viewing and information on the display area The stereoscopic image display apparatus according to any one of claims 1 to 8, characterized in that it has a part.

Furthermore, stereoscopic vision information including at least stereoscopic image identification information indicating that the plurality of images are stereoscopic images, and information indicating the range of change of the parallax amount is included. The stereoscopic image display device according to any one of claims 1 to 8, characterized in that it has a stereoscopic information recording unit for recording.

1 2 A recording method for recording a plurality of images corresponding to a plurality of viewpoints, Stereoscopic image identification information indicating that the plurality of images are images for stereoscopic vision, and a step of recording stereoscopic vision control information for stereoscopically displaying the plurality of images, wherein the stereoscopic vision control information is small. A recording method characterized by including any one of information indicating a change in parallax amount, information indicating a change range of the parallax amount, and information indicating a display area of the plurality of images.

1 3. A transmission method for transmitting a plurality of images corresponding to a plurality of viewpoints, comprising: stereoscopic image identification information indicating that the plurality of images are images for stereoscopic vision; and stereoscopically viewing the plurality of images For transmitting stereoscopic vision control information, wherein the stereoscopic vision control information includes at least information indicating a change in the amount of parallax, information indicating the range of change in the amount of parallax, and the plurality of images A transmission method characterized by including any one of information indicating a display area.

Furthermore, the image processing apparatus further comprises: an enlargement / reduction processing unit that enlarges or reduces the plurality of images; and a stereoscopic control information conversion unit that converts stereoscopic control information.

The stereoscopic control information conversion unit is characterized in that the plurality of images are converted by the enlargement / reduction processing unit on the basis of the enlargement / reduction ratio to convert the information indicating the change in the parallax amount. A stereoscopic image display apparatus according to any one of claims 1 to 11, wherein the 3D image is displayed.

Furthermore, the image processing apparatus comprises: an enlargement / reduction processor for enlarging or reducing the plurality of images; and a stereoscopic control information converter for converting stereoscopic control information;

The stereoscopic display control information conversion unit is characterized in that information of the display area is converted based on an enlargement / reduction ratio at which the plurality of images are enlarged / reduced by the enlargement / reduction processing unit. Range A stereoscopic image display device according to any one of the items 1 to 11. '

Furthermore, the image processing apparatus further includes: an enlargement / reduction processing unit for enlarging or reducing the plurality of images; and a stereoscopic control information conversion unit for converting stereoscopic control information, wherein the stereoscopic control information conversion unit The multiple image is the above enlargement · reduction process The information according to any one of claims 1 to 11, characterized in that the information indicating the change range of the amount of parallax is converted based on the enlargement, reduction, or scaling factor in the part. The three-dimensional image display device as described in.

A recording method for recording a plurality of images corresponding to a plurality of viewpoints, comprising: a step for recording stereoscopic control information for stereoscopically displaying the plurality of images;

The stereoscopic control information includes at least information indicating a change in the amount of parallax, and the information indicating the change in the amount of parallax is configured by parameters determined under a predetermined observation condition. Recording method.

The recording method according to claim 17, wherein the parameter is a parameter indicating a direction and a distance of a stereoscopic image.

The recording method according to claim 17, wherein the parameter is a parameter indicating a direction and a distance for moving at least one of the plurality of images. .

The recording method according to claim 17, wherein the information indicating the change in the amount of parallax is configured by a parameter indicating an angle of parallax of a stereoscopic image.

twenty one .

The first aspect of the present invention is characterized in that the stereoscopic display control information includes planar display image selection information indicating at least one of the plurality of images when planarly displaying a stereoscopic image. The recording method described in any one of paragraphs 7 to 20.

2 2. A transmission method for transmitting a plurality of images corresponding to a plurality of viewpoints, comprising: a step for transmitting stereoscopic control information for stereoscopically displaying the plurality of images,

The stereoscopic control information includes at least information indicating a change in parallax amount, The information indicating the change in the amount of parallax is configured by parameters determined under a predetermined observation condition.

3. The transmission method according to claim 2, wherein the parameter is a parameter indicating a pop-up direction and a distance of a stereoscopic image.

The transmission according to claim 22, wherein the parameter is a parameter indicating a direction and a distance for moving at least one of the plurality of surface images. Method.

25. The transmission method according to claim 22, wherein the information indicating the change in the amount of parallax is configured by parameters indicating an angle of parallax of a stereoscopic image.

6 6. The display control information is characterized in that at least the stereoscopic display control information includes planar display image selection information indicating which one of the plurality of images is used when planarly displaying a stereoscopic image. The transmission method according to any one of paragraphs 2 to 25.

27. A stereoscopic image display apparatus for displaying a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints,

A stereoscopic image display apparatus characterized by comprising a display area determination unit that determines a display area based on selection information of an image related to flat display.

2 8. The parallax amount adjustment information calculation unit is characterized by calculating the parallax amount adjustment information from the parameter indicating the pop-out direction and distance of the stereoscopic image. The stereo image display device according to any one of claims 2 to 7, or

29. The parallax amount adjustment information calculation unit is characterized in that the parallax amount adjustment information is calculated from a parameter indicating a direction and a distance in which at least one of the plurality of images is moved. A stereoscopic image display apparatus according to any one of claims 1 to 11, or claim 27.

3 0. The parallax amount adjustment information calculation unit is characterized by calculating the parallax amount adjustment information from the parameter indicating the angle of parallax of the stereoscopic image. The stereoscopic image display device according to any one of claims 27.

3 1. A stereoscopic image recording method for recording a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,

A stereoscopic image recording method comprising: a step of providing parallax amount adjustment information related to adjustment of the parallax amount of a stereoscopic image in the recording region and recording the same in an image recording region for recording the stereoscopic image.

3 2. A stereoscopic image recording method for recording a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,

A stereoscopic image recording method comprising: a step of recording parallax amount adjustment information related to adjustment of a parallax amount of a stereoscopic image in an audio recording area for recording audio provided in the recording area.

3 3. A stereoscopic image recording method for recording a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,

A stereoscopic image recording method comprising: a step of recording parallax amount adjustment information related to adjustment of a parallax amount of a stereoscopic image in a subcode area provided in the recording area and recording incidental information.

3 4. A stereoscopic image recording apparatus for recording a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,

Parallax amount adjustment information relating to adjustment of the parallax amount of the stereoscopic image, A three-dimensional image recording apparatus comprising: means provided for recording in the image recording area for recording the three-dimensional image.

3 5. A stereoscopic image recording apparatus for recording a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,

A stereoscopic image recording apparatus comprising: parallax amount adjustment information related to adjustment of a parallax amount of a stereoscopic image; and a unit provided in the recording region for recording the audio in the audio recording region for recording audio.

A stereoscopic image recording apparatus for recording a stereoscopic image composed of a plurality of images corresponding to a plurality of viewpoints in a predetermined recording area,

A three-dimensional image recording apparatus comprising: a parallax amount adjustment information related to adjustment of a parallax amount of a three-dimensional image, provided in the recording area and recording means in a sub-code area for recording incidental information.

3 7 A stereoscopic image display device that displays a stereoscopic surface image based on a plurality of images corresponding to a plurality of viewpoints,

A reading unit that reads out parallax amount change information indicating a change in the parallax amount; an image processing unit that generates an image for stereoscopic display based on the parallax amount change information;

A stereoscopic image display apparatus having:

The image capturing unit further interpolates a predetermined area in the plurality of images, 8 8

The image interpolation unit is configured to interpolate only the non-pixel value area using another pixel value, when there is a non-pixel value area in which no pixel value exists in the display area of the image for stereoscopic display. The three-dimensional image display device according to claim 37, characterized in that

3 9. A stereoscopic image display device that displays a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints. A reading unit that reads out parallax amount change information indicating a change in the parallax amount; an image processing unit that generates an image for stereoscopic display based on the parallax amount change information;

The image generation unit is characterized in that, when there is a non-pixel area where no pixel value exists in the display area of the image for stereoscopic display, a new image is generated only in the non-pixel area. The stereoscopic image display apparatus according to claim 39.

Furthermore, the three-dimensional image display device according to any one of claims 3 to 40, further comprising: an input unit that inputs information related to a change in parallax amount.

Furthermore, a display area determination unit that determines a display area in which stereoscopic display is to be performed, the distance between the first reference position of the stereoscopic display and the first reference position of the image for stereoscopic display is The stereoscopic image display device according to any one of claims 37 to 41, characterized by comprising a display area determination unit that determines the display area so as to be smaller.

4 3. Stereoscopic information recording stereoscopic image information including: stereoscopic image identification information indicating that at least the plurality of images are stereoscopic images; and information indicating a change in the parallax amount. The stereoscopic image display device according to any one of claims 3 to 4, characterized in that it has a visual information recording unit.

Furthermore, a stereoscopic information recording system for recording stereoscopic information including at least stereoscopic image identification information indicating that the plurality of images are images for stereoscopic viewing and information of the display area. The stereoscopic image display device according to any one of claims 3 to 4, characterized in that it has a part.

Furthermore, at least the plurality of images may be stereoscopic images. The scope of a request characterized by having a stereoscopic vision information recording unit for recording stereoscopic vision information including stereoscopic image identification information indicating a range and information indicating a range of change of the parallax amount. 4 2 3D display device according to any one of the preceding paragraphs.

6 6. A stereoscopic image display device that displays a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints,

4 7. A recording method for recording a plurality of images corresponding to a plurality of viewpoints, comprising: a step for recording stereoscopic control information for stereoscopically displaying the plurality of images;

The recording method characterized in that the stereoscopic control information includes at least a parameter for specifying an image to be a reference at the time of parallax amount adjustment out of the plurality of images.

A transmission method for transmitting a plurality of images corresponding to a plurality of viewpoints, comprising: a step for transmitting stereoscopic control information for stereoscopically displaying the plurality of images;

The transmission method, wherein the stereoscopic control information includes at least a parameter for specifying an image to be a reference at the time of parallax amount adjustment from among the plurality of images.

4 9 A stereoscopic image display device that displays a stereoscopic image based on a plurality of images corresponding to a plurality of viewpoints,

A reading unit that reads disparity amount change information indicating a change in disparity amount; A stereoscopic image display apparatus comprising: an image processing unit that generates an image for stereoscopic display based on the parallax amount change information and information indicating an image to be a parallax amount adjustment ^standard .