JP4148811B2 - Stereoscopic image display device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a stereoscopic image display technique, and more particularly to a technique for generating or displaying a stereoscopic image based on a parallax image.
[0002]
[Prior art]
In recent years, the lack of network infrastructure has been regarded as a problem. However, the transition to broadband has started, and rather, the number of types and the number of contents that effectively use a wide band are becoming conspicuous. Video has always been the most important means of expression, but many of the efforts so far have been related to improvements in display quality and data compression ratios. There is a feeling that such efforts are in the back.
[0003]
Under such circumstances, stereoscopic image display (hereinafter simply referred to as “stereoscopic display”) has been studied in various ways, and has been put into practical use in a limited market where theater applications and special display devices are used. In the future, research and development in this direction will accelerate with the aim of providing more realistic content, and it seems that an era will come when individual users will enjoy stereoscopic display even at home (Patent Document 1).
[Patent Document 1]
JP 2003-009183 A [Problems to be Solved by the Invention]
Under such circumstances, some problems have been pointed out for stereoscopic display. For example, it is difficult to optimize the parallax that causes a three-dimensional effect. Originally, it does not actually project a three-dimensional object, but rather casts the image by shifting it in pixel units with respect to the left and right eyes, and it is not easy to give the artificial three-dimensional effect a natural feeling. Absent.
[0004]
In addition, too much parallax may be a problem, and a viewer of a stereoscopic image (hereinafter, also simply referred to as “user”) may complain of mild discomfort. Of course, there are various factors such as not only the three-dimensional display but also the situation or sense of surroundings of the scene being displayed is not consistent with the situation.
[0005]
The present invention has been made in view of such a background, and an object of the present invention is to perform appropriate three-dimensional display according to the situation of an observer of a three-dimensional image. Another object is to provide an appropriate stereoscopic display according to the observation environment. Yet another object is to display an appropriate 3D image according to the nature of the 3D image.
[0006]
[Means for Solving the Problems]
One embodiment of the present invention is a stereoscopic image display device. This apparatus is based on a stereoscopic image data storage unit that stores stereoscopic image data, a parallax amount data storage unit that stores parallax amount data for each user, and parallax amount data stored in the parallax amount data storage unit. And a viewpoint image acquisition unit that acquires a plurality of viewpoint images corresponding to different parallaxes from the stereoscopic image data stored in the stereoscopic image data storage unit.
[0007]
“Stereoscopic image data” is data that is the source of a stereoscopic image. The stereoscopic image data may be an image obtained by capturing an object from a plurality of viewpoints, or may be data for three-dimensionally reproducing a stereoscopic object, such as polygon data. The “viewpoint image” refers to a two-dimensional image obtained by capturing a certain three-dimensional object from a predetermined viewpoint. “Parallax” is a parameter for producing a three-dimensional effect, and various definitions are possible, but as an example, it can be expressed by a shift amount of a pixel representing the same point between viewpoint images. Further, the parallax is referred to as “parallax amount” when used in the sense of representing the size. The viewpoint image may be acquired by generating a viewpoint image based on a predetermined amount of parallax from stereoscopic image data, or selecting appropriate two images from a plurality of viewpoint images prepared in advance. May be made by:
[0008]
By registering an appropriate amount of parallax for each user, stereoscopic display can be performed with parallax optimal for each user. Further, the parallax amount for each user may be adjusted as needed according to the usage history. Here, the range of the appropriate parallax may be specified. The identification of the appropriate parallax may be performed with the maximum value allowable as the parallax of the object that is closest to the user.
[0009]
This apparatus includes a question presenting unit that presents a question to a user, and a parallax amount data identifying unit that identifies appropriate parallax amount data to be stored in the parallax amount data storage unit based on an answer to the question. May further be included.
[0010]
Questions are asked in order to obtain information about the user about a given item, such as whether the user is age, gender, whether they are prone to car sickness, whether they regularly exercise, or whether the room is small. Is called. For example, in general, there is an empirical rule that elderly people have difficulty in stereoscopic viewing when a large amount of parallax is set. In addition, for users with a constitution that tends to get drunk on a car, a stereoscopic display with a large change in the amount of parallax is often accompanied by discomfort.
[0011]
According to the embodiment of the present invention, a user can set an appropriate amount of parallax for himself / herself only by answering a question. In addition, since the parallax can be adjusted according to the user's situation, it is also effective in terms of safety measures. For example, when the user is an elderly person, it may be necessary to treat the object so that the object is not too close to the user in order to relieve psychological pressure caused by the stereoscopic image. Further, even after the parallax amount is set by this question method, the user may be able to finely adjust the parallax amount.
[0012]
This apparatus includes a parallax response acquisition unit that acquires a user response to a stereoscopic image displayed based on a plurality of viewpoint images corresponding to different parallaxes, and a parallax amount data storage unit based on the acquired response. You may further include the parallax amount data specific | specification part which specifies the appropriate parallax amount data which should be stored.
[0013]
The parallax response acquisition unit is provided as, for example, a GUI (Graphical User Interface), and first displays while changing the parallax between the viewpoint images. When the user feels the stereoscopic effect he likes, the user inputs that effect by operating a button or the like. Alternatively, several stereoscopic images with different parallax may be prepared and the user may select from the candidates. Alternatively, the user may input the amount of parallax as a numerical value as it is.
[0014]
Another aspect of the present invention is also a stereoscopic image display device. The apparatus includes a stereoscopic image data storage unit that stores stereoscopic image data, an observation time detection unit that detects a time during which the user observes the stereoscopic image, and stereoscopic image data stored in the stereoscopic image data storage unit. In addition, the image processing apparatus includes a viewpoint image acquisition unit that acquires a plurality of viewpoint images corresponding to different parallaxes by appropriately adjusting the parallax amount based on the time detected by the observation time detection unit.
[0015]
The “observation time” may be the length of time during which the user observes the stereoscopic image (hereinafter referred to as “observation period”), or the time when the stereoscopic image is observed (hereinafter referred to as “observation time”). ”). For example, early morning is not an activity time for a user who is living at night. In this way, by considering the life biorhythm of the user and adjusting the parallax amount using the observation time as a parameter, a more appropriate value of the parallax amount can be specified. Note that the observation period is a periodical concept such as a total time of observation time or observation time.
[0016]
The viewpoint image acquisition unit of this apparatus has a plurality of viewpoint images corresponding to different parallaxes from the stereoscopic image data stored in the stereoscopic image data storage unit. It is good also as acquiring by making quantity small.
[0017]
The longer the observation period, the more fatigue the user accumulates through stereoscopic viewing. Therefore, by reducing the amount of parallax as the observation period is longer, it is possible to reduce the burden associated with the user's stereoscopic vision.
[0018]
Still another embodiment of the present invention is also a stereoscopic image display device. The apparatus includes a stereoscopic image data storage unit that stores stereoscopic image data, an observation time detection unit that detects a time during which the user observes the stereoscopic image, and stereoscopic image data stored in the stereoscopic image data storage unit. In addition, the image processing apparatus includes a viewpoint image acquisition unit that acquires a plurality of viewpoint images corresponding to different parallaxes by appropriately adjusting the display size based on the time detected by the observation time detection unit.
[0019]
“Display size” refers to any one of the area, vertical width, horizontal width of a screen on which stereoscopic display is performed, or any combination thereof. For example, by reducing the area of the stereoscopic display screen, the burden associated with the user's stereoscopic vision can be reduced.
[0020]
The viewpoint image acquisition unit of this apparatus increases the display size of the viewpoint image corresponding to the different parallax from the stereoscopic image data stored in the stereoscopic image data storage unit as the time during which the user observes the stereoscopic image is longer. It is good also as acquiring small.
[0021]
Still another embodiment of the present invention is also a stereoscopic image display device. This apparatus includes a stereoscopic image data storage unit that stores stereoscopic image data, an observation environment measurement unit that measures an observation environment in which a user observes a stereoscopic image, and stereoscopic image data stored in the stereoscopic image data storage unit And a viewpoint image acquisition unit that acquires a plurality of viewpoint images corresponding to different parallaxes by adjusting the amount of parallax based on information about the observation environment measured by the observation environment measurement unit.
[0022]
“Observation environment” refers to the external environment where the user is performing stereoscopic viewing. Specifically, the observation environment measurement unit evaluates the observation environment by measuring parameters such as the brightness, temperature, humidity, noise, and atmospheric pressure of the room in which the user is viewing stereoscopically.
[0023]
The viewpoint image acquisition unit of this device has a plurality of viewpoint images corresponding to different parallaxes from the stereoscopic image data stored in the stereoscopic image data storage unit, and the amount of parallax increases as the environment in which the user observes the stereoscopic image is brighter. It is good also as acquiring by making small.
[0024]
In general, there is an empirical rule that the brighter the surroundings, the greater the fatigue from stereoscopic viewing. Therefore, by reducing the parallax amount as the surroundings become brighter, it is possible to reduce the user's burden associated with stereoscopic viewing.
[0025]
Still another embodiment of the present invention is also a stereoscopic image display device. This apparatus includes a stereoscopic image data storage unit that stores stereoscopic image data, an observation environment measurement unit that measures an observation environment in which a user observes a stereoscopic image, and stereoscopic image data stored in the stereoscopic image data storage unit The image processing apparatus includes a viewpoint image acquisition unit that acquires a plurality of viewpoint images corresponding to different parallaxes by adjusting the display size based on information about the observation environment measured by the observation environment measurement unit.
[0026]
The viewpoint image acquisition unit of this device displays a plurality of viewpoint images corresponding to different parallaxes from the stereoscopic image data stored in the stereoscopic image data storage unit, as the environment in which the user observes the stereoscopic image becomes brighter. It is good also as acquiring by making small.
[0027]
Still another embodiment of the present invention is also a stereoscopic image display device. The apparatus includes a stereoscopic image data storage unit that stores stereoscopic image data, a fatigue level measurement unit that estimates user fatigue, and a plurality of stereoscopic image data stored in the stereoscopic image data storage unit that correspond to different parallaxes. A viewpoint image acquisition unit that acquires the viewpoint image by adjusting the amount of parallax based on information on the user's fatigue estimated by the fatigue level measurement unit.
[0028]
A user's fatigue degree is estimated based on a user's physical information. Specifically, the fatigue level measurement unit measures blinks, pupil diameter, eye congestion, nasal cavity temperature, and the like, and estimates the fatigue level of the user based on these measured values.
[0029]
Still another embodiment of the present invention is also a stereoscopic image display device. The apparatus includes a stereoscopic image data storage unit that stores stereoscopic image data, a fatigue level measurement unit that estimates user fatigue, and a plurality of stereoscopic image data stored in the stereoscopic image data storage unit that correspond to different parallaxes. Including a viewpoint image acquisition unit that adjusts the display size of the viewpoint image based on information on the user's fatigue estimated by the fatigue level measurement unit.
[0030]
Still another embodiment of the present invention is also a stereoscopic image display device. This apparatus stores a plurality of viewpoint images corresponding to different parallaxes in a stereoscopic image data storage unit from a stereoscopic image data storage unit that stores stereoscopic image data and the stereoscopic image data stored in the stereoscopic image data storage unit. And a viewpoint image acquisition unit that adjusts and acquires the parallax amount based on the property of the stereoscopic image data.
[0031]
“Characteristics of stereoscopic image data” refers to the contents of stereoscopic image content provided by this apparatus, such as the number, shape, size, color, and change of objects, the speed of movement of objects, and patterns of movement. Depending on the nature of the stereoscopic image data, for example, even if the observation period is the same, the fatigue level of the user is generally different. In determining the amount of parallax, a parameter representing this property may be registered together with storing stereoscopic image data in the stereoscopic image data storage unit. Alternatively, during the stereoscopic display, a parameter representing the property of the stereoscopic image data may be changed according to a scene with intense motion and a gentle scene.
[0032]
Still another embodiment of the present invention is also a stereoscopic image display device. This apparatus stores a plurality of viewpoint images corresponding to different parallaxes in a stereoscopic image data storage unit from a stereoscopic image data storage unit that stores stereoscopic image data and the stereoscopic image data stored in the stereoscopic image data storage unit. And a viewpoint image acquisition unit that adjusts and acquires the display size based on the property of the stereoscopic image data.
[0033]
Depending on the nature of the stereoscopic image data, changing the aspect ratio of the screen may contribute to reducing the burden associated with the user's stereoscopic vision.
[0034]
It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.
[0035]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 shows the positional relationship among the user 30, the screen 32, and the playback object 34 displayed in a three-dimensional manner. The distance between the eyes of the user 30 is E, the distance between the user 30 and the screen 32 is D, and the width of the reproduction object 34 when displayed is W. Since the reproduction object 34 is stereoscopically displayed, it has pixels that are sensed closer to the screen 32, that is, pixels that are located closer to each other, and pixels that are sensed farther than the screen 32, that is, pixels that are located farther away. Pixels without parallax appear on the screen 32 at the same position from both eyes, and are thus detected on the screen 32.
[0036]
FIG. 2 shows an imaging system for generating the ideal display of FIG. The distance between the two cameras 22 and 24 is E, the distance from the camera 20 to the optical axis crossing position when the object 20 is viewed is D, the angle of view is the same width as the screen 32, and the width is actually W. If a certain object 20 is photographed, two viewpoint images can be obtained from the two cameras. If this is displayed on the screen 32 of FIG. 1, the ideal state of FIG. 1 is realized.
[0037]
FIG. 3 shows a hardware configuration of the stereoscopic image display apparatus 10 according to the present embodiment. The server apparatus 100 provides services to a plurality of terminals (hereinafter simply referred to as “client terminals 14”) such as a first client terminal 14a and a second client terminal 14b connected via a LAN (Local Area Network) 16. To do. The user 30 may perform stereoscopic viewing via the client terminal 14, but may be able to perform stereoscopic viewing using the screen display function of the server device 100. The client terminal 14 may be connected to the server apparatus 100 via a WAN (Wide Area Network) 12.
[0038]
The server device 100 and the client terminal 14 can be realized in hardware by elements such as a CPU of a computer, and are realized in software by a program having a data transmission / reception function. Etc. depict functional blocks realized by their cooperation. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by a combination of hardware and software.
[0039]
FIG. 4 is a functional block diagram of the stereoscopic image display apparatus 10. The user interface processing unit 120 performs processing related to the user interface with the user 30. Of the user interface processing unit 120, the input unit 110 accepts external input including operations from the user 30. The image providing unit 108 provides an image to the user 30. The stereoscopic image data registration unit 112 registers stereoscopic image data via the input unit 110. The registered stereoscopic image data is stored in the stereoscopic image data storage unit 104.
[0040]
The parallax amount data registration unit 200 identifies the parallax amount data for each user 30 via the input unit 110 and stores this in the parallax amount data storage unit 106. The functional blocks inside the parallax amount data registration unit 200 will be described in detail later. The viewpoint image acquisition unit 102 acquires two viewpoint images having different parallaxes from the stereoscopic image data stored in the stereoscopic image data storage unit 104 based on the parallax amount data stored in the parallax amount data storage unit 106. . The image providing unit 108 provides a stereoscopic image to the user 30 based on the two viewpoint images acquired by the viewpoint image acquiring unit 102.
[0041]
The stereoscopic effect adjusting unit 300 detects the state of the user 30 and the observation environment via the input unit 110. The viewpoint image acquisition unit 102 specifies values of parallax and display size in consideration of various types of information detected by the stereoscopic effect adjustment unit 300, and acquires viewpoint images from the stereoscopic image data storage unit 104. The functional blocks inside the stereoscopic effect adjusting unit 300 will also be described in detail later.
[0042]
FIG. 5 is a functional block diagram inside the parallax amount data registration unit 200. The parallax amount data registration unit 200 identifies the parallax amount data for each user 30 and stores it in the parallax amount data storage unit 106. The question data storage unit 202 stores questions to the user 30. The question presentation unit 204 presents the question stored in the question data storage unit 202 to the user 30 and obtains an answer. The parallax amount data specifying unit 208 specifies the amount of parallax to be stored in the parallax amount data storage unit 106 based on the answer acquired by the question presenting unit 204. The parallax response acquisition unit 206 uses the stereoscopic image data stored in the stereoscopic image data storage unit 104 to acquire the response of the user 30 to the stereoscopic display. Also in this case, the parallax amount data specifying unit 208 specifies the amount of parallax to be stored in the parallax amount data storage unit 106 based on the response of the user 30 acquired by the parallax response acquisition unit 206.
[0043]
FIG. 6 is a functional block diagram inside the stereoscopic effect adjusting unit 300. The stereoscopic effect adjusting unit 300 detects the state of the user 30 and the observation environment. The fatigue level measurement unit 350 acquires physical information from the user 30 in order to estimate the fatigue of the user 30. Then, the acquired information is converted into an appropriate index and transmitted to the viewpoint image acquisition unit 102. The observation time detection unit 400 detects the observation time of the user 30. Of the observation time detector 400, the observation time detector 402 detects the observation time. In addition, the observation period detection unit 404 detects the observation period. Information obtained from the observation time detection unit 402 and the observation period detection unit 404 is sent to the determination unit 406. The determination unit 406 converts these pieces of information into appropriate indexes and transmits them to the viewpoint image acquisition unit 102.
[0044]
The observation environment measurement unit 450 measures the observation environment. Of the observation environment measurement unit 450, the brightness detection unit 452 detects the brightness of the environment in which the user 30 observes the stereoscopic image. The temperature detector 454 detects the temperature in the observation environment. The sound detection unit 456 detects sound such as noise in the observation environment. The determination unit 458 converts the information obtained from the brightness detection unit 452, the temperature detection unit 454, and the sound detection unit 456 into an appropriate index and transmits it to the viewpoint image acquisition unit 102. As described above, the viewpoint image acquisition unit 102 adjusts the parallax of the viewpoint image and the display size of the stereoscopic image based on various indexes obtained from the stereoscopic effect adjustment unit 300.
[0045]
FIGS. 7A and 7B show the left eye image 500 and the right eye image 502 displayed in the process of acquiring the parallax response by the parallax response acquisition unit 206, respectively. In each image, five black circles are displayed, and a larger parallax is placed closer to the top, and a larger parallax is placed farther down.
[0046]
FIG. 8 schematically shows a sense of distance sensed by the user 30 when these five black circles are displayed. The user 30 responds that the range of these five senses of distance is “appropriate”, and the parallax response acquisition unit 206 acquires this response. In the figure, five black circles with different parallaxes are displayed simultaneously or sequentially, and the user 30 inputs whether or not the parallax is acceptable. On the other hand, in FIG. 9, the display itself is performed with a single black circle, but the parallax is continuously changed, and a response is made when the user 30 reaches the limit allowed in each of the far and near positions. The response may be performed by using a known technique such as normal key operation, mouse operation, voice input, and the like.
[0047]
FIG. 10 shows a flowchart from when the user 30 registers the amount of parallax to three-dimensionally displays and ends this display. The user 30 registers his / her parallax amount data in the parallax amount data storage unit 106 by using the parallax amount data registration unit 200 (S10). In performing stereoscopic display, the viewpoint image acquisition unit 102 specifies a display size (S12) and specifies an appropriate parallax (S14). Based on this, the viewpoint image acquisition unit 102 acquires an appropriate viewpoint image from the stereoscopic image data storage unit 104, and the stereoscopic image providing unit 108 performs stereoscopic display on the user 30 (S16). When the stereoscopic display is ended, or when the user 30 ends the observation of the stereoscopic image (Y in S18), the process ends.
[0048]
When the stereoscopic display is not finished (N in S18), the viewpoint image acquisition unit 102 determines whether to change the display size based on various information acquired by the stereoscopic effect adjustment unit 300 (S20). . When changing (Y of S20), a process returns to S12. When the display size is not changed (N in S20), the viewpoint image acquisition unit 102 determines whether or not to change the parallax based on various information acquired by the stereoscopic effect adjustment unit 300 ( S22). When changing (Y of S22), a process returns to S14. When not changing (N of S22), a process returns to S16.
[0049]
As described above, in the embodiment, the user can register the parallax amount appropriate for himself / herself in advance. Therefore, when performing stereoscopic display for personal use, stereoscopic display corresponding to individual differences is possible. In addition, the stereoscopic image display apparatus determines the observation time, fatigue of the user, and the like and adjusts the parallax amount and the display size as appropriate even while the user is viewing stereoscopically. For this reason, the fatigue of the user accompanying stereoscopic vision can be reduced. As a result, stereoscopic display can be performed for a long time without burdening the user.
[0050]
The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .
[0051]
【The invention's effect】
According to the present invention, appropriate three-dimensional display can be performed according to the situation of an observer of a three-dimensional image. Further, according to the present invention, appropriate three-dimensional display can be performed according to the observation environment. Furthermore, according to the present invention, it is possible to display an appropriate stereoscopic image according to the properties of the stereoscopic image.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a positional relationship among a user, a screen, and a reproduction object that are ideally viewed stereoscopically.
FIG. 2 is a diagram illustrating an example of an imaging system that realizes the state of FIG. 1;
FIG. 3 is a configuration diagram of a stereoscopic image display apparatus according to the present embodiment.
FIG. 4 is a functional block diagram of the stereoscopic image display apparatus according to the present embodiment.
FIG. 5 is a functional block diagram of a parallax amount data registration unit according to the present embodiment.
FIG. 6 is a functional block diagram of a stereoscopic effect adjusting unit according to the present embodiment.
FIGS. 7A and 7B are diagrams showing a left eye image and a right eye image displayed by the stereoscopic effect adjusting unit of the stereoscopic image display device, respectively.
FIG. 8 is a diagram illustrating a plurality of objects having different parallaxes displayed by a stereoscopic effect adjusting unit of the stereoscopic image display device.
FIG. 9 is a diagram illustrating an object whose parallax changes, which is displayed by the stereoscopic effect adjusting unit of the stereoscopic image processing apparatus.
FIG. 10 is a flowchart from when a user registers a parallax amount to when a stereoscopic image is displayed until the end of the stereoscopic image display device according to the present embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 stereoscopic image display apparatus, 102 viewpoint image acquisition part, 104 stereoscopic image data storage part, 106 parallax amount data storage part, 200 parallax amount data registration part, 204 question presentation part, 206 parallax response acquisition part, 208 parallax amount data specification part , 300 stereoscopic effect adjustment unit, 350 fatigue measurement unit, 400 observation time detection unit, 450 observation environment measurement unit.

Claims (2)

A stereoscopic image data storage unit for storing stereoscopic image data;
A parallax amount data storage unit for storing parallax amount data for each user;
Viewpoint image acquisition for acquiring a plurality of viewpoint images corresponding to different parallaxes from the stereoscopic image data stored in the stereoscopic image data storage unit based on the parallax amount data stored in the parallax amount data storage unit A three-dimensional image display device comprising:
A question presentation unit for presenting questions to the user;
A parallax amount data specifying unit for specifying appropriate parallax amount data to be stored in the parallax amount data storage unit for the user based on an answer to the question;
A stereoscopic image display device comprising: A stereoscopic image data storage unit for storing stereoscopic image data;
A parallax amount data storage unit for storing parallax amount data for each user;
Viewpoint image acquisition for acquiring a plurality of viewpoint images corresponding to different parallaxes from the stereoscopic image data stored in the stereoscopic image data storage unit based on the parallax amount data stored in the parallax amount data storage unit A three-dimensional image display device comprising:
A parallax response acquisition unit that acquires a user's response to a stereoscopic image displayed based on a plurality of viewpoint images corresponding to different parallaxes;
A parallax amount data specifying unit for specifying appropriate parallax amount data to be stored in the parallax amount data storage unit based on the acquired response;
A stereoscopic image display device comprising:

Images