JP3504111B2 - Stereoscopic system, stereoscopic method, and storage medium for storing computer program for displaying a pair of images viewed from two different viewpoints in a stereoscopic manner - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a binocular stereoscopic vision system, and in particular, it is possible to reduce user fatigue by automatically changing the stereoscopic vision conditions in accordance with changes in the scene displayed on the display. The present invention relates to a stereoscopic vision system and a stereoscopic vision method.

[0002]

2. Description of the Related Art Conventionally, a display surface, that is, 2
Binocular stereoscopic vision for displaying stereoscopic vision by displaying an image for the left eye and an image for the right eye that have been given a predetermined amount of displacement (binocular parallax) on a dimensional plane, and fusing these images The system is known. In such a binocular stereoscopic system, there is an essential problem of contradiction between accommodation and convergence. That is, since the image is actually displayed on the display surface, the eye focus (adjustment position) always matches the display surface, but the position of the gazing point determined by the vergence eye movement is on the display surface. Since it is determined by the amount of shift between the image for the left eye and the image for the right eye displayed in, it is almost not located on the display surface.

When there is a contradiction between the adjustment and the convergence as described above, when the amount of the contradiction is small, a human has the ability to fuse the image for the left eye and the image for the right eye and can perform stereoscopic vision. However, as the amount of contradiction increases, humans cannot merge the left-eye image and the right-eye image, or even if they are able to fuse, the eye fatigue increases and stereoscopic vision becomes difficult. It becomes impossible or difficult to do.

Therefore, conventionally, it has been necessary to adjust the stereoscopic condition so that the user of the system can easily see the image due to trial and error. In the case of a still image, this work is relatively easy, but in the case of a moving image, the position of the gazing point of the viewer changes depending on the shooting target. Therefore, it is necessary to reset the stereoscopic condition depending on the scene. Not only that, it will be a very troublesome task. Therefore, in the case of a moving image, the actual condition is that the stereoscopic condition during reproduction is not changed.

Further, graphic user interface items such as character information, menus and buttons are often displayed on the display of such a stereoscopic system. However, with respect to this graphic user interface item, since processing according to the stereoscopic processing of the stereoscopic display image has not been performed, the user often feels something strange on the screen.

On the other hand, in recent years, with the spread of DVDs and the like, a system having a function of allowing a user of the system to view an image from an angle desired by the user has been realized. Also for such an image display system, a system for providing a more effective stereoscopic effect is desired, and also for this purpose, it is strongly desired to solve the above problems. .

[0007]

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above circumstances, and realizes a system by realizing a function of automatically determining a stereoscopic viewing condition according to a change in a scene. It is an object of the present invention to provide a stereoscopic vision system and a stereoscopic vision method that do not require the user to perform complicated operations and do not give the user a feeling of fatigue.

Another object of the present invention is to realize a function of displaying a graphic user interface item displayed at the same time as the model without causing a feeling of discomfort with respect to the model, thereby reducing a user's feeling of fatigue. It is to provide a stereoscopic system and a stereoscopic method which do not give.

Still another object is a method of generating information for changing a stereoscopic condition according to a scene, and information generated by the method is stored together with the right eye image information and the left eye image information. The present invention provides a recording medium such as a DVD, and further provides a manufacturing apparatus and a reproducing apparatus for this recording medium.

[0010]

In order to achieve the above-mentioned object, the invention according to claim 1 is a pair of right and left eyes in which a model is displayed on a display from two different viewpoints. In the stereoscopic system for stereoscopically displaying using the image for viewing, a gaze range determining unit that determines a range in which the observer is likely to gaze at the model,
Model moving means for moving the model in a direction orthogonal to the display surface so that the gaze range determined by the gaze range determining means falls within the easy stereoscopic view by changing the data of the model, and the model moving means. By changing the data of the model changed by the moving means, the size of the model moved by the model moving means is increased or reduced according to the movement distance, and the model moving means moves the model. And image data creating means for creating image data to be displayed on the display based on the model data whose size has been changed by the model scaling means.

According to the first aspect of the present invention, the gaze range is estimated by the gaze range determining means, and the model is moved in the depth direction so that the gaze range falls within the easy stereoscopic view region. Next, the size of the model is changed by the model enlarging / reducing means according to the moving distance. The data of the model thus modified is sent to the image data creating means, and the image data creating means creates the image data to be displayed on the display.

In the stereoscopic system according to the first aspect, the scale in the depth direction viewed from the viewpoint is set so that at least a main part of the model moved by the model moving means is within the stereoscopic easy area. It is preferable to further include depth distance reducing means.

The stereoscopic system according to claim 1 further comprises line-of-sight / gaze-range detecting means for detecting a line-of-sight and a gaze range with respect to an actually displayed image, and the model moving means has the line-of-sight / gaze range detecting means. It is preferable to move the model in a direction orthogonal to the display based on the gaze range detected by.

The model moving means of the stereoscopic system according to claim 1 changes the arrangement condition of the model viewed from the viewpoint, and when the model is moved in accordance with this change, the time when the arrangement condition changes. It is preferable to have a means for preventing the movement amount of the gaze range of the model from becoming 1 degree or more in terms of the convergence angle within a predetermined time.

The invention according to claim 5 is a display,
In a stereoscopic system that stereoscopically displays a model using a pair of images viewed from two different viewpoints, a gaze range deciding unit that decides a part that is likely to be gazed by an observer with respect to the model, Based on the output of the gaze range determining means, a perspective transformation determining means for determining a perspective transformation method of the model data on the pair of left and right eye images, and a perspective transformation determined by the perspective transformation determining means. Image data creating means for creating image data to be displayed on the display based on a method.

With such a configuration, it is possible to generate image data that is easily stereoscopically viewed by the perspective conversion method without actually moving and enlarging / reducing the model as in the first aspect of the invention.

According to a fifth aspect of the present invention, in the perspective transformation determining means, the deviation from the reference position in the pair of images of the gaze range image determined by the gaze range determining means becomes a viewing angle. It is preferable to determine the perspective transformation method so that it is within 2 degrees.

According to a fifth aspect of the present invention, when the arrangement condition of the model viewed from the viewpoint changes, the stereoscopic vision system is created by the image data creating means by using the perspective transforming method created by the perspective transform creating means. It is preferable to further have a function of preventing the change amount of the image shift from being approximately 1 degree or more in terms of the angle of convergence.

In the stereoscopic system according to any one of claims 1 and 5, the gaze range deciding means is arranged from the viewpoint of an observer, and the arrangement condition, the color tone, the brightness, the saturation, the size, the complexity, and the moving speed of the model. It is preferable to have means for estimating the gaze range of both eyes in consideration of all or part of the above.

It is preferable that the stereoscopic system according to claims 1 and 5 further comprises an out-of-view data excluding means for excluding model data existing outside the field of view.

The stereoscopic system according to claim 7 further comprises line-of-sight / gaze-range detecting means for detecting a line-of-sight and a gaze range with respect to an actually displayed image, and the out-of-field data excluding means includes the line-of-sight / gaze range. It is preferable to calculate the visual field based on the line of sight detected by the detection means and exclude the data existing outside the visual field.

It is preferable that the stereoscopic system according to claims 1 and 5 further comprises color tone changing means for bringing the color tone of the displayed model closer to the color tone of the background color as it goes away from the viewpoint in the depth direction.

It is preferable that the image data creating means of the stereoscopic vision system according to any one of claims 1 and 5 further has a function of determining a stereoscopic vision condition of the graphic user interface item displayed on the display.

In the stereoscopic system according to the first and fifth aspects, it is preferable that the stereoscopic system has a blurring means for blurring the edges of the left and right images when the model is located at the edge of the display screen.

It is preferable that the stereoscopic vision system according to claims 1 and 5 further comprises means for reducing eye strain by changing the stereoscopic vision condition to weaken the stereoscopic vision effect.

According to a thirteenth aspect of the present invention, in a stereoscopic vision system in which a pair of images viewed from two different viewpoints are stereoscopically displayed on a display, an image portion which an observer is likely to gaze is determined. Gaze range determining means, and based on the output of the gaze range determining means, an image conversion determining means that determines an image converting method such that the gaze range becomes an area that facilitates stereoscopic viewing, and an output of the image conversion determining means. And image data creating means for creating image data to be displayed on the display based on the above.

With such a configuration, it can be applied to the case of stereoscopically viewing a real shot image, and it is possible to generate image data for this real shot image that is easily stereoscopically viewed.

In the stereoscopic vision system according to the thirteenth aspect, the gaze range determining means is all or one of a position, a color tone, a lightness, a saturation, a size, a complexity, and a movement of a pixel or an image area in a screen. It is preferable to have a function of estimating at least one image portion that is highly likely to be gazed at by the observer, by considering the parts.

[0029] In the stereoscopic vision system according to the thirteenth aspect, the image conversion determining means has a deviation of a reference point within the gazing range determined by the gazing part determining means between the left and right images within a viewing angle of 2 degrees. It is preferable to determine the conversion method so that

It is preferable that the gaze range determining means of the stereoscopic system according to the thirteenth aspect is constituted by a gaze / gaze range detecting means for detecting a gaze and a gaze range with respect to an actually displayed image.

It is preferable that the stereoscopic system according to the thirteenth aspect has means for changing the lightness and color tone of the displayed image so that the visibility becomes low in a portion other than the gaze portion.

According to a thirteenth aspect of the present invention, in the stereoscopic system, the amount of deviation between each of the pixels of the pair of images or between the left and right images of the image region changes by at least 1 degree in terms of the convergence angle within a predetermined time. It is preferable to have a means for preventing

It is preferable that the image data creating means of the stereoscopic vision system according to claim 13 further has a function of defining a stereoscopic vision condition of the graphic user interface item displayed on the display.

According to a fourteenth aspect of the present invention, in a stereoscopic viewing method for displaying a pair of images of the model viewed from two different viewpoints in a stereoscopic manner on a display, an observer may gaze at the model. A gaze range determining step of determining a high range, and by changing the data of the model, the model is orthogonal to the display surface such that the gaze range determined by the gaze range determining means falls within the easy stereoscopic view region. By moving the model moving step in the direction and changing the data of the model changed by the model moving means, the model enlargement for enlarging / reducing the size of the model moved by the model moving means in accordance with the moving distance. The reduction step, and the size is moved by the model moving means while being moved by the model moving means. Based on further modeled data after was, in which and an image data generation step of generating image data to be displayed on said display.

According to the fourteenth aspect of the present invention, the gaze range is estimated in the gaze range determining step, and the model is moved in the depth direction so that the gaze range falls within the easy stereoscopic view region. Next, in the model scaling step, the size of the model is changed according to the moving distance. The data of the model thus modified is sent to the image data creating step, and in the image data creating step, the image data displayed on the display is created.

According to a fourteenth aspect of the present invention, the scale of the depth distance viewed from the viewpoint is set so that at least a main part of the model moved by the model moving step falls within the easy stereoscopic view area. It is preferable to further include a depth distance reduction step.

The stereoscopic viewing method according to claim 14 further comprises a line-of-sight / gaze-range detecting step of detecting a line-of-sight and a gaze range with respect to an actually displayed image, and the model moving step includes the line-of-sight / gaze range detecting step. It is preferable to move the model in a direction orthogonal to the display based on the gaze range detected by.

[0038] In the model moving step of the stereoscopic method according to claim 14, when the placement condition of the model in the three-dimensional space viewed from the viewpoint changes and the model is moved according to this change, the placement condition is changed. It is preferable to have a step of preventing the movement amount of the gaze range of the model from being converted into a convergence angle and becoming approximately 1 degree or more within a predetermined time from the time when the change occurs.

According to the eighteenth aspect of the present invention, in the stereoscopic viewing method of displaying the model on the display so that the model can be viewed stereoscopically using a pair of images viewed from two different viewpoints, the observer gazes at the model. A gaze range determining step of determining a portion having a high possibility, and a perspective transform determining a method of perspective transforming the model data on the pair of left and right eye images based on the output of the gaze range determining step. And a step of creating image data for creating image data to be displayed on the display based on the perspective transformation method determined by the perspective transformation determining step.

According to such a configuration, it is possible to generate image data that is easily stereoscopically viewed by the perspective transformation method without actually moving and enlarging / reducing the model as in the twelfth aspect of the invention.

In the perspective transformation determining step of the stereoscopic viewing method according to claim 18, a deviation from a reference position in the pair of images of the gaze range image determined in the gaze range determining step becomes a viewing angle. It is preferable to determine the perspective transformation method so that it is within 2 degrees.

According to the eighteenth aspect, when the arrangement condition of the model viewed from the viewpoint changes, the stereoscopic vision method is created by the image data creating step by using the perspective transforming method created by the perspective transform creating step. It is preferable to have a step of preventing the change amount of the image shift from being equal to or larger than about 1 degree in terms of the convergence angle.

In the stereoscopic viewing method according to any one of claims 14 and 18, the gaze range determining step includes the layout condition, color tone, brightness, saturation, size, complexity of the model viewed from the observer's viewpoint.
It is preferable to have a step of estimating the gaze range of both eyes in consideration of all or part of the moving speed.

It is preferable that the stereoscopic vision method according to the fourteenth and eighteenth aspects further comprises an out-of-field-of-view data excluding step of excluding model data existing outside the field of view. in this case,
It further comprises a line-of-sight / gaze range detection step for detecting a line-of-sight and a gaze range for an actually displayed image, and the out-of-field-of-view data exclusion step calculates a field of view based on the line-of-sight detected by the line-of-sight / gaze range detection step. Then
It is preferable to exclude data that is outside the field of view.

It is preferable that the stereoscopic method according to the fourteenth and eighteenth aspects further comprises a color tone changing step of bringing the color tone of the displayed model closer to the color tone of the background color as the distance from the viewpoint increases in the depth direction.

In the stereoscopic vision method according to any one of claims 14 and 18, it is preferable that the image data creating step includes a step of defining a stereoscopic vision condition of the graphic user interface item displayed on the display.

According to a twenty-sixth aspect of the present invention, in a stereoscopic viewing method for displaying a pair of images viewed from two different viewpoints on a display in a stereoscopic manner, an image portion which an observer is likely to gaze at is displayed. A gaze range determination step to determine,
Based on the output of the gaze range determination step, an image conversion determination step of determining an image conversion method such that the gaze range becomes an area that facilitates stereoscopic viewing, and on the display based on the output of the image conversion determination step. And an image data creating step of creating image data to be displayed.

According to such a configuration, it can be applied to the case of stereoscopically viewing a real shot image, and it is possible to generate image data for this real shot image that is easily stereoscopically viewed.

The step of determining the gaze range of the method according to claim 26, wherein the position of the pixel or the image region in the screen is
It is preferable to estimate at least one image portion that is likely to be watched by the observer by considering all or part of the color tone, lightness, saturation, size, complexity, and movement.

In the image conversion determining step of the method according to claim 26, the deviation between the left and right images of the reference point within the gazing range determined by the gazing section determining step is within 2 degrees as a viewing angle. Therefore, it is preferable to determine the conversion method.

It is preferable that the gaze range determining step according to the twenty-sixth aspect is configured by a gaze / gaze range detecting step for detecting a gaze line and a gaze range with respect to an actually displayed image.

It is preferable that the method according to the twenty-sixth aspect includes a step of changing the lightness and color tone of the displayed image so that the visibility becomes low in a portion other than the gaze portion.

According to a twenty-sixth aspect of the present invention, the deviation amount between the left and right images of each pixel or image area of the pair of images is prevented from changing by more than 1 degree in terms of the convergence angle within a predetermined time. It is preferable to have a step of

The image data creating step of the method according to claim 26 preferably further comprises the step of defining a stereoscopic viewing condition of the graphic user interface item displayed on the display.

In a twenty-seventh aspect of the present invention, a computer system has two different models on the display.
In a storage medium storing a computer program for stereoscopically displaying a pair of images viewed from one viewpoint,
Gaze range determination command means for giving a command to the computer system to determine a range in which an observer is likely to gaze, and by the computer system by changing the data of the model, the gaze range determination command means Model movement command means for giving a command to move the model in a direction orthogonal to the display surface so that the focused gazing range falls within the stereoscopic easy area, and the model data changed by the model moving means to the computer system. By changing the model moving means, model scaling command means for giving a command to change the size of the model moved by the model moving means in accordance with the moving distance,
An image that gives a computer system a command to create image data to be displayed on the display based on the data of the model that has been moved by the model moving unit and has been resized by the model enlarging / reducing unit. And a data creation command means.

According to the twenty-seventh aspect of the present invention, first, the gaze range is determined, and the model is moved in the depth direction so that the gaze range falls within the easy stereoscopic view region. Then
The model enlarging / reducing means changes the size of the model in accordance with the moving distance. The data of the model modified in this way is used to generate the image data displayed on the display based on the image data generation command.

According to a 27th aspect of the present invention, there is provided a storage medium in which, from the viewpoint, the computer system is configured so that at least a main part of the model moved based on the command of the model moving command means is within the easy stereoscopic view area. It is preferable to further include depth distance reduction command means for giving a command to set the scale in the viewed depth direction.

The storage medium according to claim 27 further comprises line-of-sight / gaze range detection command means for giving a command to the computer system to detect the line-of-sight and gaze range of the actually displayed image, and the model movement command means. It is preferable to give a command to the computer system to move the model in a direction orthogonal to the display based on the gaze range detected based on the command of the gaze / gaze range detection command means.

In the storage medium according to the twenty-seventh aspect, the model movement command means changes the arrangement condition of the model in the three-dimensional space viewed from the viewpoint in the computer system, and moves the model in accordance with the change. In this case, a means for giving a command for preventing the movement amount of the gaze range of the model from becoming approximately 1 degree or more in conversion to the convergence angle within a predetermined time from the time when the change of the arrangement condition occurs may be provided. preferable.

According to a thirty-first aspect of the present invention, there is provided a storage medium for storing a computer program for causing a computer system to stereoscopically display a model on a display by using a pair of images viewed from two different viewpoints. , The model data based on the output based on the gaze range determination command to the computer system, and a gaze range determination command means for giving a command to the computer system to determine a part that the observer is likely to gaze at A pair of left and right perspective conversion determination command means for giving a command to determine a perspective conversion method on the image for the right eye, and the computer system based on the perspective conversion method determined based on the perspective conversion determination command. Image data creation command means for giving a command to create the image data displayed on the display; It is as it has.

According to such a configuration, it is possible to generate image data that is easy to stereoscopically view by the method of perspective transformation without actually moving and enlarging / reducing the model unlike the invention of claim 23.

In the storage medium according to the thirty-first aspect, the perspective transformation determination commanding means instructs the computer system from a reference position in the pair of images of the gaze range image determined by the gaze range determining means. It is preferable to have a means for giving a command to determine the perspective conversion method so that the deviation is within 2 degrees in terms of viewing angle.

The storage medium according to claim 31 is created by the image data creating means by using the perspective transforming method created by the perspective transform creating means when the arrangement condition of the model viewed from the viewpoint changes in the computer system. It is preferable to further include a command unit that gives a command for preventing the amount of change in the image shift, which is converted to a convergence angle, from being approximately 1 degree or more.

In the storage medium according to any one of claims 27 and 31, the gaze range determining command means instructs the computer system to arrange the model from the viewpoint of the observer, the arrangement condition of the model, color tone, brightness, saturation, size, and complexity. Now, it is preferable to have means for giving a command for estimating the gaze range of both eyes in consideration of all or part of the moving speed.

It is preferable that the storage medium according to the twenty-seventh and thirty-first aspects further comprises an out-of-field-of-view data exclusion command means for instructing the computer system to exclude the data of the model existing outside the field of view. In this case, the storage medium further includes a line-of-sight / gaze range detection command unit that gives the computer system a command to detect a line-of-sight and a gaze range for an actually displayed image, and the out-of-field-of-view data exclusion command unit is the In the computer system,
It is preferable that the visual field is calculated based on the line of sight detected based on the command of the gaze range detection commanding means, and a command for excluding data existing outside the visual field is given.

The storage medium according to the twenty-seventh and thirty-first aspects is characterized in that the color tone of the model displayed on the computer system is
It is preferable to further include a color tone change command unit that gives a command to approach the color tone of the background color as the distance from the viewpoint increases in the depth direction.

In the storage medium according to any one of claims 27 and 31, the image data creation command means has means for giving the computer system a command for determining a stereoscopic viewing condition of a graphic user interface item displayed on a display. Preferably.

According to a thirty-ninth aspect of the present invention, there is provided a storage medium for storing a computer program for causing a computer system to stereoscopically display a pair of images viewed from two different viewpoints on a display. A region in which the gaze range is easy to stereoscopically view, based on an output based on the gaze range determination command to the computer system, and a gaze range determination command means for giving a command to determine an image portion that is likely to be gazed by the observer. And an image which gives the computer system an instruction to create image data to be displayed on the display based on the output of the image conversion determination means. And a data creation command means.

According to such a configuration, the present invention can be applied to a case where a real shot image is viewed stereoscopically, and it is possible to generate image data for this real shot image that is easily stereoscopically viewed.

In the storage medium according to the 39th aspect, the gaze range determining command means instructs the computer system to select a position of a pixel or an image area in the screen, a color tone, a brightness, a saturation, a size, a complexity, and a movement. It is preferable to have a command unit that gives a command to estimate at least one image portion that is likely to be gazed by the observer by considering all or a part of the above.

In the storage medium according to the 39th aspect, the image conversion determination command means causes the computer system to determine, as a viewing angle, a deviation between the left and right images of the reference point within the gazing range determined by the gazing section determination command. It is preferable to have a means for giving a command to determine the conversion method so that the conversion method is within 2 degrees.

In the storage medium of claim 39, the gaze range determination command means is constituted by a gaze / gaze range detection command means for giving a command to the computer system to detect a gaze line and a gaze range for an actually displayed image. 40. The storage medium according to claim 39, preferably further comprising means for giving a command to the computer system to change the brightness and color tone of the displayed image so that the visibility is reduced in a portion other than the gaze portion. It is preferable to have.

According to a thirty-ninth aspect of the present invention, in the storage medium according to the thirty-ninth aspect, in the computer system, a shift amount between the left and right images of each pixel or image area of the pair of images is converted into a convergence angle within a predetermined time of approximately 1 degree. It is preferable to have a means for giving an instruction to prevent the above changes.

In the storage medium of the thirty-ninth aspect, it is preferable that the image data creation commanding means further has means for giving a command for determining a stereoscopic viewing condition of the graphic user interface item displayed on the display.

According to a forty-third aspect of the present invention, which is a storage medium storing image data, the left-eye and right-eye images are displayed on a display, and these images are independent for the left-eye and the right-eye. In a storage medium applied to a stereoscopic viewing system, the storage medium stores an image conversion method for locating the gaze range in the image in the easy stereoscopic viewing region for each scene. It is characterized by having a storage part for

According to this structure, a storage medium capable of providing image data that facilitates stereoscopic viewing can be obtained by applying it to the stereoscopic system (storage medium reproducing device).

The storage medium according to claim 40, wherein the image conversion method comprises: an image enlargement ratio for locating the gaze range in the easy stereoscopic view region centering on the viewpoint, the depth distance, and the distance from the viewpoint to the display screen. It is preferable to include each data.

The invention of claim 41 is a storage medium manufacturing system for manufacturing the storage medium according to claim 40, wherein a gaze range deciding means for deciding an image portion which is highly likely to be gazed by an observer in the image data. And an image conversion deciding means for deciding an image converting method such that the gaze range becomes an area where stereoscopic viewing is easy, based on the output of the gaze range deciding means,
The image conversion method further comprises means for writing the image data in the storage medium together with the image data.

With this structure, it is possible to manufacture the storage medium in which the stereoscopic information is stored.

The gaze range determining means of the system according to claim 41 is the position of the pixel or the image area in the screen, the color tone,
It is preferable to have a function of estimating at least one image portion that is likely to be gazed by the observer by considering all or some of the brightness, the saturation, the size, the complexity, and the movement.

In the image conversion determining means of the system according to claim 41, the deviation between the left and right images of the reference point within the gazing range determined by the gazing part determining means is within 2 degrees in terms of viewing angle. It is preferable to determine the conversion method.

It is preferable that the gaze range determining means of the system according to the 41st aspect is constituted by a gaze / gaze range detecting means for detecting a gaze and a gaze range with respect to an actually displayed image.

It is preferable that the system according to claim 41 has a means for changing the lightness and color tone of the displayed image so that the visibility becomes low in a portion other than the gaze portion.

The system according to claim 41 preferably further comprises means for determining a stereoscopic viewing condition of the graphic user interface item displayed on the display.

According to a forty-second aspect of the present invention, in a stereoscopic system for generating image data to be displayed on a display by using the storage medium according to the forty-fourth aspect, the image data stored in the storage medium is stored in the storage medium. And a means for generating a pair of left-eye and right-eye images to be displayed on the display by processing based on the image data conversion method stored in.

With such a configuration, it is possible to reproduce the storage medium in which the stereoscopic information is stored and create an image that is easily stereoscopically viewed.

[0087]

BEST MODE FOR CARRYING OUT THE INVENTION The embodiments of the present invention will be described below, but before that, the principle of stereoscopic vision will be described.

FIG. 10 shows a solid represented by the distance from the center of the viewpoint (the midpoint of the line segment connecting the left eye and the right eye. The same applies hereinafter in this specification) and the distance from the center of the viewpoint. It is a graph showing the relationship between the visible region and the easy stereoscopic region (Source: From Physiological Engineering 15 "3D Display" by Toyohiko Hatada published in OplusE (magazine name)).

Here, the distance from the viewpoint center to the display surface means the distance from the viewpoint center to the position where both eyes are in focus. In addition, the size of the stereoscopically viewable (easy) region indicates the allowable range of fluctuation of the convergence angle. As can be seen from FIG. 10, the smaller the distance from the viewpoint center to the display surface, the narrower the stereoscopically viewable (easy) region.

Consider a case where the binocular stereoscopic system is applied to a CAD system for displaying and producing a three-dimensional shape.
In this case, if the distance between the center of the user's viewpoint of the system and the display surface is 60 cm, the depth distance from the center of the line of sight in the stereoscopic viewable area is An 2 (cm) to Af.
The area is up to 2 (cm). Furthermore, in the area where stereoscopic viewing is possible with less eye strain, that is, in the area where stereoscopic viewing is easy, the depth distance from the center of the line of sight is from An1 (cm) to A
Limited to the area up to f1 (cm).

However, the model (display target) is not always located between An1 and Af1. Therefore, in this case, it is necessary to change the stereoscopic viewing conditions so that all of the models are in a region where the stereoscopic image is easy to see.

On the other hand, in the case of a normal image (real image such as a photograph or video image) that does not have three-dimensional position data, the region where the stereoscopic image is easily seen is the right eye image and the left eye image. It can be expressed as a critical value of deviation. This value is based on the literature (The CrystalEyes Handbook: L.Lipton, StereoGraphi
According to cs Corp. 1991), the viewing angle is about 1.5 degrees. Moreover, even if it is slightly overestimated, it will not exceed twice. When the image is perceived deeper than the display surface, the maximum image shift is the same as the distance between the eyes, so if the distance between the center of the viewpoint and the display surface is 1 m or more, all areas become stereoscopic viewing areas. However, when the image is to be perceived in front of the screen, this condition cannot be satisfied if the image protrudes from the screen to some extent. Therefore, as in the case of the stereo model as described above, in order to make the stereoscopic image easier to see, the shift between the right-eye image and the left-eye image is appropriately changed so that the shift amount is always below the critical value. It is necessary to change the conditions.

However, the conventional stereoscopic system does not have means for obtaining and controlling the stereoscopic condition according to the change of the display image. For this reason, the cameraman must create an image in advance at the time of shooting in consideration of legibility, and the user himself cannot change the display interactively.

The present invention has a function of controlling the stereoscopic viewing condition based on the gaze range information for the scene and generating a stereoscopic image which is easy for the observer to see.

Embodiments of the present invention will be described below with reference to the drawings.

First Embodiment First, the first embodiment will be described. 1 to 1
5 is a diagram showing a first embodiment of the present invention.

First, the outline of the stereoscopic system according to the present invention will be described with reference to FIG. In addition, the case where the stereoscopic system is a time-division type stereoscopic system will be described below.

FIG. 1 shows a state in which an image 7a for the left eye and an image 7b for the right eye are simultaneously displayed on the display 1 (which may be a normal CRT). Left eye image 7a
Consists of a left-eye background 8a and a left-eye image 9a projected on this background. The image 7b for the right eye includes a background 8b for the right eye and an image 8b for the left eye projected on the background.

In practice, the left-eye image 7a and the right-eye image 7b are displayed alternately at high speed, so that for the observer, one screen 7, one background 8 and one image 9 are displayed.
Will be recognized as being displayed three-dimensionally.

That is, the display 1 is connected to the arithmetic unit 5 for outputting an image signal. The arithmetic unit 5 calculates the stereoscopic viewing condition of the image 9 displayed on the display 1, creates or processes the left eye image data and the right eye image data based on the stereoscopic viewing condition, The image 7a for right eye and the image 7b for right eye are alternately sent to the display 1 at a predetermined cycle.

The arithmetic unit 5 is provided with an input means 6, for example, a keyboard 6a, a mouse 6b, a dial, for instructing processing such as rotation, movement and coordinate designation of an image displayed on the display surface 1a of the display 1. 6c etc. are connected.

This stereoscopic system also has liquid crystal shutter glasses 2. This liquid crystal shutter glasses 2
The input of light to one eye is blocked in synchronization with the switching cycle of the image for left eye 7a and the image for right eye 7b generated by the arithmetic unit 5. That is, the human left eye 3
a is a left-eye image 7 out of the images 7 displayed on the display 1 through the left-eye lens 2a of the liquid crystal shutter glasses 2.
Only the image a is viewed, and the right eye 3b is configured to view only the image 7b for the right eye among the images 7 displayed on the display 1 through the lens 2a for the right eye of the liquid crystal shutter glasses 2.

Therefore, if appropriate stereoscopic viewing conditions are set, the user can perceive a stereoscopic image in which the left-eye image 9a and the right-eye image 9b are fused. Further, the distance Za between the midpoint of the line segment connecting the left eye 3a and the right eye 3b (hereinafter referred to as the center of the viewpoint) and the center 1b of the display surface 1a changes according to the individual preference of the user (the position at which the screen is viewed). However, in the following description, the description will be made on the assumption that the distance Za is constant.

Hereinafter, an example of the configuration of the stereoscopic system will be described in detail with reference to FIG.

As shown in FIG. 2, the stereoscopic system has a control means 10 for controlling the system.

The control means 10 has a function of receiving a command input from the input means 6 and creating a command to be given to the system.

The control means 10 is connected to a recording means 11 for recording information such as the shape, size and arrangement state of the model existing in the three-dimensional space.
The model information as described above is received from.

The control means 10 is connected to a GUI data management means 21 for managing data related to graphic user interface items such as character information displayed on the display 1 and menus and buttons.
The above information is received from the data management means 21.

Further, this system has a stereoscopic condition calculating unit for calculating a stereoscopic condition. The stereoscopic condition calculating unit includes an out-of-field data excluding unit 12 that excludes model data existing outside the field of view, and a gaze range that estimates a gaze range (a range including the gaze point) of both eyes based on the model data. The estimation unit 13, the model moving unit 14 that performs the process of moving the model in the depth direction, the model scaling unit 15 that performs the scaling process of the model, and the reference for creating the left-eye image and the right-eye image. The depth distance reducing means 16 for setting the depth reduction distance, the color tone changing means 17 for changing the color tone of the model to give the effect of aerial perspective to the displayed image, and the model arranged in the coordinate system of the three-dimensional space are provided. Coordinate system conversion means 18 for appropriately arranging in the coordinate system of the stereoscopic system, and image shift amount adjustment means 2 for adjusting the shift amount between the left-eye image and the right-eye image according to eye fatigue.
And a screen edge processing unit 23 for blurring the screen edge.
It has and.

These out-of-field data excluding means 12, gaze range estimating means 13, model moving means 14, model enlarging / reducing means 15, depth distance reducing means 16, color tone changing means 1
The coordinate system converting means 18, the screen shift amount adjusting means 22, and the screen edge processing means 23 are connected to the model data managing means 19, respectively. This model data management means 1
9 is adapted to send the data received from each of the above-mentioned means 12 to 18 to the image data creating means shown in FIG. 20 based on the command from the control means 10.

This image data creating means 20 creates the data for the left eye image 7a and the data for the right eye image 7b displayed on the display 1 based on the model data received from the model data managing means 19. , Sends the signal to the display 1.

These means 10 to 21 are constituted by the arithmetic unit 5 (FIG. 1) provided in the stereoscopic system. It may be realized by installing a computer program in which a procedure for executing each of the means 10 to 21 is described in a computer system.

Next, the operation of this embodiment will be described.

Before specifically explaining the operation, the coordinate system of the stereoscopic system used in the present specification will be described with reference to FIG. As shown in FIG. 3, the coordinate system of the stereoscopic system is represented by an XYZ coordinate system whose origin is a midpoint O between the left eye 3a and the right eye 3b.

In this coordinate system, the X axis coincides with the straight line connecting the left eye 3a and the right eye 3b and extends parallel to the display surface 1a of the display 1. Here, the left eye 3a direction is defined as a positive direction.

The Z axis coincides with a straight line connecting the origin O and the center 1b of the display surface 1a. From origin O to center 1
The direction toward b is the positive direction. The Z axis coincides with the center of the line of sight.

The Y axis passes through the origin O and extends in the direction perpendicular to the XZ plane. Here, the upward direction is the positive direction. Further, the length of a line segment connecting the left eye 3a and the right eye 3b is referred to as an inter-eye distance, and the midpoint of the line segment connecting the left eye 3a and the right eye 3b, that is, the origin O is referred to as the viewpoint center.

The operation of this embodiment will be described below with reference to the flowcharts shown in FIGS.

As shown in FIG. 4, after the stereoscopic system is started up, the input means 6 first inputs the initial value to the control means 10 (step S100). Here, the initial value refers to an initial set value of the inter-eye distance which is a basis of the determination when determining the shift amount of the image 7a for the left eye and the image 7b for the right eye later, and on the coordinate system of the three-dimensional space. Showing how to arrange the model placed on the coordinate system of the stereoscopic system, the viewpoint center O and the center 1b of the display surface 1a.
And the distance Za (= 60 cm), etc.
Remembered in. The eye-to-eye distance as an initial value is set to a value equal to the actual eye-to-eye distance of a human, for example, 6 cm.

When the input of the initial value is completed, the control means 10 then reads the data of the model corresponding to the coordinate system of the three-dimensional space from the recording means 11 (step S101),
It is sent to the model data management means 19 together with the initial value.
The model data here means the model's dimensions and shape,
Data such as the arrangement state of the model in the coordinate system of the three-dimensional space and the color tone of the model (including all of hue, lightness, and saturation; the same in this specification).

Next, the model data managing means 19 sends the initial value and model data to the coordinate system converting means 18,
The coordinate system conversion means 18 arranges the model in the coordinate system of the stereoscopic system as shown in FIG. 6 based on these initial values and the model data (step S102).

Next, data is input to the model data management means 19 via the control means 10 by the input means 6 including the keyboard 6a, the mouse 6b, the dial 6c, etc.
The display position, the posture (meaning the direction in which the model is facing), etc. of the model arranged on the coordinate system of the stereoscopic system are adjusted (step S104).

Next, the model data managing means 19 sends the data of the model arranged in the coordinate system of the stereoscopic system to the out-of-view data excluding means 12, and the out-of-view data excluding means 12 is shown in FIG. As shown in, the data of the model (see the broken line in FIG. 7A) existing outside the visual field is deleted from the model data (S step S105). Step S
The data of the model changed by 105 (see the solid line in FIG. 7A) is returned to the model data management means 19 and the data of the model is rewritten.

Next, the model data managing means 19 sends the model data to the gaze range estimating means 13. The gaze range estimation means 13 is used by the user for the model placed in the coordinate system of the stereoscopic system based on the placement condition of the model received from the model data management means 19 in the three-dimensional space and the above-mentioned initial value. The gaze range in the depth direction of the eye is estimated (step S106). In this embodiment, for example, the estimation of the gazing point is performed by one of the two methods described below.

First, the first method will be described. First, as shown in FIG. 8, each element Ei included in the model
Envelope cuboid Ri (i that encloses (i = 1 to N) respectively
= 1 to N) are set.

Next, using each envelope rectangular parallelepiped Ri as one unit, the weight W of the representative point of the element Ei enveloped in each envelope rectangular parallelepiped Ri is weighted.
Find i. In determining the weight, first, the number of vertices of each envelope rectangular parallelepiped Ri is calculated. In addition, each envelope rectangular parallelepiped R
Although the number of vertices i has is usually 8, as for the envelope rectangular parallelepiped R1 existing at the boundary of the visual field as shown in FIG. 8, only the number of vertices (vertical circles in FIG. 8) existing in the visual field is used. It is calculated, and this is used as the number of vertices of the envelope rectangular parallelepiped R1.

Next, the coefficient expressing the complexity of the element Ei enveloped in each envelope rectangular parallelepiped Ri is calculated. This coefficient is calculated based on the number of sides and faces included in the element Ei enveloped in each envelope rectangular parallelepiped Ri when each element is a shape model on a computer, and based on the spatial frequency in the case of an image. It is calculated. This coefficient is the envelope rectangular parallelepiped Ri
And the weight Wi of each envelope rectangular parallelepiped is calculated.

Next, the coordinates (xi, yi, zi) of the representative point of each envelope rectangular parallelepiped Ri are obtained. Further, the Z coordinate Zi of the coordinates (xi, yi, zi) of the representative point is defined as the depth distance Zi of the element from the viewpoint. The element E
The coordinates of the representative point of i are the coordinates of the center of gravity of the element Ei enveloped in each envelope rectangular parallelepiped Ri.

Next, the distance di between the representative point of each envelope rectangular parallelepiped Ri and the line-of-sight center (Z axis of the stereoscopic system coordinate system) is calculated.

Next, the weight Wi of the representative point of each envelope rectangular parallelepiped Ri obtained as described above, the distance Zi from the viewpoint center of the representative point of each envelope rectangular parallelepiped Ri, the representative point of each envelope rectangular parallelepiped Ri and the line-of-sight center ( The depth coordinate Zd of the gazing point is calculated by Equation 1 using the distance di from the Z axis).

[0131]

[Equation 1]

In Expression 1, N is the number of envelope rectangular parallelepipeds, and Z
i is the depth coordinate of the representative point of the i-th enveloped rectangular parallelepiped, Wi is the weight of the i-th enveloped rectangular parallelepiped, and di is the distance from the line-of-sight center of the representative point of the i-th enveloped rectangular parallelepiped.

The standard deviation σ of the depth coordinates of the representative point is calculated by the mathematical formula 2.

[0134]

[Equation 2]

From this, the gazing range is calculated as in Expression 3 using the depth coordinate Zd of the reference point, the standard deviation σ, the minimum value Zi0 of Zi, and the maximum value Zi1 of Zi.

[0136]

[Equation 3]

Here, max (a, b) is a, b
Is a function that returns the larger of the two, min (a, b)
Is a function that returns the lesser one.

In the equation 3, a is a coefficient of a real number, and any number may be applied, but normally it takes a value from 0 to 2.

Next, the second method will be described. Figure 9
As shown in FIG. 1, first, a Voronoi polyhedron is created in each envelope rectangular parallelepiped Ri in the envelope rectangular parallelepiped Ri with the representative point Tij as a reference.
The Voronoi polyhedron is for dividing a space into the following regions based on a plurality of (n) given points (the region of points Tij (j = 1 to n) is Vi (Tij)). Vi (Tij) = {T | d (Ti, Tij) <d (Ti, Tik)} (k = 1n) k ≠ j where Ti is a point in the envelope rectangular parallelepiped Ri, and d (Ti, Ti
j) represents the distance from the point Tij to Ti.

That is, in the envelope rectangular parallelepiped Ri, the area Vi
The point Ti included in (Tij) is Ti that is greater than that of any other representative point.
It will be close to j.

Next, the coefficient expressing the complexity of the element Ei enveloped in each divided area Vi (Tij) is calculated. When each element is a shape model on a computer, this coefficient is an element Ei that is enveloped in each area Vi (Tij).
It is calculated based on the number of sides and faces included in, and in the case of an image, it is calculated based on the spatial frequency. This coefficient becomes the weight Wij of each area.

Next, the coordinates (xij, yi) of the representative point Tij
j, zij), the Z coordinate Zij is defined as the depth distance Zij from the viewpoint center O of each representative point.

Next, the distance dij between each representative point and the line-of-sight center (Z axis of the stereoscopic system coordinate system) is calculated.

Next, as described above, each envelope rectangular parallelepiped Ri
Using the weight Wij of each representative point Tij obtained for each, the depth distance Zij from the viewpoint center O of each representative point Tij, and the distance dij between each representative point Tj and the line-of-sight center (Z axis), The depth coordinate Zd of the gazing point is calculated by the following mathematical formula 4.

[0145]

[Equation 4]

In Equation 4, N is the number of envelope rectangular parallelepipeds and Z
ij is the depth coordinate of the representative point of the j-th area of the envelope rectangular parallelepiped Ri, Wij is the weight of the j-th area of the envelope rectangular parallelepiped Ri,
dij is the distance from the line-of-sight center of the j-th area of the envelope rectangular parallelepiped Ri, and i is the envelope rectangular parallelepiped Ri of the i-th eye.

Further, the standard deviation σ of the depth coordinates of the representative point is calculated by the mathematical expression 5.

[0148]

[Equation 5]

From this, the gazing range is calculated as shown in Expression 6 using the depth coordinate Zd of the reference point, the standard deviation σ, the minimum value Zi0 of Zi, and the maximum value Zi1 of Zi.

[0150]

[Equation 6]

In the expression 6, a is a coefficient of a real number, and any number may be applied, but normally it takes a value from 0 to 2.

In the above formula, the weight W is multiplied by the distance d from the center of the line of sight of the representative point, but a formula such as a × W + b × d (where a and b are coefficients) may be used.

In both the first method and the second method, Ri is a rectangular parallelepiped that envelops each element Ei (i = 1 to N) of the model. It may be a rectangular parallelepiped enclosing the visible surface.

In the following, as an example, a = 0, that is,
A method of moving the model based on the reference point of the gazing range (hereinafter referred to as "gazing point P") will be described.

The depth coordinate (Z coordinate) Zd of the gazing range obtained by the above-mentioned first method or second method is sent to the model data management means 19 and stored therein.

Next, the model data managing means 19 delivers the model data and the data of the depth coordinate Zd of the gazing range to the model moving means 14. Next, the model moving unit 14 determines whether the image displayed this time, that is, the image currently subjected to the above processing is a new image or a changed image (step S107).

Here, the changed image is an image obtained by simply rotating the image displayed on the display surface 1a up to immediately before or changing the posture (see step S104), that is, having continuity with the immediately preceding image. An image.
The new image is an image other than that.

When the image is a new image, the model moving means 1
4 is based on the depth coordinates Zd of the gazing range calculated by the gazing range estimating means 13, and as shown in FIGS. 7B and 7C, the gazing point P is within the stereoscopic view easy region shown in FIG. The entire model is translated in the depth direction (Z-axis direction) so as to enter (step S110). In this embodiment,
Since the distance from the viewpoint center O to the center of the display surface 1a is 60 cm, the gazing point has a depth distance from the viewpoint center O shown on the horizontal axis of the graph in FIG. 10 from An1 (cm) to A.
It may be located within the range of f1 (cm).

When the image is a modified image, the model moving means 1
4 indicates that the gazing point position, assuming that the process of step S110 has been performed as it is, moves once or more in terms of the convergence angle with respect to the gazing point position of the image immediately before the image currently being processed. It is determined whether or not (step S10
8) If it is once or less, perform step S in the same manner as above.
The process of 110 is performed. Here, the criterion is set to 1 degree or less because the burden on the eye becomes too large when the gazing point position moves by 1 degree or more in terms of the convergence angle.
Since there is an individual difference in whether or not the burden on the eyes is felt, the criterion may be set to a value of 1 degree or less, for example, 0.5 degree.

On the other hand, the model moving means 14 converts the gaze point position on the assumption that the processing of step S110 is performed as it is into the convergence angle with respect to the gaze point position of the image displayed immediately before, When moving more than once, a predetermined limit is applied to the amount of movement of the model. That is, in this case, the model moving means 14 restricts the position of the point of interest after movement to the position of the point of interest of the immediately preceding image so as not to move by 1 degree or more in terms of the convergence angle, and the position of the point of interest should be as much as possible. The entire model is translated in the depth direction (along the Z-axis) so as to approach the stereoscopic view easy region (step S109). The position data of the model moved in this way (and also to that effect when the amount of movement of the gazing point is limited) is sent to and stored in the model data management means 19.

Next, the model data managing means 19 sends the model moving distance data to the model enlarging / reducing means 15.
The model enlarging / reducing means 15 which has received this data changes the size of the model in accordance with the moving distance (step S111). For example, if the gazing point P is the display surface 1a
(The size of the model before the movement): (the size of the model after the movement) = (Z coordinate Zd of the gazing point P before the movement): (Z coordinate of the gazing point after the movement) The size of the model is changed in proportion to the distance of the gazing point from the center of the viewpoint so that This allows
Since the apparent size of the model before and after the movement (the size of the image reflected on the user's retina) is the same, the user does not notice the change in the depth distance of the model. By doing so, when the model is rotated sequentially and the model is viewed from different angles, the user does not feel discomfort that the model suddenly approaches or moves away when the screen is rewritten. , Is effective in reducing fatigue of the eyes of the user. The model size data changed as described above is sent to and stored in the model data management unit 19.

Although the proportional coefficient is set here so that the size of the appearance of the model does not change, it is not always necessary to be faithful to this.

Next, the model data management means 19 sends the model position data and the model size data to the depth distance reduction means 16. Here, the perceptual depth distance is reduced by changing the inter-eye distance. Upon receiving the data, the depth distance reduction unit 16 determines whether or not the entire model is within the stereoscopic view easy region based on the graph shown in FIG. 11 based on the interocular distance set as the initial value (step S112). ).

If a part of the model is outside the area for easy stereoscopic viewing, the depth distance reducing means 16 corrects the inter-eye distance set as an initial value to be small (step S).
113). By reducing the inter-eye distance from D0 to D1, as shown in FIGS. 12A and 12B, the shift amount between the left-eye image and the right-eye image displayed on the display surface 1a is d.
It is reduced from 0 to d1. Whether or not stereoscopic viewing is easy is determined by whether or not the amount of displacement between the left-eye image and the right-eye image is within a predetermined range.Therefore, the entire model can be stereoscopically viewed by appropriately reducing the inter-eye distance. It becomes possible to fit within the easy area.

FIG. 11 is a graph showing the correction standard of the interocular distance when the distance from the viewpoint center to the display surface 1a is constant, where the vertical axis is the interocular distance and the horizontal axis is the viewpoint center of the model. Depth distance from O. The method of creating this graph will be described below.

First, based on FIG. 10, the distance An1 (cm) from the viewpoint center O at both ends of the stereoscopic view easy region corresponding to the depth distance (60 cm in this embodiment) from the viewpoint center O to the display surface 1a, Calculate Af1 (cm). As a result, the depth distance from the viewpoint center O at both ends of the stereoscopic view easy region in the case of the original inter-eye distance D0 set as the initial value is obtained. As a result, the points L and M in FIG. 11 are determined.

Next, based on FIG. 13 (a), assuming that the element E of the model is located at the front side end of the easy stereoscopic region, from the inter-eye distance D0 and the viewpoint center O of the element E. The positions of the left-eye image IL and the right-eye image IR when the element E is displayed on the display surface 1a are obtained based on the distance An1 of
A shift amount d0 between the left-eye image IR and the right-eye image IL is calculated.
The shift amount d0 is the maximum allowable shift amount for the images IL and IR that display the elements located on the outer side of the easy stereoscopic region.

Next, as shown in FIG. 13A, in the case of the inter-eye distance D0, the model is located outside the front side of the easy stereoscopic viewing area and the depth distance from the viewpoint center O is An. Consider element E '. Next, straight lines passing through the element E ′ are created from the left-eye image IL and the right-eye image IR, and the intersections of these straight lines with the X axis are obtained. The distance D between these intersections is the maximum value of the inter-eye distance D that enables stereoscopic viewing easily with respect to the model element E ′ located at the position where the depth distance from the viewpoint center O is An.

However, when the model element E ′ is extremely close to the center of the viewpoint, for example, when the depth distance from the center O of the viewpoint is 1/10 or less of the distance Ai from the center O of the viewpoint to the screen, the inter-eye distance becomes extremely large. Since it is small, there is almost no parallax and a stereoscopic effect cannot be obtained. Therefore, when the depth distance of the model element E ′ is 1/10 or less of Ai from the viewpoint center O, the interocular distance is the same as when the model element E ′ is 1/10 of Ai from the viewpoint center O. . When the model is extremely close to the center of the viewpoint, the apparent size of the model becomes large, so that the part protruding from the field of view increases and it becomes difficult to recognize the shape as a whole. Therefore, it does not matter if an image is displayed that exceeds the maximum allowable shift amount defined above.

In this way, the depth distance An from the viewpoint center O is changed, and the maximum value of the interocular distance D corresponding to the changed distance An is obtained, and the graph (distance An,
By plotting the interocular distance D) corresponding to the distance An, the curve KL in the graph of FIG. 11 can be determined. With this curve KL, it is possible to determine the inter-eye distance at which the element located on the front side of the easy stereoscopic view area can easily perform stereoscopic view.

Further, as shown in FIG. 13B, a curve MN indicating the correction standard of the inter-eye distance for facilitating the stereoscopic viewing of the elements located on the outer side of the stereoscopic viewing easy region is obtained in the same manner as the above method. Can be set.

Based on the graph of FIG. 11 obtained as described above, the depth distance reducing means 16 corrects the inter-eye distance set as the initial value to be small. For example, as shown in the graph of FIG. 11, it is assumed that each element of the model is distributed in the range indicated by the arrow F with the gazing point P as the center. In this case, when the depth distance from the viewpoint center O of the element located closest to the front in the range indicated by the arrow F is An, if the inter-eye distance is set to D according to the curve KL, all the elements of the model are stereoscopically viewed. Can be easily performed.

Looking at the distribution of each element forming the model, for example, when only one element is located far behind the positions of other elements, all the elements are not necessarily in the stereoscopic view easy region. You don't have to enter. That is, at least the main part of the model may be included in the easy stereoscopic view region.

The case where a part of the model is outside the stereoscopic view easy area has been described above. However, if the entire model is within the stereoscopic view easy area at the end of step S111, the depth distance reducing means 16 is The eye distance is maintained at the initial value.

The inter-eye distance set as described above is
It is sent to and stored in the model data management means 19.

When the processing is performed on the basis of the entire gazing range instead of the gazing point P, the position of the foreground of the gazing range is used instead of the foremost element of the model, and the gazing point is used for the deepest element of the model. By using the position at the back of the range as a reference, the model can be moved, enlarged, and the interocular distance can be changed in the same manner as described above.

Next, the model data managing means 19 sends the model position data, the model size data, and the interocular distance data to the color tone changing means 17. The color tone changing unit 17 determines whether or not the effect of the air see-through is necessary for the displayed image based on these data (step S114).

The case where the effect of the air see-through is required is, for example, the case where there are many model elements in the depth direction from the gazing point. In such a case, even if the depth-side elements (Z-coordinates) of the elements at the back are different from each other, the amount of displacement between the binocular images does not change so much when the image is actually displayed on the display 1, so that the image is viewed. This is because it is difficult for humans to recognize the difference in depth between each element.

As another example of the case where the effect of the air see-through is required, the inter-eye distance set by the depth distance reducing means 16 may be small. The reason is that when the inter-eye distance is set small, the amount of deviation between the left-eye image and the right-eye image displayed on the display 1 is smaller than when the inter-eye distance is set large. Therefore, it is difficult for a person who views the image to recognize the difference in depth between the elements.

When it is judged that the effect of fluoroscopy is necessary,
The color tone changing unit 17 modifies the color tone of the model so as to approach the background color as the depth distance from the viewpoint center O increases, that is, as the Z coordinate of the element E of the model increases (step S115). When it is determined that the effect of the fluoroscopy is not necessary, the color tone changing unit 17 maintains the color tone of the model as it is.

The effect of the air see-through may be always added or may not be always added.

The color tone of the model changed (or maintained) as described above is sent to the model data management means 19 and stored therein.

As described above, step S115
When the processes up to are completed, the model data management unit 19 causes the model 1 to display on the display 1 the position of the model to be displayed on the display 1 in the coordinate system of the stereoscopic system, the size (including the shape) and the color tone of the model. The inter-eye distance, which is the basis of calculation of the display position of the model images (left-eye image and right-eye image) to be displayed on the screen, is held.

Next, the model data managing means 19 sends the above data to the image data creating means 20. In addition, the control unit 10 controls the GUI data management unit 21 to display the display 1
Image data creating means 2 for reading data related to graphic user interface items to be displayed on
Deliver to 0. The image data creating means 20 places the graphic user interface item on or near a plane parallel to the XY plane having the same Z coordinate as the gazing point after being moved by the model moving means 14 in the coordinate system of the stereoscopic system. Arrange (step S11
6).

The image data creating means 20 determines the display position and the display size as the image for the left eye and the image for the right eye for each element of the model on the display surface 1a based on the above data, and The position of the graphic user interface item to be displayed on the display unit 1 in the coordinate system of the stereoscopic system is determined (step S117).

The operation of the image data creating means 20 will be specifically described below with reference to FIGS. 14 (a) and 14 (b). First,
As shown in FIG. 14A, the element E1 that constitutes the model
Is the display surface 1a in the coordinate system of the stereoscopic system.
The case of being on the front side will be described.

First, a straight line L1 passing through the left eye 3a and the center CE1 of the element E1 (see the alternate long and short dash line in FIG. 14A) is set. Next, the intersection C between the straight line LL and the display surface 1a
L1 is obtained, and the intersection CL1 is determined as the center position (display position) of the left-eye image IL1 of the element E1 displayed on the display surface 1a. Then, the ratio of the distance between the left eye 3a and the element E1 and the distance between the left eye 3a and the intersection point CR1,
The size of the image IL1 for the left eye is determined so that the ratio of the size of the element E1 to the size of the image IL1 for the left eye is equal, and the image data for the left eye is created based on this determination. The center position CR1 and size of the right-eye image IR1 are determined by the same method, and the right-eye image data is created based on this determination.

Note that FIG. 14B shows the case where the element E2 that constitutes the model is on the back side of the display surface 1a in the coordinate system of the stereoscopic system, and in this case also, according to FIG. The center positions and sizes of the left-eye image and the right-eye image are determined by the same method as described above.

The image data creating means 20 also applies to the graphic user interface item,
The same processing as above is performed to determine the display positions and sizes of the left-eye image and the right-eye image.

Further, the element E1 and the element E2 are displayed at the same time, and in the above-mentioned step S115, the color tone changing means 17 corrects the color tone of the model so as to approach the background color as the depth distance from the center of the viewpoint increases. If it is, the image data creating means 20 uses the element E2.
The color tones of the image IL2 for the left eye and the image IR2 for the right eye corresponding to
Image data that is closer to the background color than the color tones of the left-eye image IL1 and the right-eye image IR1 corresponding to the element E1 is created.

Further, when the element E1 and the element E2 are displayed at the same time, and the images for the left eye and / or the right eye corresponding to the elements E1 and E2 respectively overlap each other, the image data creating means 20 is Element E1 on the side
Of the image data corresponding to the element E2 on the back side is deleted, and the image data of the portion overlapping with the image corresponding to the element E1 on the front side is deleted.

Next, the image data creating means 20 sends the image data for the left eye and the image data for the right eye to the display 1, and the display 1 displays an image on the display surface 1a based on the image data (step). S11
8).

Next, the control means 10 inquires whether or not the stereoscopic system will be continuously used (step S1).
03). Here, for example, when the user wants to change the orientation of the model for display, the operation of step S104 is performed, and then the operations of step S105 and subsequent steps are performed.

As described above, according to this embodiment, the image representing the model can be automatically displayed on the display under the optimum stereoscopic viewing condition. Therefore, it is possible to reduce the feeling of discomfort in the image that the user feels and the eye fatigue.

In the stereoscopic vision system of this embodiment, the following method is adopted in order to further reduce the fatigue of the eyes of the user. That is, the binocular fusion type stereoscopic system is compared with a system which performs a normal two-dimensional display.
After all, it sometimes gives a certain burden to the eyes of the observer. Therefore, in this system, in order to further alleviate the burden on the observer, as shown in FIG. 1, the shift amount adjusting means 22 for adjusting the shift amount between the image for the left eye and the image for the right eye.
Have.

The deviation amount adjusting means 22 monitors the image display time, and when a certain time has elapsed from the start of display, the deviation amount adjusting command is sent to the image data creating means 20 via the model data managing means 19. Emit.

The effect of stereoscopic vision depends on the amount of deviation (the distance between CL1 and CR1 in FIG. 14A) between the image for the left eye and the image for the right eye. The smaller the amount of deviation, the smaller the effect of stereoscopic vision. Become. On the other hand, if this shift amount becomes large, the stereoscopic effect becomes large, but if it becomes too large, a person cannot combine the images for the left eye and the right eye, that is, stereoscopic vision becomes impossible.

In this embodiment, the data creating means 2
0 is CL1 in FIG. 14 (a) based on the command,
With the distance between CR1 and the distance between CL2 and CR2 in FIG. 14 (b) as the critical shift amount, the shift amount is changed as shown in FIGS. 20 (a) and 20 (b).

That is, as shown in FIG. 20A, the deviation amount is a critical deviation amount until a predetermined time t1 (for example, 20 minutes) elapses from the start of the display.
When the number exceeds 1, the data creating means 20 gradually changes the deviation amount in a predetermined cycle based on a command from the deviation amount adjusting means 22 to weaken the stereoscopic effect.

By carrying out such a processing, it is possible to prevent an image having a strong stereoscopic effect from being continuously viewed for a long time, so that fatigue of the eyes of the observer can be reduced.

The processing shown in FIG. 20 (b) may be performed. In this example, when the image display is started, the shift amount is set smaller than the critical value to weaken the stereoscopic effect. Then, it approaches the critical value with the passage of time.

That is, although stereoscopic vision is realized by a human fusing the image for the left eye and the image for the right eye, this is not the case at the beginning to see, so that the fusion cannot be performed successfully or It is conceivable that the burden on However, if the processing shown in FIG. 20B is performed, such a problem is solved.

On the other hand, in the present embodiment, in determining the stereoscopic viewing condition, after the entire model is moved on the coordinate system of the stereoscopic system (see steps S109 and 110), the entire model can be easily stereoscopically viewed. When it does not enter the area, the depth distance is reduced by correcting the inter-eye distance (see step S115) to compensate for this. Therefore, stereoscopic viewing can be performed with a sufficient stereoscopic effect as compared with the case where the space in the depth direction is simply reduced.

The problems occurring when the inter-eye distance is corrected preferentially will be described below with reference to FIGS.
This will be described with reference to (c) and FIGS. 16 (a) and 16 (b).

As shown in FIG. 15A, it is assumed that the model elements E1, E2, E3 are arranged in the coordinate system of the stereoscopic system. When the interocular distance is the initial value D0, the left-eye images IL1 to IL3 and the right-eye images IR1 to IR3 corresponding to the elements E1, E2, and E3 are displayed on the display surface 1a. Here, as shown in FIG. 15B, it is assumed that the interocular distance is reduced from the initial value D0 to D. Then, on the display surface 1a, the elements E1, E2,
Left eye images IL1 'to IL3' and right eye images IR1 'to IR3' respectively corresponding to E3 are displayed. According to the method of changing the interocular distance, the images IL1 to IL3 for the left eye and the images IR1 to IR3 for the right eye before changing the interocular distance, and the image IL1 for the left eye after changing the interocular distance. There is an advantage in that the sizes of the'-IL3 'and the right-eye images IR1'-IR3' do not change.

However, the correction of the interocular distance is merely performed in the image processing, and the actual interocular distance of the user who looks at the display surface 1a does not change. Therefore, as shown in FIG. 15C, the user who looks at the display surface 1a sees the elements E1, E2, and E3.
It will be perceived as being at the positions of E1 ', E2', and E3 ', respectively. That is, in this case, the user perceives the widths of the elements E1, E2, and E3 in the depth direction in a significantly compressed state.

The present embodiment solves the above problem by preferentially moving the entire model. That is, in this embodiment, before performing the correction of the interocular distance,
As shown in FIG. 16B, the elements E1, E2, E3 are moved in the depth direction on the coordinate system of the stereoscopic system. In this way, the positional relationship of the elements E1, E2, E3 perceived by the user does not change, as shown in FIGS. 16 (a) and 16 (b). However, when the entire model is moved, the sizes of the left-eye image and the right-eye image displayed on the display surface 1a change. Therefore, in order to correct this change in size, each element forming the model is corrected. The size of each element is corrected in accordance with the moving distance of (see step S111). In the present embodiment, in order to fit the model in the area for easy stereoscopic vision, the interocular distance is corrected only for the portion where movement of the entire model is insufficient, and this is dealt with. Therefore, the reduction in the sense of depth of each element of the model is minimized.

Further, in such a binocular fusion type stereoscopic system, when the model is located at the edge of the screen, when the model is displayed in front of the screen, the model for the right eye with this model is used. It is conceivable that the image is displayed on only one of the image and the image for the left eye. In such a case, the user perceives that the image of the model that should be in front of the screen is hidden in the screen that should be behind this image,
The context is inconsistent. Therefore, it is considered that correct stereoscopic vision becomes impossible.

There are two possible solutions to this problem. First, all display objects (models) are set deeper than the screen, and secondly, the edges of the screen are not clearly noticed. To be However, according to the first method, the image does not pop out from the display surface so that the image becomes less powerful (the stereoscopic effect is difficult to obtain and the commercial value of the system is reduced).
Therefore, in the system of this embodiment, the screen edge processing unit 23 shown in FIG. 1 executes the second method.

The screen edge processing section 23 receives model position data, model size data, and interocular distance data from the model data management means 19. From this data, the screen edge processing unit 23 detects whether the model is located at the screen edge and determines whether to perform the screen edge blurring process. Further, the range in which the blurring process is performed on the edges of the left and right images is determined.

Based on these determinations, the screen edge processing unit 23 darkens the lightness of the model as it approaches the edge of the determined screen edge, or the color tone of the model to the edge. The closer to, the closer to the background color.

This data, together with other data, is sent to the image data creating means 20 through the model data managing means 19.
And processed in step S118.

By such processing, the user feels that the edge of the screen is somehow invisible, and even if there is a contradiction as described above, it is possible to effectively prevent the user from being aware of it.

Second Embodiment Next, a second embodiment will be described. 17 to 19 are views showing a second embodiment of the present invention. Figure 1
As shown in FIG. 7, in the second embodiment, a line-of-sight / gaze range detecting means 30 is provided instead of the gaze range estimating means of the first embodiment, and accordingly, a stereoscopic vision condition calculation flow is performed. The difference is the same as the first embodiment except for the difference. Therefore, in the second embodiment, the same parts as those in the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

As shown in FIG. 17, the stereoscopic system is
A line-of-sight / gaze range detecting means 30 for detecting an actual line-of-sight and a point of gaze with respect to the displayed image is provided. The line-of-sight / gaze range detection means 30 is, for example, a binocular line-of-sight analysis device manufactured by Takei Kikai Kogyo. The line-of-sight / gaze range detection means 30 is connected to the model data management means 19 so as to transmit to the model data management means 19 in real time data on which part of the display surface 1a the user's line of sight is located. It has become. As the line-of-sight / gaze range detecting means 30, another known binocular line-of-sight analysis device having the same function as the binocular line-of-sight analysis device manufactured by Takei Kikai Kogyo may be used.

Next, the operation of this embodiment having such a configuration will be described with reference to FIGS.

As shown in FIG. 18, similar to the first embodiment, the processing operations of steps S100 to S104 are performed.

Next, the model data management means 19 receives the data of the line of sight and the point of gaze of the user from the line-of-sight / gazing point detection position 30 (step S200), and the user looks at the screen from the detected line-of-sight direction. It is determined whether or not there is (step S201). When the user is not looking at the display surface 1a, the center 1b of the display 1a is set as the gazing point position (step S202). On the other hand, when the user is looking at the display surface 1a, the position where the user's line of sight is longest is set as the gazing point position (step S).
203).

In this case, the gazing range in the depth direction can be determined, for example, from the standard deviation of the depth position of the gazing point.

Next, as in the first embodiment, the processing operations of steps S110 to 115 are performed, whereby the model data management means 19 causes the model coordinate system of the stereoscopic system which is optimum for stereoscopic viewing. The shape and size of the model in
Obtain data such as arrangement state and color tone. These data are sent to the image data creating means 20, and the image data creating means 20 creates the image data for the left eye and the image data for the right eye based on this data (step S117).
The display 1 displays an image on the display surface 1a based on the left-eye image data and the right-eye image data (step S118).

Next, the control means 10 inquires whether or not the stereoscopic system will be continuously used (step S1).
03). When it is continuously used, the above operation starting from step S104 is repeated.

As described above, according to this embodiment, the line-of-sight / gaze range detecting means 30 detects the actual line-of-sight and gaze point of the user.
The gaze range can be determined more accurately. Therefore, a more appropriate stereoscopic viewing condition can be calculated.

In the first and second embodiments described above, the model moving means 14 and the depth direction reducing means 16 operate so that the gazing range or the whole model falls within the easy stereoscopic viewing area in FIG. However, the present invention is not limited to this, and the gaze range or the entire model may be put into the stereoscopic viewable region.

Third Embodiment Next, a third embodiment will be described.

FIG. 21, FIG. 22, and FIG. 23 are views showing the third embodiment.

In the first and second embodiments, by moving the model and enlarging or reducing it, the gaze range for this model is located in the stereoscopic viewable region, and the stereoscopic view is facilitated. In the form, the perspective transformation storage means 33 determined by the perspective transformation determination means 32 shown in FIG.
Similar processing is performed based on the perspective conversion method stored in.

The processing in this system will be described below with reference to FIGS. The same processes as those in the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

In the first and second embodiments, the process of changing the position of the model, the enlargement and the change of the inter-eye distance is used to create the image. However, if the viewpoint is changed, a three-dimensional model is obtained. It can be summarized as a perspective transformation method when projecting onto a two-dimensional image.

That is, in the perspective transformation, the inter-eye distance is set to 2
e, the point O (0,0,0) at the center of the viewpoint, and D the distance from the viewpoint to the screen, the point P (xp, y in the three-dimensional space
p, zp) is in the left-eye image: (D · xp / Zp + e · (D / zp−1), D · yp /
zp) In the right-eye image: (D · xp / Zp−e · (D / zp−1), D · yp /
zp) will be projected.

Therefore, using this perspective transformation method,
Even if the three-dimensional model is actually moved and the processing such as enlargement / reduction is not performed as in the first embodiment, the same processing can be performed as follows.

That is, in this system, first, FIG.
In step S301, the gaze range is obtained by the same method as described above, and then, in step S302, space expansion centering on the viewpoint is performed based on the original position of the model and the movement distance for moving the model within the gaze range. The rate m (perspective transformation method) is determined.

When the entire space in which the model is located is expanded or contracted at this spatial expansion ratio m, the size of the model is multiplied by m, and the point P in this model moves to the following point P '.

Point P '(m.xp, m.yp, m.zp) Next, in step S303, it is determined whether or not each point in the model after perspective transformation is within the easy stereoscopic region. That is, the point N ′ after the movement of the representative point N closest to the viewpoint and the point M ′ after the movement of the representative point M farthest from the viewpoint are calculated, and it is determined whether or not they fall within the stereoscopic view easy region. .

If it does not enter, in step S304, the inter-eye distance is adjusted based on the same idea as shown in FIG. 11, and correction is performed so that all points are located within the easy stereoscopic viewing area. . That is, assuming that the reduction rate of the inter-eye distance is t, the point P in the three-dimensional space is eventually: (D · xp / Zp + et · (D / (m · zp) −
1), D · yp / zp) In the right-eye image: (D · xp / Zp−e · t · (D / (m · zp) −
1), D · yp / zp).

Therefore, the perspective conversion method is defined by the model enlargement ratio m based on the center of the viewpoint, the original eye distance 2e, the distance D from the viewpoint to the screen, and the eye distance reduction ratio t obtained from FIG. It is possible (S305).

In this embodiment, these data (perspective conversion method) are stored in the perspective conversion storage means 33, and are used by the image data creating means 20 when creating image data (S1).
16-S118). This makes it possible to create image data that facilitates stereoscopic viewing, as in the first and second embodiments.

In this example, the perspective conversion method is stored in the perspective conversion storage means 33.
The perspective conversion method for each scene obtained in advance by the system of the present embodiment may be stored in a storage medium such as D together with the image data.

That is, if the image data before processing (actual image) is stored in the storage medium, the position of the model from the viewpoint, the model enlargement ratio m, the interocular distance reduction ratio t, etc., are stored. At the time of reproduction, the image data creating means 20 can create an image that is easy to stereoscopically view without going through the stereoscopic view easy condition calculating unit.

Fourth Embodiment Next, a fourth embodiment of the present invention will be described.

24, 25 and 26 are diagrams showing the fourth embodiment. In these figures, the same components as those in the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

In the first to third embodiments, the shape,
The system according to the fourth embodiment is a system for easily displaying a three-dimensional model given by information such as size and arrangement position in a stereoscopic manner. The present invention relates to a system for easily displaying a stereoscopic image.

In such a case, whether stereoscopic viewing is easy or not is determined by the positional relationship of the gaze range in the left and right images. Therefore, in the system of this embodiment, as shown in FIG. 24, the image conversion determining unit 34 that calculates / converts the conversion method for converting the image data into the image data that facilitates stereoscopic viewing, and the determined image conversion. The image calculation means 35 for obtaining the converted image data based on the determination method.

Hereinafter, the processing in this system will be described with reference to FIG.
5, 26 will be described. As mentioned above,
The same steps as those in the first embodiment are designated by the same reference numerals, and the description thereof will be omitted.

This system is provided with two parallax images such as an image taken by the right-eye camera and an image taken by the left-eye camera. This data is sent to the gazing range determining means 13, and the gazing range determining means 13 first determines the gazing range using either image (step S4).
01).

To determine the gazing range, for example, the standard deviation σ of the barycentric position of the pixel weighted by the color tone and the distance from the barycentric position is obtained, and the gazing range is within a radius 2σ from the barycenter. That is, the coordinates of each pixel are (xi, y
i), the position of the center of gravity (xG, yG) and the standard deviation σ
Is expressed by the following equations 7 and 8.

[0246]

[Equation 7]

[0247]

[Equation 8]

As a method different from this, for example, a corresponding image area between both images is detected, three-dimensional coordinates are obtained from the correspondence between the image areas, and the model of the first to third embodiments is calculated. Similarly to the case, the gaze range in the depth direction may be obtained.

In order to find the corresponding part of the image area, it is general to emphasize the edges in both images and find the one with a high correlation (for example, refer to Yamaguchi et al., "Reliable Correspondence"). Prioritized multi-stage stereo method ", IEICE Transactions D-II, Vol. J74-D-II, No. 7, etc.).

Next, in step S402, the corresponding points of the portions other than the background within the determined gaze range that have the largest deviation between the left and right images are searched under the following two conditions.

That is, first, when the coordinate origin is on the left side of the screen, the horizontal (X direction) coordinate of one point (corresponding point) in the left eye image is Xl, and the right eye image is the coordinate. The coordinates are Xr. Then, as a first condition, a point S that satisfies Xl> Xr is searched for.

At this point S, the coordinates in the left-eye image are larger than the coordinates in the right-eye image. That is, this point S is a point recognized on the front side of the screen like E1 shown in FIG. Then, the point S is the point having the largest displacement amount between the left and right images among such points, and is thus the most recognized point in the gaze range.

Next, as a second condition, a point T satisfying Xr> Xl is searched for. This point T is a point that is recognized deepest in the gazing range, as indicated by E2 in FIG. Since this point T is the point having the largest displacement amount between the left and right images among such points, it is the most recognized point in the gazing range.

Here, the position of the point S in the image for the left eye is (Xpl, Yp), and the position in the image on the right side is (Xp, Yp).
r, Yp), and the shift amount Xpl-X between the left and right images at this point
Let pr be dXp. Further, the position of the point T in the image on the left side is (Xql, Yq), and the position in the image on the right side is (Xql, Yq).
r, Yq), and the amount of deviation Xqr-X between the left and right images at this point
Let ql be dXq.

As an example, consider the case where dXp> dXq, that is, the amount of deviation of the point S is larger than the amount of deviation of the point T (the same applies when dXq> dXp). Then, it is determined whether or not this point S is within the easy stereoscopic view range. In this embodiment, this determination is made using the viewing angle in step S403.

That is, in this case, since the maximum shift of the image within the gazing range is dXp corresponding to the shift of the point S, it is determined whether or not this is a shift of 2 degrees or more in visual angle. As described above, the difference between the left and right images is 2
This is because the stereoscopic view is lost when the number of rotations exceeds the limit.

In step S403, if the shift amount converted into the visual angle is within 2 degrees, it is not necessary to change the image data. Therefore, the amount of deviation of the viewing angle at that time is set as a “set value”. However, if it becomes twice or more, the set value is set exactly twice in step S404.

On the other hand, in this case, it is also necessary to consider the amount of shift change. That is, as described above, stereoscopic viewing becomes difficult when the amount of change in displacement becomes 1 degree or more in terms of the convergence angle.

Therefore, in step S405,
When the amount of change in deviation is 1 degree or more, the set value is determined so that the change is within 1 degree. That is, in this case, the set value of the viewing angle is not exactly twice, but a value that suppresses the change once (step S406).

The viewing angle (set value) is determined by the distance D from the viewpoint to the display surface.

That is, for example, when the viewing angle is 2 degrees, the approximate distance (L) on the screen is: L = 2 × D × tan (π / 180) = π × D / 90 = 0.035 × D Become.

Next, in step S407, the spatial expansion rate m is determined using this set value. Here, the spatial expansion rate m is defined as follows when the value corresponding to the inter-eye distance is 2e and the viewing angle setting value is 2 degrees.

M = (dxp + 2 × e) / (L + 2 × e) When the image is enlarged or reduced at this spatial enlargement ratio, the entire gazing range may not fall within the easy stereoscopic view range. In this case, in step S408, the horizontal reduction ratio is calculated using the shift amount dXq of the point S.
Find t.

That is, the horizontal reduction ratio t is (d
xq + 2 × e) /m−2×e≦0.035D, t = 1.0 (dxq + 2 × e) / m−2 × e> 0.035D, t = 0.035 × m × D / (dxq + 2 × e-2 × e ×
m).

After the space enlargement ratio and the horizontal reduction ratio t have been obtained in this way, the image calculation method is determined in step S409.

That is, according to the image calculation method of this embodiment, an arbitrary point A (xal,
y) and (xar, y) are converted into the following positions, respectively.

Left image: (Xal + Xar) / 2 + e × t
X ((Xal-Xar + 2xe) / (2xmxe)-
1) Right image: (Xal + Xar) / 2−e × t × ((Xal
-Xar + 2 * e) / (2 * m * e) -1) The image calculation means 35 converts the image data for each scene, and the image data creation means 20 creates a stereoscopic image based on this image conversion. To generate. Therefore, according to such a system, it is possible to generate image data that is easy to stereoscopically view, even for a real image, as in the first to third embodiments.

In this example, the image calculation means 35 performs image conversion for each scene, but the image data converted by this method in advance or the calculated stereoscopic condition and the original image data are stored in a DVD or the like. If it is stored in the medium, it is possible to create an image that is easily stereoscopically viewed by the image data creating means 20 without having to go through the stereoscopically easy condition calculation unit during reproduction.

The system of this embodiment is effective when it is desired to generate an easy-to-see stereoscopic image based on visual images appropriately taken by two cameras from different viewpoints.

This system can also be applied to a system for stereoscopic viewing by making a time difference from a moving image. Further, the present invention can be applied to a binocular stereoscopic system other than the time division type stereoscopic system.

[0271]

As described above, according to the present invention,
A model or a photographed image arranged in a three-dimensional space can be displayed on the display under optimal stereoscopic viewing conditions according to changes in the scene. For this reason, it is possible to reduce the discomfort and the eye fatigue of the stereoscopic image that the user feels.

[Brief description of drawings]

FIG. 1 is a diagram showing a first embodiment of the present invention,
The figure which shows the outline of a stereoscopic vision system.

FIG. 2 is a diagram showing a first embodiment of the present invention,
The figure which shows the structure of a stereoscopic vision system.

FIG. 3 is a diagram showing a coordinate system of a stereoscopic system.

FIG. 4 is a flowchart showing a stereoscopic viewing condition calculation process according to the first embodiment.

FIG. 5 is a flowchart showing a stereoscopic viewing condition calculation process according to the first embodiment.

FIG. 6 is a diagram showing an operation of coordinate system conversion means.

FIG. 7 is a diagram showing a flow of processing operations of a stereoscopic condition calculating unit.

FIG. 8 is a diagram showing a method of estimating a gazing point.

FIG. 9 is a diagram showing a method of estimating a gazing point.

FIG. 10 is a graph showing a relationship between the distance from the viewpoint center of the display surface and a stereoscopic view easy (possible) area.

FIG. 11 is a graph showing a method of correcting an inter-eye distance.

FIG. 12 is a diagram showing a change in the amount of image shift on the display surface when the interocular distance is corrected.

13 is a diagram showing a method of creating the graph of FIG.

FIG. 14 is a diagram showing an operation of image data creating means.

FIG. 15 is a diagram showing an operation when inter-eye distance correction is performed.

FIG. 16 is a diagram showing an operation when the entire model is moved.

FIG. 17 is a diagram showing a second embodiment of the present invention, which is a diagram showing a configuration of a stereoscopic system.

FIG. 18 is a flowchart showing a stereoscopic viewing condition calculation process according to the second embodiment.

FIG. 19 is a flowchart showing a stereoscopic viewing condition calculation process according to the second embodiment.

FIG. 20 is a graph showing a process of relaxing the easy stereoscopic viewing condition according to the passage of time in the first and second embodiments.

FIG. 21 is a diagram showing the third embodiment of the present invention and is a diagram showing a configuration of a stereoscopic system.

FIG. 22 is a flowchart showing a stereoscopic viewing condition calculation process according to the third embodiment.

FIG. 23 is a flowchart showing a stereoscopic viewing condition calculation process according to the third embodiment.

FIG. 24 is a diagram showing the fourth embodiment of the present invention and is a diagram showing the configuration of a stereoscopic system.

FIG. 25 is a flowchart showing a stereoscopic viewing condition calculation process according to the fourth embodiment.

FIG. 26 is a flowchart showing a stereoscopic viewing condition calculation process according to the fourth embodiment.

[Explanation of symbols]

1 ... Display 12 ... Out-of-field data excluding means 13 ... Gaze range determining means 14 ... Model moving means 15. Model scaling means 16 ... Interocular distance setting means 17 ... Image data creating means 30 ... Viewpoint / gazing point detection means 32 ... Perspective transformation determination means 34 ... Image conversion determining means

Continuation of the front page (56) Reference JP-A-7-230556 (JP, A) JP-A-2-280280 (JP, A) JP-A-7-44735 (JP, A) JP-A-7-167633 (JP , A) JP 7-239952 (JP, A) JP 6-337756 (JP, A) JP 8-19003 (JP, A) JP 6-8830 (JP, A) JP 8-106548 (JP, A) JP-A-7-182535 (JP, A) Hiroaki Kubota 3 others, "Concept and development of prototype model of vision processor aiming at recognition and discrimination of moving body", electronic information communication Technical Report of the Society PRU89-101 to 112 Pattern recognition and understanding, The Institute of Electronics, Information and Communication Engineers, January 26, 1990, Vol. 89, No. 390, p. 49-56 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T 17/40 G06F 3/00 G06F 3/14 G09G 5/36 H04N 13/04 CSDB (Japan Patent Office)

Claims (43)

(57) [Claims] 1. A stereoscopic system for stereoscopically displaying a model on a display by using a pair of right-eye and left-eye images viewed from two different viewpoints. By changing the data of the gaze range determining means and the model for determining the range that the observer is likely to gaze, so that the gaze range determined by the gaze range determining means falls within the easy stereoscopic region. By changing the model moving means for moving the model in the direction orthogonal to the display surface and the data of the model changed by the model moving means, the size of the model moved by the model moving means is set to the movement distance. A model scaling unit that scales correspondingly; and a model scaling unit that is moved by the model moving unit and scaled by the model scaling unit. An image data creating unit that creates image data to be displayed on the display based on the model data after the change of the stereoscopic image. 2. The stereoscopic system according to claim 1, wherein the scale in the depth direction as seen from the viewpoint is such that at least a main part of the model moved by the model moving means is within the easy stereoscopic view area. A stereoscopic viewing system further comprising depth depth reducing means for setting. 3. The stereoscopic system according to claim 1, further comprising a line-of-sight / gaze-range detecting unit that detects a line-of-sight and a gaze range of an actually displayed image, and the model moving unit includes the line-of-sight / gaze range. A stereoscopic vision system, wherein the model is moved in a direction orthogonal to the display based on the gaze range detected by the detection means. 4. The stereoscopic vision system according to claim 1, wherein the model moving means changes a placement condition of the model viewed from a viewpoint, and changes the placement condition when the model is moved according to the change. A stereoscopic system comprising a means for preventing the movement amount of the gaze range of the model from converting to a convergence angle of about 1 degree or more within a predetermined time from the time of occurrence. 5. A stereoscopic vision system in which a model is displayed on a display so as to be stereoscopically visible by using a pair of images viewed from two different viewpoints, and a portion where an observer is likely to gaze at the model is selected. Gaze range determining means for determining, and perspective transformation determining means for determining a perspective transformation method on the pair of left and right eye images of the model data based on the output of the gaze range determining means; An image data creating means for creating image data displayed on the display based on the perspective conversion method determined by the determining means. 6. The stereoscopic vision system according to claim 1, wherein the gaze range determining unit determines an arrangement condition, a color tone, a lightness, a saturation, a size, a complexity of the model viewed from the observer's viewpoint.
A stereoscopic vision system comprising means for estimating the gaze range of both eyes in consideration of all or part of the moving speed. 7. The stereoscopic system according to claim 1, further comprising an out-of-field data excluding means for excluding data of a model existing outside the field of view. 8. The stereoscopic system according to claim 7, further comprising a line-of-sight / gaze-range detecting unit that detects a line-of-sight and a gaze range with respect to an actually displayed image, and the out-of-view-data excluding unit includes the line-of-sight / gaze range. A stereoscopic vision system characterized in that a visual field is calculated based on the line of sight detected by the gaze range detecting means, and data existing outside the visual field is excluded. 9. The stereoscopic system according to claim 1, further comprising color tone changing means for bringing the color tone of the model to be displayed closer to the color tone of the background color as it goes away from the viewpoint in the depth direction. Vision system. 10. The stereoscopic vision system according to claim 1, wherein the image data creating means further has a function of defining a stereoscopic vision condition of a graphic user interface item displayed on a display. And stereoscopic system. 11. The stereoscopic system according to claim 1, further comprising a blurring unit that blurs edges of left and right images when the model is located at an edge of a display screen. 12. The stereoscopic vision system according to claim 1, further comprising means for reducing eye fatigue by changing stereoscopic vision conditions to weaken the stereoscopic vision effect. 13. A gaze range deciding means for deciding an image part which is highly likely to be gazed by an observer in a stereoscopic system for stereoscopically displaying a pair of images viewed from two different viewpoints on a display. An image conversion deciding means for deciding an image conversion method such that the gaze range becomes an area where stereoscopic viewing is easy based on the output of the gaze range deciding means; An image data creating means for creating image data to be displayed on the stereoscopic viewing system. 14. A stereoscopic viewing method for displaying a pair of images of a model viewed from two different viewpoints on a display in a stereoscopic manner so as to determine a range in which an observer is likely to gaze at the model. Range determination step, and model change by moving the model in a direction orthogonal to the display surface so that the gaze range determined by the gaze range determining means falls within the stereoscopic easy region by changing the model data. And a model scaling step of scaling the size of the model moved by the model moving means according to a moving distance by changing the data of the model changed by the model moving means, The model after being moved by the moving means and changed in size by the model scaling means. Based on the data, stereoscopic method characterized by and an image data generation step of generating image data to be displayed on said display. 15. The stereoscopic viewing method according to claim 14, wherein the scale of the depth distance seen from the viewpoint is adjusted so that at least a main part of the model moved in the model moving step falls within the easy stereoscopic viewing area. A stereoscopic viewing method further comprising a depth distance reduction step of setting. 16. The stereoscopic viewing method according to claim 14, further comprising a line-of-sight / gaze-range detecting step of detecting a line-of-sight and a gaze range with respect to an actually displayed image, wherein the model moving step includes the line-of-sight / gaze range. A stereoscopic viewing method, wherein the model is moved in a direction orthogonal to the display based on the gaze range detected by the detecting step. 17. The stereoscopic viewing method according to claim 14, wherein in the model moving step, a placement condition of the model in a three-dimensional space viewed from a viewpoint changes, and the model is moved in accordance with the change. A stereoscopic method comprising a step of preventing the movement amount of the gaze range of the model from being converted into a convergence angle and becoming substantially 1 degree or more within a predetermined time from the time when the change of the arrangement condition occurs. . 18. A stereoscopic viewing method for displaying a model on a display so that the model can be viewed stereoscopically by using a pair of images viewed from two different viewpoints. Gaze range determining step to determine, based on the output of the gaze range determining step, the pair of left of the model data,
A perspective transformation determining step of determining a perspective transformation method on the image for the right eye, and an image data creating step of creating image data displayed on the display based on the perspective transformation method determined by the perspective transformation determining step. And a stereoscopic viewing method. 19. The stereoscopic viewing method according to claim 14, wherein the gazing range determining step includes placement conditions, color tone, brightness, saturation, size, complexity of the model viewed from the observer's viewpoint. A stereoscopic viewing method comprising the step of estimating the gaze range of both eyes in consideration of all or part of the moving speed. 20. The stereoscopic viewing method according to claim 14, further comprising an out-of-field-of-view data excluding step of excluding data of a model existing outside the field of view. 21. The stereoscopic viewing method according to claim 20, further comprising a line-of-sight / gaze-range detecting step of detecting a line-of-sight and a gaze range of an actually displayed image, and the step of excluding data outside the visual field includes the line-of-sight / gaze range. A stereoscopic viewing method, wherein a visual field is calculated based on the line of sight detected by the gaze range detecting step, and data existing outside the visual field is excluded. 22. The stereoscopic vision method according to claim 14, further comprising a color tone changing step of bringing the color tone of the displayed model closer to the color tone of the background color as the distance from the viewpoint increases in the depth direction. How to see. 23. The stereoscopic viewing method according to claim 14, wherein the image data creating step includes a step of defining a stereoscopic viewing condition of a graphic user interface item displayed on a display. Method. 24. The stereoscopic viewing method according to claim 14, further comprising a step of blurring edges of the left and right images when the model is located at an edge of a display screen. Stereoscopic method to do. 25. The stereoscopic vision method according to claim 14, further comprising a step of reducing eye strain by changing a stereoscopic vision condition to weaken the stereoscopic vision effect. And stereoscopic method. 26. In a stereoscopic viewing method for displaying a pair of images viewed from two different viewpoints on a display in a stereoscopically viewable manner, a gaze range deciding step for deciding an image portion that is highly likely to be gazed by an observer. An image conversion determining step that determines an image conversion method such that the gaze range becomes an area where stereoscopic viewing is easy based on the output of the gaze range determining step, and the display on the display based on the output of the image conversion determining step An image data creating step of creating image data to be displayed on the stereoscopic viewing method. 27. A storage medium storing a computer program for causing a computer system to stereoscopically display a pair of images of a model viewed from two different viewpoints on a display, wherein an observer is included in the computer system. By changing the data of the model to the gaze range determination command means that gives a command to determine the range with a high possibility of gaze, the gaze range determined by the gaze range determination command means can be easily stereoscopically viewed. Model movement command means for giving a command to move the model in a direction orthogonal to the display surface so as to enter the area, and the model data by changing the data of the model changed by the model movement means to the computer system. Transfer the size of the model moved by the moving means. Model scaling instruction means for giving a command to change in accordance with the moving distance, and to the computer system based on the model data after being moved by the model moving means and changed in size by the model scaling means And an image data creation command means for giving a command to create the image data to be displayed on the display. 28. The storage medium according to claim 27, wherein in the computer system, at least a main part of the model moved based on the instruction of the model movement instruction means is included in the stereoscopic view easy region. A storage medium further comprising depth distance reduction command means for giving a command to set a scale in the depth direction viewed from the viewpoint. 29. The storage medium according to claim 27, further comprising line-of-sight / gaze-range detection command means for giving a command to the computer system to detect a line-of-sight and a gaze range for an actually displayed image, and the model movement command. The means gives the computer system a command to move the model in a direction orthogonal to the display based on the gaze range detected based on the command of the gaze / gaze range detection command means. . 30. The storage medium according to claim 27, wherein the model movement command means causes the computer system to display a model 3 from a viewpoint.
Arrangement condition in the dimensional space changes, when moving the model according to this change, within a predetermined time from the time when the change in the arrangement condition occurs, the movement amount of the gaze range of the model is converted to the convergence angle. A storage medium having means for giving a command to prevent the temperature from becoming almost once or more. 31. A storage medium storing a computer program for causing a computer system to stereoscopically display a model on a display by using a pair of images viewed from two different viewpoints. Against the gaze range determination command means for giving a command to determine the part that the observer is likely to gaze, the computer system, based on the output based on the gaze range determination command, the pair of left of the model data, A perspective transformation determination command means for giving a command to determine a perspective transformation method on the image for the right eye, and a computer system, which is displayed on the display based on the perspective transformation method determined based on the perspective transformation determination command. Image data creation command means for giving a command to create image data Storage medium. 32. The storage medium according to claim 27, wherein the gaze range determination command means instructs the computer system to arrange conditions, color tone, lightness, saturation, and size of the model viewed from the observer's viewpoint. , A storage medium having means for giving a command for estimating a gaze range of both eyes in consideration of all or part of complexity and moving speed. 33. The storage medium according to claim 27, further comprising an out-of-view data exclusion command means for giving an instruction to exclude data of a model existing outside the field of view to the computer system. . 34. The storage medium according to claim 27, further comprising line-of-sight / gaze range detection command means for giving the computer system a command to detect a line-of-sight and a range of gaze with respect to an actually displayed image, The out-of-field-of-view data exclusion command means gives the computer system a command of calculating a field of view based on the line of sight detected based on the command of the line-of-sight / gaze range detection command means, and excluding data existing outside the field of view. A storage medium characterized by the above. 35. The storage medium according to claim 27, further comprising a color tone change command means for giving the computer system a command to bring the color tone of the model to be displayed closer to the color tone of the background color as the distance from the viewpoint is increased in the depth direction. A storage medium further comprising: 36. The storage medium according to claim 27, wherein said image data creation commanding means is means for giving said computer system a command for determining a stereoscopic viewing condition of a graphic user interface item displayed on a display. A storage medium characterized by having. 37. The storage medium according to claim 27, wherein the computer system has a commanding unit for blurring the edges of the left and right images when the model is located at the edge of the display screen. Storage medium. 38. The storage medium according to claim 27, further comprising a command means for causing the computer system to perform a process of reducing eye fatigue by changing a stereoscopic condition and weakening a stereoscopic effect. A storage medium characterized by the above. 39. A storage medium storing a computer program for causing a computer system to stereoscopically display a pair of images viewed from two different viewpoints on a display, wherein an observer gazes at the computer system. Based on the output based on the gaze range decision command, the gaze range decision command means for giving a command for deciding a highly probable image portion, and image conversion such that the gaze range becomes an area where stereoscopic vision is easy Image conversion determination command means for giving a command for determining the method, and image data creation command means for giving a command to the computer system for creating the image data displayed on the display based on the output of the image conversion determination means. A storage medium provided with. 40. A storage medium storing image data, wherein the left-eye image and the right-eye image are displayed on a display, and these images are independently viewed by the left eye and the right eye,
In a storage medium applied to a system for performing stereoscopic vision, the storage medium stores, for each scene, data for image conversion for locating a gaze range in the image in an easy stereoscopic view region. A storage medium having a part. 41. A storage medium manufacturing system for manufacturing the storage medium according to claim 40, comprising: a gaze range deciding means for deciding an image portion of the image data that is likely to be gazed by an observer; An image conversion determination unit that determines an image conversion method such that the gaze range becomes an area that facilitates stereoscopic viewing based on the output of the range determination unit, and data for this image conversion in the storage medium together with the image data. A storage medium manufacturing apparatus, comprising: the described means. 42. Using the storage medium according to claim 40,
In a stereoscopic system that generates image data to be displayed on a display, image data stored in a storage medium is processed based on the data for image data conversion stored in this storage medium, and is displayed on the display. A stereoscopic viewing system comprising means for generating a pair of left-eye and right-eye images for 43. The stereoscopic vision system according to claim 1, wherein the gaze range deciding unit determines a portion, which is highly likely to be gazed by an observer with respect to the model, based on initial values of the model and the observer. A stereoscopic vision system characterized by making decisions.