JP2018186319A - Stereoscopic image display control device, stereoscopic image display control method and stereoscopic image display control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a stereoscopic image display control device capable of allowing a viewer to perceive stereoscopic image without wearing any glasses with a simple device configuration.SOLUTION: A stereoscopic image display control device 1 comprises: a motion update part 13 configured to display a piece of content on a display part of a content display device 3; and a shielded object parameter calculation part 12 configured to form a semi-shielding state in which a display object image included in the content, which can be seen by one eye but cannot be seen by the other eye of a viewer on the display part of the shield object display device 2 disposed between the content display device 3 and the viewer of the content.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for making an observer perceive a stereoscopic image.

  In the glasses-equipped method, which is the basis of the stereoscopic display method, a parallax image is displayed on the same screen, separated into a right-eye image and a left-eye image by an optical filter provided in the glasses, and transmitted to the right eye and the left eye, respectively. Thus, a method of transmitting depth information based on binocular parallax to an observer is employed.

  With the glasses-equipped method, even if the observer moves freely in the space, the optical filter with the image separation function also moves, so stereoscopic images can be viewed at any position and location, and multiple people You can also enjoy stereoscopic images. However, wearing eyeglasses puts a burden on the observer and does not always wear eyeglasses, so the usage scenes that allow stereoscopic viewing are limited. Therefore, there is a demand for a method without glasses that does not require glasses.

  In the glasses-equipped method, since there is an optical filter in front of the observer's eyes, the parallax image from the screen can be separated from the observer's viewpoint. On the other hand, in the glassesless method, the optical filter cannot be placed in front of the observer's eye, but instead, an optical filter is provided in front of the screen of the display device to separate the parallax images at a position away from the observer. Then, a method of delivering the right eye image and the left eye image from the display device side to the observer's viewpoint set in the space is employed.

Japanese Patent Laying-Open No. 2006-161912

  However, in the case of the conventional glassesless method, the parallax image is separated at a position away from the observer, so that the right eye image and the left eye image can be delivered to the observer's right eye and left eye respectively from the screen. It is necessary to give directivity to the rays of the emitted parallax image, and the display cost is extremely high both in terms of hardware and software. For example, in the case of the “autostereoscopic image display method having light beam directivity and smooth motion parallax” described in Patent Document 1, a special screen and a plurality of projectors must be used.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to make a viewer perceive a three-dimensional image with a simple device configuration without wearing glasses.

  In order to solve the above problems, a stereoscopic image display control device according to claim 1 includes a content display control unit that displays content on a display unit of a first display device, the first display device, and an observer of the content. A display target image included in the content is formed on the display unit of the second display device installed between the viewers in a semi-shielded state that is visible to one eye of the observer and not visible to the other eye. A stereoscopic image formation control unit.

  A stereoscopic image display control device according to a second aspect is the stereoscopic image display control device according to the first aspect, wherein the second display device is a transmissive display device capable of changing a transmittance or a transmissive region of a display unit. It is characterized by that.

  The stereoscopic image display control device according to claim 3 is the stereoscopic image display control device according to claim 1 or 2, wherein the stereoscopic image formation control unit uses the object image related to the content to display the second display device. A shielding region for shielding the display target image is formed on the display unit.

  A stereoscopic image display control device according to a fourth aspect is the stereoscopic image display control device according to any one of the first to third aspects, wherein the second display device is a transmissive display device including a transmissive display unit. Then, the stereoscopic image formation control unit performs the transmission to shield the display target image so that the display target image is visible to one eye of the observer and not visible to the other eye. The position, size, or shape of the object image displayed on the mold display device is changed.

  The stereoscopic image display control method according to claim 5 is a stereoscopic image display control method performed by a stereoscopic image display control device, wherein the stereoscopic image display control device displays content on a display unit of a first display device; On the display unit of the second display device installed between the first display device and the viewer of the content, the display target image included in the content is visible to one eye of the viewer, and the other piece And a step of forming a semi-shielding state invisible to the eyes.

  A stereoscopic image display control program according to a sixth aspect causes a computer to function as the stereoscopic image display control apparatus according to any one of the first to fourth aspects.

  According to the present invention, a stereoscopic image can be perceived with a simple apparatus configuration without causing an observer to wear glasses.

It is a figure which shows the example of the appearance of the display target by this invention. It is a figure which shows the whole structure of a three-dimensional image display control system. It is a figure which shows the example of real space. It is a figure which shows the example of real space. It is a figure which shows the operation | movement flow of a stereo image display control apparatus. It is a top view which shows the positional relationship of each object in real space. It is a top view which shows the positional relationship of each object in the virtual space corresponding to the real space of FIG. It is a top view which shows the positional relationship (when a flat screen is used as a shielding object display device) of each object in real space.

<Outline of the invention>
In the case of the conventional glassesless system, a special screen and a plurality of projectors are used to give a stereoscopic effect to the display target in the content, so that the display cost is extremely high. The cause of this is a method of “simultaneously generating independent images (right-eye image and left-eye image) corresponding to each eye of the observer and giving these images directivity”.

  On the other hand, the present invention employs a method of “not using the binocular parallax of the observer for the display target in the content, but determining the depth of the display target only with a monocular depth cue”. Since binocular parallax is not used, the content is not separated into a right-eye image and a left-eye image, and the same image region in the content is not shown by the right eye and the left eye.

  Specifically, a shielding object display device is installed near an intermediate point between a content display device that displays content and an observer who observes the content. Then, the display target included in the content being displayed / reproduced is not shielded by one eye of the observer and is shielded by the other eye in the shielding object display device. That is, by using a monocular viewpoint image that is not occluded or occluded, a stereoscopic effect is given to the display target with only monocular depth cues.

  Accordingly, the observer can perceive a stereoscopic image with a simple apparatus configuration including two general-purpose display devices without causing the observer to wear glasses.

<Proposed technology of invention>
In the present invention, the following technique is proposed.

(First proposed technology)
The first is that “a shielding object is provided between the content display device and the viewer so that the viewer does not perform binocular parallax processing on the content, and the stereoscopic effect of the content given to the viewer by the monocular viewpoint image is enhanced. "Technology" is proposed.

  In a normal human environment, there is a natural semi-shielded state where the object is visible only to one eye but not to the other eye. For example, as shown in FIG. 1, when a shielding object is present in front of an object in the content, the object cannot be seen with one eye of the observer, and the object can be seen with the other eye. By using this semi-shielded state, it is possible to realize autostereoscopic display realized by separating a parallax image into a right-eye image and a left-eye image using a monocular viewpoint image.

  Conventionally, image separation is performed at the observer's viewpoint position or image display position. However, in the present invention, in order to form the above-described semi-shielded state, a shielded object is displayed near the intermediate point between the content display device and the viewer. By placing a shielding object display device having local selectivity of the transmitted transparency and increasing the monocular viewing area of the viewer for the content, a stereoscopic effect is given. The occluding object itself is also used as part of the content.

  On the other hand, this method can also be interpreted as a method in which an optical filter having an image separation function is installed at an intermediate point between the observer's viewpoint and the display device. However, when an opaque optical filter is provided at the intermediate point, another problem arises in that the optical filter itself is perceived by the observer.

  Therefore, in the present invention, if the optical filter itself becomes conscious by changing the position of the optical filter, the optical filter itself is taken as a part of content for creating a semi-shielded state.

  Specifically, the contents and behavior are natural in relation to the contents so that the occluding object does not give the viewer a sense of incongruity. For example, the occlusion object may be selected from the content. The effect is particularly high when the shielding object display device has local selectivity of transparency.

  By this method, for example, a flat panel display whose diffusion resolution is about to reach 4K can be used as the display device of the present invention without reducing the resolution and brightness.

(Second proposed technology)
The second is “so that the occlusion area where the occlusion object is visible to one eye of the observer is as large as possible, and the occlusion area (visible area) where the object in the content is visible to the other eye is as large as possible. We propose a technology that controls the position, size, and shape of occluded objects and transparent areas.

  Specifically, for example, when a transmissive display is used as the occluding object display device, the position, size, and shape of the occluding object are necessary and sufficient to hide the display target in the content from the eye on the occluding side. The eye is adjusted so that it does not get in the way of seeing the display object.

  Accordingly, the display target cannot be observed with the shielded eye, and the visible region of the display target is maximized with the non-shielded eye, so that the observer's unshielded feeling can be reduced.

<Configuration example of stereoscopic image display control system>
Next, the configuration of the stereoscopic image display control system according to the present embodiment will be described. As illustrated in FIG. 2, the stereoscopic image display control system 100 includes a stereoscopic image display control device 1, a shielding object display device 2, a content display device 3, a binocular position acquisition device 4, a shielding object information storage unit 5, and content information. A storage unit 6 is provided.

  The content display device 3 (first display device) is a display device that displays the content output from the stereoscopic image display control device 1 on a display unit (display screen). For example, the content display device 3 displays image content on the screen and reproduces and displays video content on the screen. The content display device 3 can be realized using, for example, a liquid crystal display.

  The shielding object display device 2 (second display device) is installed near an intermediate point between the content display device 3 and the observer, and the display target image included in the content is visible to one eye of the observer, and the other It is a display device that forms and displays a semi-shielded state invisible to one eye on a display unit (display screen). For example, the shielding object display device 2 forms this semi-shielding state using the shielding object image and the non-shielding object image output from the stereoscopic image display control device 1. For example, the shielding object display device 2 can be realized by using a display device that can adjust the transparency of the display unit and can change and select the position, size, and shape of the transmission region of the display unit. .

  The binocular position acquisition device 4 is a device that is installed in the vicinity of the content display device 3 and photographs the eyes of the observer. The binocular position acquisition device 4 can be realized by a general-purpose Web camera, for example.

  Next, the stereoscopic image display control device 1 will be described. The stereoscopic image display control apparatus 1 outputs predetermined content to the content display device 3 and uses the content as a real space to be processed (a real space in which the shielding object display device 2 and the content display device 3 are installed). A virtual space corresponding to is formed in the device itself (inside the computer), and the virtual space is observed from the position / distance relationship between the position of the content display device 3 and the positions of the left and right eyes of the observer. A shielding object image that shields one eye of the person and a non-shielding object image that does not shield the other eye are generated and output to the shielding object display device 2.

  Specifically, the stereoscopic image display control device 1 includes a virtual camera parameter update unit 11, a shielding object parameter calculation unit 12, and a motion update unit 13. The stereoscopic image display control device 1 is communicably connected to the occlusion object display device 2, the content display device 3, the binocular position acquisition device 4, the occlusion object information storage unit 5, and the content information storage unit 6.

  The virtual camera parameter update unit 11 calculates the positions (three-dimensional spatial coordinates) of the left and right eyes of the observer imaged by the binocular position acquisition device 4, and applies the left and right eyes of the observer based on the position information. It is a functional unit that updates the position of each virtual camera in the corresponding virtual space.

  The occlusion object parameter calculation unit 12 (stereoscopic image formation control unit) displays the display target image included in the content on the display unit of the occlusion object display device 2 with one eye of the observer and the other eye with the other eye. It has a function to form an invisible semi-shielded state.

  Specifically, for example, the semi-shielded state is formed by using the position coordinates of each virtual camera corresponding to the left and right eyes of the observer set in the virtual space and the position coordinates of the display target image in the content. The position coordinates of the shielding object image and the non-shielding object image to be calculated are calculated, and the shielding object image and the non-shielding object image acquired from the shielding object information storage unit 5 are displayed at the position corresponding to the calculated position coordinates. On the display. When the shielding object display device 2 is a transmissive display, the shielding area and the non-shielding area can be formed simultaneously using only the shielding object image.

  The occlusion object parameter calculation unit 12 has a function of generating an occlusion object image using an object image related to the content being displayed / reproduced. As for the method for generating the occlusion object image, a related object may be acquired from the content data being displayed / reproduced, or the file name / title of the content may be used as a search key. You may get it.

  Further, the occlusion object parameter calculation unit 12 has a function of changing the position, size, and shape of the occlusion object image so that the display target image is visible to one eye of the observer but not to the other eye. Is provided. For example, the non-shielding area for one eye of the observer and the shielding area for the other eye are both controlled to be larger than the display target image.

  The motion update unit 13 (content display control unit) has a function of outputting predetermined content acquired from the content information storage unit 6 to the content display device 3 and displaying / reproducing the content on the display unit of the content display device 3.

  The above is the function of the stereoscopic image display control device 1. The stereoscopic image display control device 1 can be realized by a computer including a CPU, a memory, a hard disk, and the like. It is also possible to create a stereoscopic image display control program for causing a computer to function as the stereoscopic image display control apparatus 1 and a storage medium for the stereoscopic image display control program.

<Specific examples of real space to be processed>
Here, examples of the real space to be processed are shown in FIGS. As shown in FIG. 3, on the display unit 3 ′ of the content display device 3, the content related to insect ecology is reproduced and a stag beetle is displayed. In addition, a shielding object 2 ′ is installed at an intermediate point between the content display device 3 and the observer. The shielding object 2 ′ is made of grass or leaves grown in an insect habitat so that the observer is not conscious as much as possible. Further, the binocular position acquisition device 4 is installed directly above the content display device 3 so as to easily photograph the observer. In the example shown in FIG. 4, a transmissive display is used as the shielding object display device 2. Hereinafter, a region where a shielding object for shielding one eye of the observer exists may be referred to as a shielding layer.

<Operation example of stereoscopic image display control device>
Next, the operation of the stereoscopic image display control apparatus 1 will be described. An operation flow of the stereoscopic image display control apparatus 1 is shown in FIG.

  In this operation example, a case will be described in which a transmissive display is used as the occluding object display device 2 and both the occluded area and the non-occluded area are formed simultaneously only by the occluding object image.

  In this operation example, the images output to the content display device 3 and the occlusion object display device 2 are images rendered by converting the virtual space in the stereoscopic image display control device 1 (in the computer) into a screen space.

  In determining the position / shape of the occluding object, it is necessary to consider the three-dimensional positional relationship between the position of the observer's viewpoint, the position of the display target in the content, and the position of the occluding object. Consider a virtual space (FIG. 7) corresponding to the configuration of the real space (FIG. 6) before being converted to a three-dimensional screen space.

  The virtual space shown in FIG. 7 includes a polygon model of a display target (a model having a three-dimensional shape imitating a display target) included in the content, a polygon model of a occluding object constituting the occluding layer, and the right and left eyes of the observer. It is assumed that there are two virtual cameras VR and VL respectively corresponding to. Hereinafter, the operation of the stereoscopic image display control apparatus 1 will be described in detail with reference to FIGS.

  First, the stereoscopic image display control device 1 acquires content stored in the content information storage unit 6 and inputs the content to the motion update unit 13 (step S1).

  Next, the stereoscopic image display control device 1 is an object that is subjectively selected from the occlusion object information storage unit 5 by a user of the stereoscopic image display control device 1 (for example, a provider of stereoscopic video), and has content. At the same time, an optimal object that is naturally felt even if displayed is input to the occlusion object parameter calculation unit 12 and the motion update unit 13 as an occlusion object (step S2).

  The “naturally-sensed object” here refers to, for example, a virtual space formed by content, such as grass and seaweed, when living creatures that live in forests or the sea are displayed as display targets in the content. An object that feels comfortable as a surrounding environment. In addition, if the content is composed of a plurality of display targets and there is something that plays a role as a surrounding environment that does not give a sense of incongruity to some of the contents, the display target may be used as a shielding object image.

  Thereafter, the virtual camera parameter update unit 11 continuously captures and calculates the positions of both eyes of the observer using a binocular position acquisition device 4 such as a web camera with a program of a fast and robust algorithm (step S3). ), The positions of the two virtual cameras VR and VL in the virtual space are respectively updated using the calculated position information of both eyes of the observer (step S4). As for this algorithm, for example, “P. Viola and M. Jones,“ Robust real-time facedetection ”, International Journal of Computer Vision, vol. 57, no. 2, 2004, p.137-p.154. It can be realized using the Viola-Jones method described in the above.

  Next, the motion update unit 13 determines the position / posture / shape / size in the display target virtual space included in the content based on the image information in the content input in step S1 (step S5). Then, the position / posture / shape / size in the virtual space of the occluding object input in step S2 is determined (step S6).

  If the content or the occluding object includes animation information (moving image), the motion updating unit 13 in steps S5 and S6, the display target included in the content or the position / posture / shape of the occluding object according to the movement. -Update the size.

  Next, the occlusion object parameter calculation unit 12 uses the two virtual cameras VR and VL, the display object, and the position information of the occlusion object so that the display object is occluded with respect to one eye of the observer. Then, the position / posture / shape / size of the shielding object is adjusted (step S7).

  The operation in step S7 will be described in detail. In this operation example, in order to reduce the non-shielding feeling of the observer, the eye that observes the viewing area of the polygon model that is relatively narrow when viewed from each eye of the observer is selected as the eye that the shielding object hides. To do.

  Hereinafter, the control contents each time one frame of the shielding layer is rendered based on this policy will be described. Note that the position of the virtual camera VR for the right eye is VR (x_r, y_r, z_r), and the position of the virtual camera VL for the left eye is VL (x_l, y_l, z_l). Further, for easier understanding, a simpler bounding box (a rectangular parallelepiped composed of straight lines in contact with the polygon model) is used instead of the polygon model.

  In the case of FIG. 7, first, considering the straight line on the y_r plane passing through the right-eye virtual camera VR and in contact with the bounding box to be displayed, two straight lines are obtained for the right-eye virtual camera VR. The contact positions of these straight lines and the bounding box are R1 and R2.

  Similarly, when a straight line on the plane y_l passing through the left-eye virtual camera VL and in contact with the bounding box to be displayed is considered, two straight lines are obtained for the left-eye virtual camera VL. The contact positions of these straight lines and the bounding box are L1 and L2.

  Then, the magnitude relationship between | R1-R2 | and | L1-L2 | is obtained. The value of | R1-R2 | is the “surface area factor of the display target at the right eye viewpoint VR”, and the value of | L1-L2 | is the “surface area factor of the display object at the left eye viewpoint VL”.

  These calculations are performed not only on the respective planes y_r and y_l, but also for all tangents that pass through the two virtual cameras VR and VL and touch the bounding box to be displayed, and the surface area of each display target at each viewpoint VR and VL. A virtual camera corresponding to a contact group having a smaller average factor is a shielding target virtual camera, and the other is a non-shielding target virtual camera. However, when the two average values compared are equal, the virtual camera closer to the center coordinate or the center coordinate of the bounding box is set as the shielding target virtual camera.

  Through the processing so far, the above-described policy (selecting the eye that observes the viewing area of the relatively narrow display target polygon model as viewed from each eye of the observer as the eye hidden by the occlusion object) is realized. Is done.

  Next, a method for adjusting the position / shape / size of the shielding object will be described. The area of the occlusion area changes depending on the z-coordinate of the occlusion object (the distance from the occlusion target virtual camera to the occlusion object). Here, the occlusion object is farthest from the occlusion target virtual camera, and the occlusion object is the non-occlusion target virtual. Consider meeting the two requirements of not being photographed by a camera.

  Of the four straight lines described above, the intersection of the straight lines that intersect with each other is A, and an arbitrary point on the straight line that does not include the intersection A through the shielding target virtual camera (VL in the case of FIG. 7) and the intersection A are both ends. Or move the occluding object to include both ends. In addition, when a shielding object is moved, if the shielding width (width on the x-axis) of the shielding object is less than the minimum shielding width including the intersection A and the display target is observed from the virtual camera to be shielded, the shielding is performed. Increase the occlusion width of the occlusion object until it reaches the minimum width.

  When the shielding object and the display target are both simple rectangles, the display pair can be shielded by performing the above processing on one position in the y-axis direction of the shielding object. On the other hand, when the shape of the shielding object and the display target is complicated, for example, when the periphery of the shape of the shielding object has nonlinearity, the above processing is performed at all positions (heights) in the y-axis direction of the shielding object. I do. Further, if necessary, the height of the shielding object may be adjusted to the minimum height at which the display target is not observed from the shielding target virtual camera.

  On the other hand, with respect to the non-shielding target virtual camera (VR in the case of FIG. 7), two straight lines passing through the non-shielding target virtual camera and the two straight lines are in contact with each other in order to secure a viewing area as large as possible. It is determined whether or not a shielding object is included in the area formed by the contact points (R1, R2,...) Of the bounding box. If a shielding object is included, the position of the shielding object is moved outside the viewing area by a minimum distance. Let For example, it is determined whether or not there is an overlap with the shielding object on the y axis including the contact point at the maximum height when viewed from the position of the non-shielding target virtual camera.

  Such processing of step S7 is performed for all polygon models to be displayed. It should be noted that in order to prevent the calculation processing in the computer that obtains the shielding area and the non-shielding area from being increased due to an increase in polygon models, only the polygon model that is a display object with high priority selected in advance by the user may be set as the processing object.

  Further, when a flat screen is used as the occlusion object display device 2, it is difficult to move the occlusion object in the z-axis direction in the real world, so a monocular viewpoint image cannot be generated at the intersection A shown in FIG. There is a possibility that image separation processing is performed at different positions.

  In such a case, a part of the obscured object enters the field of view for the non-occluded eye, and the display target is missing. Therefore, in order to prevent this, as shown in FIG. 8, a display device having local selectivity of transparency and movable in the z-axis direction is used. When this display device is used, the position of the display device on the z-axis is moved to the position A ′ corresponding to the coordinate position of the intersection A in the virtual space.

  After the obscuration object in the processing target frame and the parameters indicating the position, shape, and size of the obscuration object are determined by the above processing, the stereoscopic image display control device 1 renders the virtual space. In step S8, the pixel value of the virtual space converted to the screen space is passed to each display device (step S9).

  Specifically, the motion update unit 13 outputs the pixel value of the content to the content display device 3, and the shielding object parameter calculation unit 12 outputs the pixel value of the shielding object to the shielding object display device 2. Note that the virtual camera that renders the pixel value to be output to the occluding object display device 2 may be handled by a VC (see FIG. 7) positioned between VR and VL.

  Each process of step S3-step S9 demonstrated above is performed for every frame from which the binocular position information of the observer was acquired from the binocular position acquisition device 4. FIG.

<Effect>
According to the present embodiment, the display target image included in the content is visible to one eye of the viewer on the display unit of the shielding object display device 2 installed between the content display device 3 and the viewer. Because it forms a semi-shielded state that cannot be seen by the other eye, it does not require the observer to wear glasses, and it is a simple device configuration consisting of two general-purpose display devices. A stereoscopic image can be perceived. For example, a general-purpose flat panel display and a transmissive display can be realized with a relatively inexpensive hardware configuration of one each.

DESCRIPTION OF SYMBOLS 100 ... Stereoscopic image display control system 1 ... Stereoscopic image display control apparatus 11 ... Virtual camera parameter update part 12 ... Occlusion object parameter calculation part 13 ... Motion update part 2 ... Occlusion object display device 3 ... Content display device 4 ... Binocular position acquisition Device 5 ... Occlusion object information storage unit 6 ... Content information storage unit

Claims (6)

A content display control unit for displaying content on the display unit of the first display device;
On the display unit of the second display device installed between the first display device and the viewer of the content, the display target image included in the content is visible to one eye of the viewer and the other A three-dimensional image formation control unit that forms a semi-shielded state invisible to one eye;
A stereoscopic image display control device comprising: The second display device
The stereoscopic image display control device according to claim 1, wherein the stereoscopic image display control device is a transmissive display device capable of changing a transparency or a transmissive region of the display unit. The stereoscopic image formation control unit
The stereoscopic image display control according to claim 1, wherein a shielding area for shielding the display target image is formed on a display unit of the second display device using an object image related to the content. apparatus. The second display device is a transmissive display device including a transmissive display unit,
The stereoscopic image formation control unit
Position of an object image to be displayed on the transmissive display device for shielding the display target image so that the display target image is visible to one eye of the observer and not visible to the other eye The stereoscopic image display control device according to claim 1, wherein the size or shape is changed. In the stereoscopic image display control method performed by the stereoscopic image display control device,
The stereoscopic image display control device includes:
Displaying the content on the display unit of the first display device;
On the display unit of the second display device installed between the first display device and the viewer of the content, the display target image included in the content is visible to one eye of the viewer and the other Forming a semi-shielded state invisible to one eye;
A stereoscopic image display control method characterized by:   A stereoscopic image display control program for causing a computer to function as the stereoscopic image display control device according to claim 1.