JP2019050583A - Display and program - Google Patents

Abstract

To reduce a user's discomfort and fatigue in three-dimensional display.SOLUTION: A display comprises: a display unit that displays a first object image observed by an observer; a detection unit that detects an object present between the display unit and the observer; and a control unit that, when observation of the first object image by the observer is at least partially blocked by the object, performs control of changing the display of the first object image by the display unit to display in which the first object image is observed to be closer to the display unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display device for displaying an image in a three-dimensional manner, and a program.

  By showing different images to the left and right eyes, there is a three-dimensional display that shows the user a three-dimensional image. According to Patent Document 1, when viewing an image using shutter-type glasses, when the image is blocked by a hand, the shutter on one side of the shutter-type glasses is closed so that the image can be delivered to only one eye. It is described that it prevents outbreak.

JP, 2011-90611, A

  However, when viewing an image on a three-dimensional display, it may be considered that if the left and right eyes look at completely different images, they cause binocular rivalry and give the user a sense of discomfort or fatigue.

  The present invention has been made in view of such a situation, and provides a display device and a program that reduce the user's discomfort and fatigue in three-dimensional display.

  In order to solve the problems described above, the present invention provides a display unit for displaying a first object image observed by an observer, a detection unit for detecting an object between the display unit and the observer, When the observer's observation of the first object image is at least partially blocked by the object, the display of the first object image by the display unit is performed with the first object image being closer to the display unit side. And a control unit that performs control to change to a display to be observed as it is.

  Further, according to the present invention, a process of causing a display unit to display a first object image to be observed by an observer, a process of detecting an object present between the display unit and the observer, and a process by the observer When the observation of the first object image is at least partially blocked by the object, the display of the first object image by the display unit is observed such that the first object image is closer to the display unit side. It is a program for making a computer perform the process to which it is changed to the display made.

  As described above, according to the present invention, binocular rivalry can be reduced.

It is a block diagram showing an example of composition of a display by one embodiment of the present invention. It is a figure which shows the external appearance of the display apparatus by one Embodiment of this invention. It is a figure which shows the positional relationship of the display apparatus by one Embodiment of this invention, and a user. It is a figure which shows the positional relationship of the display apparatus by one Embodiment of this invention, and a user. It is a figure which shows the outline | summary of the solid image which the display part by one Embodiment of this invention displays. It is a figure which shows the example of the shielding area | region determined not to change the parallax amount of a stereo image by one Embodiment of this invention. It is a figure which shows the example of the shielding area | region determined to change the amount of parallax of a stereo image by one Embodiment of this invention. It is a figure which shows the example of the shielding area | region determined to change the amount of parallax of a stereo image by one Embodiment of this invention. It is a figure which shows the example of the positional relationship of the user and three-dimensional image by one Embodiment of this invention. It is a figure which shows the example of the solid image which the display part by one Embodiment of this invention displays. It is a flowchart which shows the operation example of the display apparatus by one Embodiment of this invention.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a display device 100 according to the present embodiment. The display device 100 includes an imaging unit 101, a detection unit 102, a display unit 103, a display image storage unit 104, a spatial frequency detection unit 105, and a control unit 106.

  FIG. 2 is a view showing the appearance of the display device 100. As shown in FIG. The display apparatus 100 displays a parallax image on the display surface of the display unit 103, and shows an image having binocular parallax to the left and right eyes of the user (symbol a), thereby displaying a cubic image (symbol b) to the user. It is a former display. The display device 100 may perform three-dimensional display by any of the naked eye type and the glasses type, but in the present embodiment, an example of the naked eye type will be described. Further, the display device 100 may apply any display display principle such as LCD (Liquid Crystal Display), plasma, organic EL (Electro Luminescence) or the like. In addition, the display device 100 includes an imaging unit 101 that captures an image of the user on the front upper side. However, the imaging unit 101 may be provided at a position at which the display direction of the screen can be imaged even if it is not on the front upper side.

  Returning to FIG. 1, the photographing unit 101 is a camera that photographs a predetermined area in the display direction by the display unit 103 and generates a photographed image. Here, the predetermined area is an area including a space area that will shield the stereoscopic image if an object is present when the user visually recognizes the stereoscopic image displayed by the display unit 103. The imaging unit 101 may capture a moving image, or may capture a still image at regular time intervals. Further, the photographing unit 101 may be one camera or a plurality of cameras.

  The detection unit 102 detects the position of the shield in a predetermined area in the display direction of the display unit 103 and the eye position of the user based on the captured image captured by the imaging unit 101. For example, the detection unit 102 analyzes a captured image captured by the imaging unit 101, and detects a moving object included in the captured image. The detection unit 102 can extract a difference area for each frame of the moving image captured by the imaging unit 101, and determine that a moving object exists in the area in which the difference occurs. For example, as illustrated in (a) of FIG. 3, the detection unit 102 detects a user (a1) present in the display direction of the display device 100 from an image captured by the imaging unit 101. At this time, for example, a photographed image as shown in (b) is photographed by the photographing unit 101 of the display device 100.

  Furthermore, the detection unit 102 detects at least one feature amount of the position of the shield in the depth direction with respect to the display unit 103, the position in the direction parallel to the display surface of the display unit 103, the size, and the shape. The detection unit 102 detects the face area of the user included in the captured image and the area of the hand that is a shielding object using a face detection algorithm, a hand area detection algorithm, etc. , Size, shape detection. Here, when the face area of the user can not be detected from the captured image, the detection unit 102 detects the position of the user's torso or arm and calculates the estimated position of the user's face. In addition, the detection unit 102 detects the eye gaze position of the user from the position of the face of the user thus detected.

  Here, when a stereoscopic image is displayed by the naked eye method, as shown in FIG. 4, the area of the eye position of the user who can observe the stereoscopic image is limited. That is, the position of such a viewable area (b) is known, and the eye position of the user is predetermined. Therefore, the position of such a viewable area may be stored in advance in the storage area of the display device 100 as the eye position of the user.

  The display unit 103 changes a parallax amount of an image to be displayed to display a stereoscopic image based on binocular parallax. FIG. 5 is a view showing an outline of a stereoscopic image displayed by the display unit 103. As shown in FIG. As shown in (a), when the user views the display unit 103 of the display device 100 from the position of a1, the right eye image given to the right eye of the user is given to the left eye of the user at the position a2 By displaying the image at the position of the code a3, a stereoscopic image based on binocular parallax can be popped out and displayed at the position of the code a4. At this time, for example, if there is an obstacle such as a hand as shown by a symbol a5 in the viewing direction from the user's eye to the display unit 103, the left eye image looks as shown in (b) and the right eye image It looks as shown in c). At this time, binocular rivalry may occur because the shielded areas of the right-eye image and the left-eye image are different.

  In addition, the display unit 103 sets the same value in the depth map of all the pixels in the display image by displaying such a parallax image, and the entire image is displayed in three dimensions uniformly, or is included in the display image. Different parallaxes can be given to each area of an object to be stereoscopically displayed, or the unevenness included in a single object can be stereoscopically displayed. Here, an object is a unit of an area of an object represented by a texture of a display image. For example, when the display image includes a mountain, a sky, and a person, the display image includes three of the mountain, the sky, and the person. Contains two objects.

Returning to FIG. 1, the display image storage unit 104 stores a display image to be displayed on the display unit 103. The display image may be stored in advance in the display image storage unit 104, or may be received and stored from the outside via a wireless or wired network.
The spatial frequency detection unit 105 detects spatial frequency components in the display image. Here, the spatial frequency detection unit 105 detects at least a part of spatial frequency components in the display image, but may detect the entire spatial frequency components in the display image.

The control unit 106 reads the display image stored in the display image storage unit 104, generates a parallax image of the right eye image and the left eye image, and causes the display unit 103 to display the parallax image. When the shielding unit is detected by the detection unit 102, the control unit 106 changes the parallax amount of the stereoscopic image displayed on the display unit 103 based on the detection result.
Here, the control unit 106 displays from the user's eye position based on the position of the shield (hand) detected by the detection unit 102, the user's eye position, and the predetermined position of the display unit 103. The shielded area of the stereoscopic image shielded by the shield in the viewing direction with respect to the unit 103 is determined. As described above, the eye-gaze position may be an estimated position of the user's face calculated based on the position of the user's torso or arm, or may be a predetermined viewable area. For example, by connecting the eye position of the user and the display screen on the display unit 103 on a straight line, and projecting the shield on the display screen based on the position, size, shape, etc. of the shield existing on the straight line. , The shielded area of the stereoscopic image can be determined.

  Further, the control unit 106 determines whether or not to change the amount of parallax of the stereoscopic image based on the shielded area of the stereoscopic image. For example, when the size of the shielding area of the three-dimensional image shielded by the shielding object detected by the detection unit 102 exceeds a predetermined value, the control unit 106 determines the parallax amount of the three-dimensional image displayed by the display unit 103 Is determined to change. When the size of the area of the three-dimensional image shielded by the shield is small, even if the three-dimensional image is shielded by the shield, the amount of parallax is not changed because fusion is possible.

  FIG. 6 is a diagram illustrating an example of a shielded area in which the control unit 106 determines not to change the amount of parallax of a stereoscopic image. As shown in (a) of FIG. 6, when the area of the three-dimensional image shielded by the shield is small and it is possible to interpolate and perceive the shielded region, the amount of parallax is not changed because fusion is possible. As shown in (b) and (c), even if the ratio of the shielding area to the entire three-dimensional image is large, fusion is possible because it can be perceived as interpolated unless the outermost periphery (frame) of the three-dimensional image is shielded. Therefore, the amount of parallax is not changed. As shown in (d), even if the shielded area is large, the ratio of the shielded area to the entire three-dimensional image is small, and it can be perceived as interpolated unless the outermost periphery (image frame) of the three-dimensional image is shielded. The amount of parallax is not changed because fusion is possible. As described above, the control unit 106 determines whether the area of the region of the three-dimensional image shielded by the shield detected by the detection unit 102 exceeds a predetermined area threshold or the shielding detected by the detection unit 102. It is determined whether the area of the region of the three-dimensional image shielded by the object exceeds a certain ratio or more with respect to the area of the entire three-dimensional image, and it is determined whether the amount of parallax is changed.

  FIGS. 7 and 8 are diagrams showing an example of a shielded area that the control unit 106 determines to change the amount of parallax of a stereoscopic image. As shown in (a) of FIG. 7, when the shielded area by the shield is large and the shielded area can not be perceived by interpolation, it is determined that the amount of parallax is to be changed. As shown in (b), even when the shielded area is large and the shielded area at the outermost periphery is large, it is not possible to interpolate and perceive the shielded area, so it is determined to change the amount of parallax. As shown in (c), even when the shielded area at the outermost periphery is large, the shielded area can not be interpolated and perceived, so it is determined that the amount of parallax is changed.

  FIG. 8 is a view showing the shielding area in the example shown in FIG. (A), (b) and (c) shown in FIG. 8 correspond to (a), (b) and (c) shown in FIG. 7, respectively. (A) is an example in case the number of places of a shielding field exceeds a threshold of a predetermined number of places, and the sum total of a shielding field exceeds a threshold of a predetermined area. For example, the ratio of the shielded area to the entire three-dimensional image is 10% or more, and the number of shielded areas is two or more, and the total of the shielded areas is 40% or more of the entire three-dimensional image. (B) is an example in case the sum total of a shielding area exceeds a threshold of a predetermined area. For example, the ratio of the shielded area to the entire three-dimensional image is 50% or more. (C) is an example in case the shielding area | region in the outermost periphery of a solid image exceeds a predetermined threshold-value. For example, the ratio of the shielded area at the outermost periphery of the entire three-dimensional image is 40% or more.

  When the control unit 106 determines that the amount of parallax is to be changed, the control unit 106 reduces the amount of parallax of the stereoscopic image. This is because the binocular rivalry of a stereoscopic image generally increases as the amount of parallax increases. Therefore, the binocular rivalry can be reduced by reducing the amount of parallax. When the parallax amount is further reduced, the display position of the stereoscopic image comes closer to the display surface of the display unit 103. Here, reducing the amount of parallax includes making the amount of parallax zero and uniformly displaying a display image on the display surface of the display unit 103 in a two-dimensional manner. Alternatively, the control unit 106 may change the amount of parallax so as to increase again when the detection unit 102 no longer detects the shield after the detection unit 102 detects the shield and reduces the amount of parallax. it can.

  Further, the control unit 106 changes the parallax amount of the stereoscopic image according to the feature amount of the shielding object detected by the detection unit 102. For example, the control unit 106 controls the stereoscopic image displayed by the display unit 103 to be displayed on the display surface side of the display unit 103 rather than the position in the depth direction of the shield detected by the detection unit 102. For example, as shown in FIG. 9, with respect to the eye position of the user (a) facing the display device 100, the position (b1) closer to the user than the hand position (b2) which is a shield of the stereoscopic image in the depth direction When a stereoscopic image is displayed on the display unit 103, control is performed so that the stereoscopic image is displayed at a position (b3) on the display surface side of the display unit 103 with respect to the hand position (b2). As described above, since the binocular rivalry of a stereoscopic image generally increases as the parallax amount increases, the parallax amount is reduced so that the stereoscopic image is displayed at least on the display surface side relative to the shielding object. This is because it is possible to reduce eye-view struggle.

  Here, when the detection unit 102 detects a shield, the control unit 106 can also decrease the amount of parallax of all the stereoscopic images included in the display image, or the shield is shielded by the shield in the region of the solid image. It is also possible to change the amount of parallax in a part of the area including the outer circumference of the part. For example, the control unit 106 can reduce binocular rivalry by reducing the amount of parallax of only the object shielded by the shield. For example, it is assumed that the display unit 103 is displaying a display image as shown in FIG. In the example shown in (a), a stereoscopic image of only the object of the icon included in the display image is displayed. In this case, if the user's hand (obstacle) is detected at a position where the object shown in a1 is pressed, the amount of parallax of only the area of the object shown in a1 is reduced to alleviate binocular rivalry. be able to. Similarly, in the example shown in (b), a stereoscopic image of only the object of the operation button included in the display image is displayed. In this case, if the user's hand (obstacle) is detected at a position where the object shown in b1 is pressed, the parallax amount of only the area of the object shown in b1 is reduced to alleviate the binocular rivalry. be able to. However, it is also possible to reduce the amount of parallax of all stereoscopic images in the display image. For example, when displaying a movie or the like including objects such as a plurality of persons or backgrounds, if there is a sense of discomfort when reducing the parallax amount of only a specific object (for example, a person), all three-dimensional objects in the display image The amount of parallax of the image can be reduced as well.

In addition, the control unit 106 changes the parallax amount based on the spatial frequency of the image of the area including the outer periphery of the solid image shielded by the shield among the areas of the solid image detected by the spatial frequency detection unit 105 It can be done. For example, when the spatial frequency is high, the binocular rivalry is considered to be large, so the parallax amount of the stereoscopic image is reduced.
Furthermore, the control unit 106 can also determine an operation on the display device 100 according to the movement of the moving body detected by the detection unit 102. That is, the control unit 106 can change the display image or the like according to the input of the operation gesture by the user's operation.
In this case, the control unit 106 controls a display image according to the input operation gesture.

Next, an operation example of the display device 100 according to the present embodiment will be described with reference to the drawings. FIG. 11 is a flowchart showing an operation example in which the display device 100 adjusts the parallax.
The control unit 106 reads the display image stored in the display image storage unit 104 to generate a parallax image, and causes the display unit 103 to display the parallax image. The photographing unit 101 photographs the display direction by the display unit 103 and generates a photographed image. When the detection unit 102 detects a shield from the captured image generated by the imaging unit 101, the detection unit 102 calculates a feature amount of the shield (step S1). The detection unit 102 also detects the eye gaze position of the user (step S2). The control unit 106 determines the shielding area of the stereoscopic image in the user's visual direction based on the feature amount of the shielding detected in step S1 and the eye position of the user detected in step S2 (step S3). . The control unit 106 determines whether or not to change the amount of parallax of the three-dimensional image based on the area of the shielded area or the like (step S4). If it is determined that the parallax amount of the stereoscopic image is to be changed (step S4: YES), the control unit 106 generates a parallax image in which the parallax amount is changed and causes the display unit 103 to display the parallax image (step S5). On the other hand, when determining that the amount of parallax of the stereoscopic image is not changed (step S4: NO), the control unit 106 does not change the amount of parallax.

  As described above, according to the display device 100 of the present embodiment, a shield is present in the display direction in which the display device 100 displays a stereoscopic image, and different parts of the left and right eyes are shielded when viewed by the user. When an image is displayed and binocular rivalry occurs, the amount of parallax of the stereoscopic image can be reduced or made zero. Thereby, even when the user is looking at the three-dimensional image displayed by the display device 100 and the three-dimensional image is blocked by the shielding object, the three-dimensional image can be comfortably without causing discomfort or stress due to binocular rivalry Can be watched. In particular, when the display device 100 receives an input by an operation gesture according to a solid image to be displayed, it is effective because it is considered that the user's hand may shield the solid image to perform an operation.

In the present embodiment, an example in which the detection unit 102 calculates the three-dimensional position of the user based on the photographed image photographed by the photographing unit 101 has been described. However, for example, a distance image sensor is applied as the detection unit 102 The position of the object or the face of the user may be detected. Alternatively, for example, when performing glasses-type three-dimensional display, the display device 100 can communicate with the glasses to determine the eye-line position based on the position of the glasses.
Further, for example, a digital photo frame, digital signage, home television, or the like can be applied to the three-dimensional display which is the display device 100 in the present embodiment. Alternatively, a smartphone capable of three-dimensional display may be applied. In this case, for example, a portion touched by the touch panel can be determined as a shielded area.

  In addition, a program for realizing the function of the processing unit in the present invention is recorded in a computer readable recording medium, and the program recorded in the recording medium is read into a computer system to execute display control. May be Here, the “computer system” includes an OS and hardware such as peripheral devices. The "computer system" also includes a WWW system provided with a homepage providing environment (or display environment). The term "computer-readable recording medium" refers to a storage medium such as a flexible disk, a magneto-optical disk, a ROM, a portable medium such as a ROM or a CD-ROM, or a hard disk built in a computer system. Furthermore, the "computer-readable recording medium" is a volatile memory (RAM) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those that hold the program for a certain period of time are also included.

  The program may be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the “transmission medium” for transmitting the program is a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the program may be for realizing a part of the functions described above. Furthermore, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

100: display device 102: detection unit 103: display unit 105: spatial frequency detection unit 10
6 ... Control unit

Claims (10)

A display unit for displaying a first object image observed by the observer;
A detection unit that detects an object between the display unit and the observer;
When the observer's observation of the first object image is at least partially blocked by the object, the display of the first object image by the display unit is performed with the first object image being closer to the display unit side. A control unit that performs control to change to a display to be observed as it is;
A display device comprising: The control unit causes the display unit to display the first object image by the display unit when the observation of the first object image by the observer is at least partially blocked by the object. The display device according to claim 1, wherein the control to change the display is performed by performing control to change the display to be observed from the object to the display unit side. The display unit displays a second object image on which the observer stereoscopically observes;
The control unit performs control to change the display of the second object image by the display unit when the observation of the first object image by the observer is at least partially blocked by the object. The display device according to 2. The display unit displays a second object image on which the observer stereoscopically observes;
The control unit performs control to change only the display of the first object image by the display unit when the observation of the first object image by the observer is partially or entirely blocked by the object. A display device according to item 1 or 2. The control unit may be configured such that the ratio of the portion of the portion where the observation of the first object image by the observer is obstructed by the object to the entire observation of the first object image is larger than a predetermined value. The display device according to any one of claims 1 to 4, which performs control to change display of an object image. When the ratio of the portion where the observation of the first object image is interrupted by the object to the whole of the observation of the first object image is larger than a predetermined value, the control unit controls the first object image A first control to change the display from the first state to the second state is performed, and thereafter, when the ratio becomes equal to or less than the predetermined value, the display of the first object image from the second state to the first state The display device according to claim 5, wherein a second control is performed to change the value of The control unit is configured to display the first object image from a first state to a second state when an amount of a portion where the observation of the first object image by the observer is interrupted by the object is larger than a predetermined value. A first control for changing to the first state is performed, and a second control for changing the display of the first object image from the second state to the first state is performed when the amount is equal to or less than the predetermined value. The display device according to any one of claims 1 to 4. The control unit is configured such that a ratio of a portion where the observation of the first object image by the observer is blocked by the object and the observation of the outer peripheral portion occupied in the outer peripheral portion of the first object image is a predetermined value If the ratio is larger, the first control is performed to change the display of the first object image from the first state to the second state, and when the ratio is less than the predetermined value, the display of the first object image is The display device according to any one of claims 1 to 4, wherein a second control for changing from a second state to the first state is performed. The control unit causes a first object image display image and a second object image display image corresponding to the first object image to display the first object image in a three-dimensional manner as the first object. The display apparatus according to any one of claims 1 to 8, wherein the display unit displays the image as an image. A process of causing the display unit to display a first object image observed by the observer;
A detection process for detecting an object between the display unit and the observer;
When the observer's observation of the first object image is at least partially blocked by the object, the display of the first object image by the display unit is performed with the first object image being closer to the display unit side. Processing to change to a display to be observed as it is;
, A program for making a computer execute.

Images