JP5879856B2 - Display device, display method, and program - Google Patents

Description

  The present invention relates to a technique for displaying an image three-dimensionally.

  There is a three-dimensional display that shows a stereoscopic image to the user by showing different images to the left and right eyes. In Patent Document 1, when viewing an image using shutter-type glasses, if the image is blocked by a hand, the shutter on one side of the shutter-type glasses is closed so that the image can reach only one eye. It is described to prevent the occurrence.

JP 2011-90611 A

  However, when viewing an image on a three-dimensional display, if the left and right eyes see completely different images, it may cause a binocular rivalry, which may cause the user to feel uncomfortable or tired.

The present invention has been made in view of such a situation, and provides a display device, a display method, and a program that reduce a user's uncomfortable feeling and fatigue in a three-dimensional display.

To solve the problems described above, an embodiment of the present invention displays an image on the display surface, and a display unit for displaying the image image based Dzu rather the image in a position different from the display surface, detecting an object a detection unit for a control unit that controls the display unit based on the size of the front portion Kizo is blocked by said detected object by the detection unit changes the position at which the image is displayed, It is a display apparatus provided with.
Further, an embodiment of the present invention displays an image on the display surface, and a process of displaying an image based on the image in a position different from the display surface, and a process for detecting an object, said by said detected object is a program for executing a process of changing the position of pre-Symbol image is displayed based on the size of the portion where the image is blocked, to the computer.
One embodiment of the present invention is a method for displaying an image on a display surface, displaying an image based on the image at a position different from the display surface, a method for detecting an object, and the detected object based on the detected object . how to change the position of pre-Symbol image is displayed based on the size of the portion where the image is blocked, a display method comprising.

  As described above, according to the present invention, binocular rivalry can be reduced.

It is a block diagram which shows the structural example of the display apparatus by one Embodiment of this invention. It is a figure which shows the external appearance of the display apparatus by one Embodiment of this invention. It is a figure which shows the positional relationship of the display apparatus and user by one Embodiment of this invention. It is a figure which shows the positional relationship of the display apparatus and user by one Embodiment of this invention. It is a figure which shows the outline | summary of the stereo image which the display part by one Embodiment of this invention displays. It is a figure which shows the example of the shielding area determined not to change the parallax amount of a stereo image by one Embodiment of this invention. It is a figure which shows the example of the shielding area determined to change the parallax amount of a stereo image by one Embodiment of this invention. It is a figure which shows the example of the shielding area determined to change the parallax amount of a stereo image by one Embodiment of this invention. It is a figure which shows the example of the positional relationship of the user and stereo image by one Embodiment of this invention. It is a figure which shows the example of the stereo image which the display part by one Embodiment of this invention displays. It is a flowchart which shows the operation example of the display apparatus by one Embodiment of this invention.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing the configuration of the display device 100 according to the present embodiment. The display device 100 includes an imaging unit 101, a detection unit 102, a display unit 103, a display image storage unit 104, a spatial frequency detection unit 105, and a control unit 106.

  FIG. 2 is a diagram illustrating an appearance of the display device 100. The display apparatus 100 displays a parallax image on the display surface of the display unit 103, and shows a stereoscopic image (symbol b) to the user by displaying images with binocular parallax to the left and right eyes of the user (symbol a). The original display. The display device 100 may perform three-dimensional display using either a naked eye type or a glasses type, but in this embodiment, an example using the naked eye type will be described. Further, the display device 100 may apply any display display principle such as LCD (Liquid Crystal Display), plasma, or organic EL (Electro Luminescence). In addition, the display device 100 includes a photographing unit 101 that photographs a user on the upper front portion. However, the photographing unit 101 need only be provided at a position where the display direction of the screen can be imaged, even if it is not at the upper front.

  Returning to FIG. 1, the photographing unit 101 is a camera that photographs a predetermined area in the display direction by the display unit 103 to generate a photographed image. Here, the predetermined area is an area including a spatial area that shields the stereoscopic image if an object is present when the user visually recognizes the stereoscopic image displayed by the display unit 103. The imaging unit 101 may shoot a moving image or may shoot a still image at regular intervals. The photographing unit 101 may be a single camera or a plurality of cameras.

  The detection unit 102 detects the position of the shield in a predetermined area and the position of the user's eye line in the display direction of the display unit 103 based on the captured image captured by the imaging unit 101. For example, the detection unit 102 analyzes a captured image captured by the capturing unit 101 and detects a moving object included in the captured image. The detection unit 102 can extract a difference area for each frame of the moving image shot by the shooting unit 101 and determine that a moving object exists in the area where the difference occurs. For example, as illustrated in FIG. 3A, the detection unit 102 detects a user (a1) that exists in the display direction of the display device 100 from an image captured by the imaging unit 101. At this time, for example, a photographed image as shown in (b) is photographed by the photographing unit 101 of the display device 100.

  Further, the detection unit 102 detects at least one feature amount of the position of the shielding object in the depth direction with respect to the display unit 103, the position in the parallel direction with respect to the display surface of the display unit 103, the size, and the shape. The detection unit 102 detects a user's face area or a hand area that is a shielding object included in the captured image using a face detection algorithm, a hand area detection algorithm, or the like, and detects the face position or hand position in three dimensions. Detect size, shape. Here, when the user's face area cannot be detected from the captured image, the detection unit 102 detects the position of the user's body or arm and calculates the estimated position of the user's face. Further, the detection unit 102 detects the user's eye position from the detected position of the user's face.

  Here, when a stereoscopic image is displayed by the naked eye method, as shown in FIG. 4, the region of the user's eye position where the stereoscopic image can be observed is limited. That is, the position of such an observable region (b) is known, and the user's eye position is determined in advance. Therefore, the position of the observable area may be stored in advance in the storage area of the display device 100 as the user's eye position.

  The display unit 103 displays a stereoscopic image based on binocular parallax by changing the parallax amount of the image to be displayed. FIG. 5 is a diagram illustrating an outline of a stereoscopic image displayed on the display unit 103. As shown in (a), when the user visually recognizes the display unit 103 of the display device 100 from the position a1, the right eye image given to the user's right eye is given to the user's left eye at the position a2 By displaying the image at the position of the symbol a3, a stereoscopic image based on the binocular parallax can be projected and displayed at the position of the symbol a4. At this time, for example, if there is a shielding object such as a hand as indicated by reference numeral a5 in the viewing direction from the user's eye to the display unit 103, the left eye image looks as shown in (b), and the right eye image is ( It looks as shown in c). At this time, the binocular rivalry may occur because the occluded areas of the right eye image and the left eye image are different.

  Further, the display unit 103 displays such a parallax image, thereby setting the same value to the depth map of all the pixels in the display image so that the entire image is uniformly displayed in three dimensions, or included in the display image. It is possible to display differently by providing different parallax for each object area, or to display unevenness included in a single object in three dimensions. Here, the object is a unit of a region of an object represented by the texture of the display image. For example, when the display image includes a mountain, the sky, and a person, the display image includes 3 of the mountain, the sky, and the person. Contains one object.

Returning to FIG. 1, the display image storage unit 104 stores a display image to be displayed on the display unit 103. The display image may be stored in advance in the display image storage unit 104, or may be received from the outside via a wireless or wired network and stored.
The spatial frequency detection unit 105 detects a spatial frequency component in the display image. Here, the spatial frequency detection unit 105 detects at least a part of the spatial frequency component in the display image, but may detect the entire spatial frequency component in the display image.

The control unit 106 reads the display image stored in the display image storage unit 104, generates a parallax image between the right eye image and the left eye image, and causes the display unit 103 to display the parallax image. When the detection unit 102 detects the shielding object, the control unit 106 changes the parallax amount of the stereoscopic image displayed on the display unit 103 based on the detection result.
Here, the control unit 106 displays from the user's eye position based on the position of the shield (hand) detected by the detection unit 102, the user's eye position, and the predetermined position of the display unit 103. The shielding area of the stereoscopic image shielded by the shielding object in the viewing direction with respect to the unit 103 is determined. As described above, the eye position may be an estimated position of the user's face calculated based on the position of the user's torso or arm, or may be a predetermined observable region. For example, the user's eye position and the display screen on the display unit 103 are connected on a straight line, and the shielding object is projected on the display screen based on the position, size, shape, and the like of the shielding object present on the straight line. The occluded area of the stereoscopic image can be determined.

  In addition, the control unit 106 determines whether or not to change the parallax amount of the stereoscopic image based on the shielding area of the stereoscopic image. For example, the control unit 106, when the size of the shielding area of the stereoscopic image shielded by the shielding object detected by the detection unit 102 exceeds a predetermined value, the parallax amount of the stereoscopic image displayed by the display unit 103 Is determined to change. If the size of the region of the stereoscopic image shielded by the shielding object is small, the parallax amount is not changed because fusion is possible even if the stereoscopic image is shielded by the shielding object.

  FIG. 6 is a diagram illustrating an example of a shielding area in which the control unit 106 determines not to change the parallax amount of the stereoscopic image. As shown in FIG. 6A, when the area of the stereoscopic image shielded by the shield is small and the shielded area can be perceived by interpolation, the amount of parallax is not changed because fusion is possible. As shown in (b) and (c), even if the ratio of the shielding area to the entire stereoscopic image is large, if the outermost periphery (image frame) of the stereoscopic image is not shielded, it can be perceived by interpolation, so that fusion is possible. Therefore, the amount of parallax is not changed. As shown in (d), even if the shielded area is large, the ratio of the shielded area to the entire stereoscopic image is small, and if the outermost periphery (image frame) of the stereoscopic image is not shielded, it can be perceived by interpolation. Since it can be fused, the amount of parallax is not changed. As described above, the control unit 106 determines whether or not the area of the stereoscopic image area shielded by the shielding object detected by the detection unit 102 exceeds a predetermined threshold value of the area, or the shielding detected by the detection unit 102. It is determined whether or not the area of the region of the stereoscopic image shielded by the object exceeds a certain ratio with respect to the area of the entire stereoscopic image, and whether or not the amount of parallax is changed is determined.

  7 and 8 are diagrams illustrating an example of a shielding area in which the control unit 106 determines to change the parallax amount of the stereoscopic image. As shown in (a) of FIG. 7, it is determined that the amount of parallax is changed because fusion is not possible when the shielded area by the shield is large and the shielded area cannot be perceived by interpolation. As shown in (b), even when the shielded area is large and the outermost shielded area is large, the shielded area cannot be perceived by interpolation, so it is determined that the amount of parallax is changed. As shown in (c), even when the outermost shielding area is large, it is determined that the parallax amount is changed because the shielding area cannot be perceived by interpolation.

  FIG. 8 is a diagram showing a shielding region in the example shown in FIG. (A), (b), and (c) shown in FIG. 8 correspond to (a), (b), and (c) shown in FIG. 7, respectively. (A) is an example in the case where the number of locations in the shielding area exceeds a predetermined threshold value for the number of locations, and the total shielding area exceeds a predetermined area threshold value. For example, this is a case where the number of locations of the shielding area where the ratio of the shielding area to the entire stereoscopic image is 10% or more is 2 and the total of the shielding areas is 40% or more of the entire stereoscopic image. (B) is an example in the case where the total of the shielding regions exceeds a predetermined area threshold. For example, this is a case where the ratio of the shielding area to the entire stereoscopic image is 50% or more. (C) is an example when the shielding area | region in the outermost periphery of a stereo image exceeds the predetermined threshold value. For example, it is a case where the ratio of the shielding area | region in the outermost periphery of the whole three-dimensional image is 40% or more.

  If the control unit 106 determines to change the amount of parallax, the control unit 106 decreases the amount of parallax of the stereoscopic image. This is because the binocular rivalry for stereoscopic images generally increases as the amount of parallax increases, and thus the binocular rivalry can be reduced by reducing the amount of parallax. When the amount of parallax is further reduced, the display position of the stereoscopic image comes closer to the display surface of the display unit 103. Here, reducing the amount of parallax includes making the amount of parallax zero and displaying the display image uniformly on the display surface of the display unit 103 in two dimensions. Alternatively, after detecting the shielding object by the detection unit 102 and reducing the amount of parallax, the control unit 106 may change the parallax amount again when the detection unit 102 no longer detects the shielding object. it can.

  Further, the control unit 106 changes the parallax amount of the stereoscopic image according to the feature amount of the shielding object detected by the detection unit 102. For example, the control unit 106 performs control so that the stereoscopic image displayed by the display unit 103 is displayed on the display surface side of the display unit 103 with respect to the position in the depth direction of the shielding object detected by the detection unit 102. For example, as shown in FIG. 9, the position (b1) on the user side relative to the position (b2) of the hand that is a stereoscopic image shielding object in the depth direction with respect to the position of the line of sight of the user (a) facing the display device 100. ), A stereoscopic image is controlled to be displayed at a position (b3) closer to the display surface of the display unit 103 than the position (b2) of the hand. As described above, since the binocular rivalry of a stereoscopic image generally increases as the amount of parallax increases, the amount of parallax is reduced by reducing the amount of parallax so that the stereoscopic image is displayed at least on the display surface side of the shielding object. This is because the eye-sight struggle can be reduced.

  Here, when the detection unit 102 detects the shielding object, the control unit 106 can also reduce the amount of all parallax of the stereoscopic image included in the display image, or can be shielded by the shielding object in the stereoscopic image region. It is also possible to change the amount of parallax in a partial region including the outer periphery of the portion. For example, the control unit 106 can reduce the binocular rivalry by reducing the amount of parallax of only the object shielded by the shield. For example, consider a case where the display unit 103 displays a display image as shown in FIG. In the example shown in (a), a stereoscopic image of only the icon object included in the display image is displayed. In this case, if the user's hand (obstacle) is detected at a position where the object indicated by a1 is pressed, the binocular rivalry is reduced by reducing the parallax amount of only the object area indicated by a1. be able to. Similarly, in the example shown in (b), a stereoscopic image of only the operation button object included in the display image is displayed. In this case, if the user's hand (shielding object) is detected at a position where the object indicated by b1 is pressed, the binocular rivalry is reduced by reducing the amount of parallax only in the area of the object indicated by b1. be able to. However, the amount of parallax of all stereoscopic images in the display image can be reduced. For example, when a movie or the like including objects such as a plurality of persons or backgrounds is displayed, if there is a sense of incongruity when the amount of parallax of a specific object (for example, a person) is reduced, all three-dimensional images in the display image are displayed. The amount of parallax of the image can be reduced similarly.

Further, the control unit 106 changes the amount of parallax based on the spatial frequency of the image of the region including the outer periphery of the three-dimensional image shielded by the shielding object among the three-dimensional image regions detected by the spatial frequency detection unit 105. Can be made. For example, when the spatial frequency is high, the binocular rivalry is considered to be large, and thus the amount of parallax of the stereoscopic image is reduced.
Furthermore, the control unit 106 can also determine an operation on the display device 100 according to the movement of the moving object detected by the detection unit 102. That is, the control unit 106 can change the display image or the like in response to an operation gesture input by a user's action. In this case, the control unit 106 controls the display image according to the input operation gesture.

Next, an exemplary operation of the display device 100 according to the present embodiment will be described with reference to the drawings. FIG. 11 is a flowchart illustrating an operation example in which the display device 100 performs parallax adjustment.
The control unit 106 reads the display image stored in the display image storage unit 104 to generate a parallax image, and causes the display unit 103 to display the parallax image. The imaging unit 101 captures a display direction by the display unit 103 and generates a captured image. When detecting the shielding object from the captured image generated by the imaging unit 101, the detection unit 102 calculates a feature amount of the shielding object (step S1). Moreover, the detection part 102 detects a user's eyes | visual_axis position (step S2). The control unit 106 determines the shielding area of the stereoscopic image in the user's viewing direction based on the feature amount of the shielding object detected in step S1 and the user's line-of-sight position detected in step S2 (step S3). . The control unit 106 determines whether or not to change the parallax amount of the stereoscopic image based on the area of the shielding region (step S4). When determining that the parallax amount of the stereoscopic image is to be changed (step S4: YES), the control unit 106 generates a parallax image with the parallax amount changed and displays the parallax image on the display unit 103 (step S5). On the other hand, when determining that the parallax amount of the stereoscopic image is not changed (step S4: NO), the control unit 106 does not change the parallax amount.

  As described above, according to the display device 100 of the present embodiment, there is a shielding object in the display direction in which the display device 100 displays a stereoscopic image, and if the user visually recognizes, different parts are shielded by the left and right eyes. When an image is displayed and a binocular rivalry occurs, the amount of parallax of the stereoscopic image can be reduced or zero. Accordingly, even when the stereoscopic image is shielded by the shielding object when the user is viewing the stereoscopic image displayed on the display device 100, the stereoscopic image is comfortably generated without causing a sense of incongruity or stress due to binocular rivalry. Can be watched. In particular, when the display device 100 receives an input by an operation gesture corresponding to a stereoscopic image to be displayed, it is effective because the user's hand may shield the stereoscopic image in order to perform the operation.

In the present embodiment, an example in which the detection unit 102 calculates the three-dimensional position of the user based on the captured image captured by the imaging unit 101 has been described. However, for example, a distance image sensor is applied as the detection unit 102 to block the image. The position of an object or a user's face may be detected. Alternatively, for example, when performing glasses-type three-dimensional display, the display device 100 can communicate with glasses and determine the eye position based on the position of the glasses.
In addition, for example, a digital photo frame, a digital signage, a home television, or the like can be applied to the three-dimensional display that is the display device 100 in the present embodiment. Alternatively, a smartphone capable of three-dimensional display may be applied. In this case, for example, the part touched by the touch panel can be determined as the shielding area.

  Note that a program for realizing the function of the processing unit in the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read by a computer system and executed to perform display control. May be. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 100 ... Display apparatus, 102 ... Detection part, 103 ... Display part, 105 ... Spatial frequency detection part, 106 ... Control part

Claims (17)

Displaying an image on a display surface, and a display unit for displaying the image image based Dzu rather the image in a position different from the display surface,
A detector for detecting an object body,
Display device and a control unit which controls the display unit based on the size of Kizo is blocked portion, it changes the position where the image is displayed before the said detected object in the detection unit. Wherein, prior to when the magnitude of Kizo is blocked portion is larger than a predetermined value, the display device according to claim 1 for changing the position where the image is displayed. Wherein, prior to when Kizo the size of occluded portions is smaller than a predetermined value, the display device according to claim 1 or claim 2 without changing the position where the image is displayed. The detection unit detects the position of the object and the position of the user,
Wherein the control unit determines the portion Kizo is interrupted before based on the position of the previous SL object and position of the user and the position of the display unit, the determined pre Kizo is obstructed portion of display device according to claims 1 to any one of claims 3 to control the pre-Symbol display unit based on the size. The control unit determines a portion where the image is blocked based on a viewing direction with respect to the position of the display unit from the position of the user.
The display device according to claim 4. The display unit displays the first image and the second image, and displaying the image by the parallax based on said first image and the second image,
Wherein, based on the size of the front portion Kizo is blocked by said detected object by the detection unit, according to claim from claim 1 for controlling the display of the first image or the second image The display device according to claim 5. Wherein, prior Kizo reduce the large case the parallax even magnitude than a predetermined value is shielded portion, a case where the size is smaller than a predetermined value the parallax before Kizo is blocked portion The display device according to claim 6 , wherein the display device is not changed. Claim wherein, before if Kizo the outer periphery of the front Kizo to have obstructed portion includes more than a predetermined value, which controls the pre-Symbol Display unit changes the position at which the image is displayed The display device according to any one of claims 1 to 7 . Includes a spatial frequency detection unit for detecting the spatial frequency component before Symbol image displayed on the display surface,
Wherein the control unit, the display device according to any one of claims 1, wherein said spatial frequency detection unit controls the pre Symbol Display unit on the basis of the spatial frequency component detected to claim 8. Wherein, prior to SL to determine the position or size or portions Kizo is interrupted before based on the shape of the object, before based on the magnitude of the determined front portion Kizo is blocked Symbol display device according to any one of claims 1 to 9 for controlling the Viewing section. Wherein the control unit controls the pre Symbol Display unit, display the previous Kizo between said display unit and said detected object by the detecting unit
The display device according to any one of claims 1 to 10 . The detection unit detects the object based on a captured image captured by the imaging unit,
The control unit controls the display unit based on the position, size, or shape of the object determined by the region of the object included in the captured image.
The display device according to any one of claims 1 to 11 . Wherein the detection unit, a display device according to any one of up to claims 1 to 12 for detecting an object in a predetermined area. The display device according to any one of claims 1 to 13, wherein the detection unit detects a hand. The display unit displays a plurality of the images,
The display device according to any one of claims 1 to 14, wherein the control unit changes a position where at least one of the plurality of images is displayed. A process of displaying an image on a display surface and displaying an image based on the image at a position different from the display surface;
Processing to detect objects;
A process of changing the position of pre-Symbol image is displayed based on the size of the portion where the image is blocked by said detected object,
A program that causes a computer to execute. A method of displaying an image on a display surface and displaying an image based on the image at a position different from the display surface;
A method of detecting an object;
Display method comprising, and methods for changing the position of pre-Symbol image is displayed based on the size of the portion where the image is blocked by said detected object.

Images