JP5896445B2 - Display device, display method, and program - Google Patents

Description

  The present invention relates to a display device that displays an image, a display method, and a program.

  Conventionally, there are various techniques as a technique for causing a user (viewer) to stereoscopically view a 3D (three-dimensional) image such as an image (still image, moving image) on a flat display unit, For example, there is a technique for applying a visual effect that makes an object in a two-dimensional (2D) image appear three-dimensionally. The technique using polygons is an example of this technique. There is also a technology that uses the parallax of the right and left eyes of the viewer. That is, when a right-eye image and a left-eye image that are slightly shifted from each other are prepared and the two images are displayed at the same time, the right-eye image is visible to the right eye and not to the left eye. Technology that makes the image look three-dimensional by placing an electronic parallax barrier (switching liquid crystal panel) that blocks the light path in an appropriate position so that the image can be seen by the left eye but not the right eye However, in addition to the need for an expensive liquid crystal display device for 3D, there is a problem that the viewing angle is limited and it is very narrow.

  In addition, as a technique that enables stereoscopic viewing of an image without using the 3D-dedicated display device as described above, for example, the rotation angle of the housing around each of the X, Y, and Z axes A technique (game device) is known in which an image corresponding to each rotation angle is displayed in a three-dimensional manner (see Patent Document 1).

JP 2002-298160 A

  However, in the related technique (the technique of Patent Document 1) described above, an image in the display unit is changed by an operation in which the operator tilts the casing. This technology is based on the premise of moving the camera. In addition, it is impossible to figure out how many times and in what direction the image will change if the housing is tilted. It will take a considerable amount of time to get used to it. Furthermore, even if you look at the displayed image from any direction, such as up, down, left, or right, as if you were looking into a real object by changing the line of sight, Without moving it, the image could not be changed.

  The subject of this invention is enabling it to display the image according to the gaze direction of the viewer who looks at a screen.

In order to solve the above-described problem, one aspect of the present invention is as follows.
Display means for displaying an image;
Imaging means for photographing a viewer facing the display means;
Based on the position of the viewer in the captured image by the imaging means, and the line-of-sight direction determining means for determining a viewing direction of the viewer to see the screen and facing said display means,
The gaze direction of the viewer is corrected based on the deviation between the optical axis position of the imaging means and the center position of the display means, and image data of a three-dimensional model is obtained based on the corrected gaze direction of the viewer. An image generating means for generating an image in a state as seen from the viewing direction by rotating ;
Display control means for causing the display means to display an image generated by the image generation means;
A display device comprising:

In order to solve the above-described problem, another aspect of the present invention provides:
Against the computer,
A function of capturing a viewer who views the screen in a face-to-face manner on a display unit that displays an image by the imaging unit, and determining the line-of-sight direction of the viewer based on the captured image;
A function of correcting the visual line direction of the viewer based on the deviation between the optical axis position of the imaging means and the center position of the display means ;
A function of generating an image in a state as seen from the gaze direction by rotating the image data of the three-dimensional model based on the gaze direction of the corrected viewer;
A function of displaying the generated image on the display means;
A program to realize

  According to the present invention, it is possible to display an image according to the line-of-sight direction of a viewer viewing the screen, and to realize an image display rich in reality without using a special 3D display device. It becomes possible.

The block diagram which showed the fundamental component of the mobile telephone applied as a display apparatus. FIG. (1) is a diagram showing image data (three-dimensional model image) of a three-dimensional model, and (2) shows a case where the three-dimensional model image displayed on the display unit 6 is viewed from the horizontal direction. Figure. (1) shows a case where the three-dimensional model image displayed on the display unit 6 is viewed from an obliquely upward direction (in the figure, obliquely upward 30 ° direction). (2) is an obliquely downward direction (in the figure, The figure which showed the case where it sees from diagonally downward 30 degrees direction. The flowchart which showed the gaze display control process which changes a display image according to a viewer's gaze direction. The flowchart which showed the operation | movement following FIG. The figure which showed the case where the position of a viewer's eyes is "deviation" in the up-down direction (vertical direction) with respect to the optical axis (horizontal) direction of the in-camera 8. FIG. The figure which showed the case where the optical axis of the in-camera 8 and the screen center position of the display part 6 have shifted | deviated. (1) is a diagram showing coordinate values and angles that change in accordance with the positional relationship between the viewer and the in-camera 8, and (2) is a diagram showing coordinate values in a captured image obtained by photographing the viewer. . The figure for demonstrating 2nd Embodiment. The functional block diagram for demonstrating the function of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)
First, a first embodiment of the present invention will be described with reference to FIGS.
This embodiment exemplifies a case where the present invention is applied to a mobile phone as a display device, and FIG. 1 is a block diagram showing basic components of the mobile phone.
This mobile phone has a voice call function for sending and receiving voice data and calling, and a TV (telephone) phone function for sending and receiving real-time images (partner image and self-image) in addition to voice. A telephone call can be made between the two. Furthermore, the mobile phone is provided with a camera function, an e-mail function, an Internet connection function, and the like.

  The central control unit 1 operates by supplying power from a power supply unit 2 having a secondary battery, and controls the overall operation of the mobile phone according to various programs in the storage unit 3. have. The storage unit 3 includes a program storage unit M1, an image storage unit M2, and the like. Note that the storage unit 3 is not limited to a built-in storage unit, and may be configured to include a removable portable memory (recording medium) such as an SD card or an IC card. The upper storage area may be included. The program storage unit M1 stores a program and various applications for realizing the present embodiment in accordance with the operation procedure shown in FIGS. 5 and 6, and stores information necessary for the program. Yes. The image storage unit M2 stores images taken by the camera function, images downloaded from the Internet, and the like.

  The wireless communication unit 4 transmits and receives data to and from the nearest base station during a call function, an e-mail function, an Internet connection function, and the like. During the call function operation, a signal is received from the receiving side of the baseband unit. Is demodulated into a reception baseband signal, and then voice is output from the speaker SP for call via the voice signal processing unit 15, and input voice data from the microphone MC for call is received from the voice signal processing unit 5 and transmitted. After encoding into a baseband signal, it is given to the transmission side of the baseband part and transmitted from the antenna. The display unit 6 displays character information, a standby image, an image, and the like, for example, with a high-definition liquid crystal or an organic EL (Electro Luminescence) display.

  The operation unit 7 inputs dial input, character input, command input, and the like. The central control unit 1 executes processing according to an input operation signal from the operation unit 7. The in-camera 8 is disposed on the surface side of the casing constituting the mobile phone, and is a camera imaging unit for a TV phone that photographs a user's (viewer) 's own face, etc. It is a camera imaging part which is arrange | positioned at the back surface side of a housing and image | photographs the external world. The in-camera 8 and the out-camera 9 include a photographing lens, a lens / mirror block such as a mirror, an image sensor, and a driving system thereof, a distance measuring sensor, a light amount sensor, an analog processing circuit, a signal processing circuit, a compression / expansion circuit, and the like. Adjusting and controlling the optical zoom and controlling the driving of the autofocus function.

FIG. 2 is an external perspective view of the mobile phone.
The cellular phone has two housings (an operation unit housing 11 and a display unit housing 12) attached to be openable and closable (foldable), and the two housings 11 and 12 are opened (open style). ), The operation unit 7 is disposed on the surface side of the operation unit housing 11, the display unit 6 is disposed on the surface side of the display unit housing 12, and the in-camera 8 is disposed in the vicinity thereof. It is installed. In this case, as shown in the drawing, the in-camera 8 is disposed in the vicinity of one end side in the longitudinal direction (upper side in the figure) in a state where the display unit 6 is oriented in a portrait screen with a portrait orientation. The face of the viewer (user) who faces the unit 6 and looks at the screen is photographed.

  FIG. 3A is a diagram illustrating image data (three-dimensional model image) of a three-dimensional model. In the illustrated example, a rectangular parallelepiped figure is shown, but a character or the like may be used. FIG. 3B shows a case where the three-dimensional model image displayed on the screen of the display unit 6 is viewed from the horizontal direction, and the three-dimensional model image is displayed in a state where only the front portion is visible.

  On the other hand, FIG. 4A shows a case where the three-dimensional model image displayed on the screen of the display unit 6 is viewed from an obliquely upward direction (an obliquely upward 30 ° direction in the figure). The image is displayed in a state where the front portion and the upper surface portion are visible. FIG. 4 (2) shows a case of viewing from an obliquely downward direction (in the figure, an obliquely downward 30 ° direction), and the three-dimensional model image is displayed in a state in which the front part and the bottom part are visible. As described above, in the present embodiment, the appearance of the three-dimensional model image is changed according to the line-of-sight direction of the viewer who faces the display unit 6 and views the screen.

  The central control unit 1 performs a process of changing the display image according to a predetermined user operation in a state where the two casings 11 and 12 are opened (open style), that is, according to the viewing direction of the viewer ( When a visual line display control process is instructed by a user operation, a viewer who views the display unit 6 is photographed by the in-camera 8, and the captured image is analyzed to identify the position of the viewer's eyes. ing. Then, the line-of-sight direction is determined according to the position of the eyes, and an image as seen from the line-of-sight direction is generated and displayed on the display unit 6. That is, when the gaze direction is changed to look into the displayed image from any direction such as up, down, left, and right, similar to the act of looking into an actual object while changing the gaze direction in real space, An image in a state as seen from the changed line-of-sight direction is generated and displayed.

  Next, the operation concept of the mobile phone in the first embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. 5 and 6 are flowcharts showing the characteristic operation of the first embodiment of the overall operation of the mobile phone. When the flow of FIG. 5 and FIG. Return to (not shown).

5 and 6 are flowcharts showing a line-of-sight display control process for changing a display image according to the line-of-sight direction of the viewer.
First, the central control unit 1 reads an image to be displayed (three-dimensional model image) from the image storage unit M2 or the like (step S1), and drives the in-camera 8 to perform forward shooting (step S2). In this case, a viewer viewing the screen facing the display unit 6 is photographed by the in-camera 8. Then, the central control unit 1 performs image recognition that identifies the position of the viewer's eyes by analyzing the captured image (step S3). It should be noted that the face and eye positions are recognized by comprehensively judging the face outline and the shape and positional relationship of the baht (eyes, mouth, nose, forehead, etc.) forming the face while analyzing the captured image. However, since this image recognition function is a technique that is generally used in cameras, and this embodiment uses the well-known technique, a detailed description thereof will be omitted. Shall.

  Next, the process of determining the viewing direction of the viewer and converting the position of the eyes into coordinate values in the same space as the three-dimensional model image is executed for the Y axis and the X axis, respectively. First, in the flow of FIG. 5 and FIG. 6, the process for the Y axis (steps S4 to S9) is performed, and then the process for the X axis (steps S10 to S14 in FIG. 6) is performed. The processing (steps S4 to S9) and the processing for the X axis (steps S10 to S14 in FIG. 6) are basically the same processing.

  FIG. 7 is a diagram for explaining the processing with respect to the Y axis, and shows the case where the position of the viewer's eyes is “shifted” in the vertical direction (vertical direction) with respect to the optical axis (horizontal) direction of the in-camera 8. In the figure, when the direction perpendicular to the screen of the display unit 6 (the optical axis direction of the in-camera 8) is the Z axis (third axis) of the three-dimensional coordinate system, the Y axis (second axis) It is shown that the angle θ of the upper eye position with respect to the Z-axis is the viewing direction of the viewer. Although the X axis of the three-dimensional coordinate system is the first axis, the Y axis is the second axis, and the Z axis is the third axis, the relationship is not limited to these (the same applies hereinafter). FIG. 7A is a diagram showing coordinate values y and z and angles θ and θmax that change in accordance with the positional relationship between the viewer and the in-camera 8, and FIG. 7B is a photograph of the viewer. It is the figure which showed the coordinate values y and ymax within the picked-up image. Although not shown in FIG. 7, the photographing surface (imaging device) of the in-camera 8 is made to coincide with the display surface of the display unit 6 and be in the same vertical plane.

First, when starting the process for the Y axis, the central control unit 11 calculates y / ymax (step S4). Here, y is a coordinate value on the Y axis corresponding to the eye position, and ymax is a coordinate value on the Y axis corresponding to the angle of view (θmax) of the in-camera 8. In this case, the specific numerical values of y and ymax are unknown, but the ratio of y and ymax (y / ymax) can be treated as a known value, so this known value and the fixed value θmax of the in-camera 8 Are used to calculate the viewing direction (angle of eye position) θ and tan θ of the viewer according to the following equation (step S5).
tan θ = (y / ymax) tan (θmax)
Note that θ itself is obtained by an arc tangent or the like.

Next, the distance z on the Z axis from the in-camera 8 to the viewer's face is acquired (step S6). In this case, in the present embodiment, the distance z is obtained by using the autofocus function of the in-camera 8, but this function is a well-known technique that is generally used in the camera, and therefore the description thereof will be given. Omitted. Note that the distance z from the in-camera 8 to the face of the viewer may be calculated approximately from the distance between the left eye and the right eye in the captured image. Further, the distance z from the in-camera 8 to the viewer's face may be calculated approximately from the size of the face in the captured image. Furthermore, the distance z may be determined as a value arbitrarily set by a user operation, for example, 30 cm. When the distance z in the real space from the in-camera 8 to the viewer's face is thus determined, the distance z is used to determine the eye position y of the viewer in the real space according to the following equation. Calculate (step S7).
y = z * tan θ

  FIG. 8 is a diagram illustrating a case where the optical axis of the in-camera 8 and the screen center position of the display unit 6 are “deviation”. In this way, when the optical axis of the in-camera 8 is “deviation” with respect to the screen center position of the display unit 6, if the value of the deviation amount is “y deviation”, the viewer's eye position y is Correction (addition) is made by “y deviation” (step S8). After obtaining the eye position coordinates y and z in the real space in this way, the position coordinates y and z are converted to coordinate values in the same space as the three-dimensional model image (step S9). For example, the coordinate values y and z may be multiplied by the ratio determined by the developer at the time of creating (designing) the 3D model image. When the screen center position of the display unit 6 is the origin of the three-dimensional coordinate system, for example, the z-coordinate value when the three-dimensional model image is arranged so as to be seen 1 cm behind the screen is determined to be minus one. In other words, the values of y and z (unit: cm) may be obtained in consideration of the depth (1 cm) of the image.

  After obtaining the coordinate values y and z in the same space as the three-dimensional model image by the process for the Y axis (steps S4 to S9) in this way, the process for the X axis (steps S10 to S18) is executed. FIG. 9 illustrates a case where the position of the viewer's eyes is “deviation” in the left-right direction (horizontal direction) with respect to the optical axis (horizontal) direction of the in-camera 8. When the vertical direction (the optical axis direction of the in-camera 8) is the Z-axis of the three-dimensional coordinate system, the angle θ of the eye position on the X-axis with respect to the Z-axis is the viewing direction of the viewer. ing. FIG. 9 (1) is a diagram showing coordinate values x and z and angles θ and θmax that change in accordance with the positional relationship between the viewer and the in-camera 8, and FIG. It is the figure which showed coordinate value x and xmax in the picked-up image. In this case as well, the photographing surface (imaging device) of the in-camera 8 is in the same vertical plane with the display surface of the display unit 6 being coincident.

First, the central control unit 11 calculates x / xmax (step S10). Here, x is a coordinate value on the X-axis corresponding to the eye position, and xmax is a coordinate value on the X-axis corresponding to the angle of view (θmax) of the in-camera 8. Then, the viewing direction (angle of the eye position) θ and tan θ of the viewer is calculated according to the following equation (step S11).
tan θ = (x / xmax) tan (θmax)
Note that θ itself is obtained by an arc tangent or the like.

Next, based on the distance z on the Z axis from the in-camera 8 to the viewer's face acquired in step S6 described above, the viewer's eye position x in real space is calculated according to the following equation ( Step S12).
x = z * tan θ
When the optical axis of the in-camera 8 is “deviation” with respect to the screen center position of the display unit 6, if the deviation amount is “x deviation”, the eye position x of the viewer is set to “x deviation”. Correction (addition) is performed (step S13). After obtaining the eye position coordinates x and z in the real space in this way, the position coordinates x and z are converted into coordinate values in the same space as the three-dimensional model image (step S14).

  After obtaining the coordinate values (eye positions) x, y, z in the same space as the 3D model image in this way, the 3D model image is viewed from this eye position. In this manner, the three-dimensional model image is rotated on the three-dimensional coordinate system, and the rotated image is displayed on the display unit 6 (step S15). Thereafter, it is checked whether or not an instruction to switch images has been issued (step S16), or whether or not an instruction to end the line-of-sight display control has been given (step S18). Now, for example, when a switching operation by a user (viewer) operation is performed or when an elapse of a predetermined time is detected during slideshow display (YES in step S16), an image to be displayed (three-dimensional model image) After selecting (step S17), the process returns to step S1 in FIG. Further, when the end operation by the user (viewer) operation is performed or when the slide show end time has been detected at the time of slide show display (YES in step S18), the flow of FIG. 5 and FIG. 6 is exited.

  As described above, in the first embodiment, an image in a state as seen from the viewing direction of a viewer who faces the display unit 6 and looks at the screen is generated and displayed. An image corresponding to the direction can be displayed, and even if a special display device for 3D is not used, a viewer can see an image like a 3D display, and the image display is rich in reality. Can be realized.

  Also, when the gaze direction on the screen is changed so that you can look into the displayed image from any direction, such as up, down, left, or right, in the same way as the action of looking at an actual object in real space while changing the gaze direction In addition, since the image is changed to a state as seen from the direction of the line of sight, for example, when a certain character is displayed on the entire display unit 6, the character is displayed from below. If the viewer approaches the screen and wants to look into the screen and takes action to look at the character from below, the image can be changed to a state of looking into from below.

  Since the visual line direction of the viewer is determined based on the image captured by the in-camera 8 that captures the viewer facing the display unit 6, the visual line direction of the viewer is reliably and easily determined by image recognition. be able to.

  When the position of the viewer's eyes is specified by analyzing the image captured by the in-camera 8 and the direction perpendicular to the screen of the display unit 6 is the Z-axis of the three-dimensional coordinate system, the Y-axis The angle of the upper eye position with respect to the Z axis and the angle of the eye position on the X axis with respect to the Z axis are determined as the viewing direction of the viewer, and the image data of the three-dimensional model is rotated based on the viewing direction. As a result, an image in a state as seen from the line-of-sight direction is generated. Therefore, an image in a state as seen from the line-of-sight direction can be obtained simply by rotating the image data of the three-dimensional model. Can do.

  Further, the position of the viewer's eyes is specified based on the viewing direction of the viewer and the distance from the display unit 6 to the viewer, and an image in a state where the object is viewed from the position of the eyes is generated. Therefore, it is not necessary for the viewer to keep the image viewing distance constant, and even if viewed from a distance or near, an image is displayed as if the object is viewed from that position. There is no need to be aware of the distance from the section 6.

(Embodiment 2)
A second embodiment of the present invention will be described below with reference to FIG.
In the first embodiment described above, the angles θ and tan θ from the screen of the display unit 6 to the position of the viewer's eyes are obtained, but in the second embodiment, the display unit 6 In consideration of the depth from the screen to the position of the display image for each image, the angles φ and tan φ from each image having a depth to the eye position of the viewer are obtained. Here, in both the embodiments, the same or the same names are denoted by the same reference numerals, the description thereof will be omitted, and the following description will focus on the features of the second embodiment. .

  FIG. 10 is a diagram for explaining an angle θ from the screen of the display unit 6 to the position of the viewer's eyes and an angle φ from the image having a depth to the position of the viewer's eyes. Here, when the position of the eyes of the viewer is “deviation” in the vertical direction (vertical direction) with respect to the optical axis (horizontal) direction of the in-camera 8, the direction perpendicular to the screen of the display unit 6 ( The optical axis direction of the in-camera 8) is the Z axis of the three-dimensional coordinate system, and the angle of the eye position on the Y axis with respect to the Z axis is the viewing direction of the viewer. Then, after obtaining the angle θ from the screen of the display unit 6 to the position of the viewer's eyes, the angle φ from the image having the depth to the position of the viewer's eyes is obtained. Here, in the figure, A indicates the depth (known value) from the screen of the display unit 6 to the image.

That is,
tan θ = y / z (1)
tan φ = y / (z + A) (2)
Since y is common to both equations (1) and (2), ztanθ = (z + A) * tanφ.
Therefore, tan φ = (z / (z + A)) * tan θ

  Here, since z and θ are obtained in the same manner as in the first embodiment described above, an angle φ from an image having a depth to the eye position of the viewer is obtained. At that time, when a plurality of images with depth are displayed from the screen of the display unit 6, the depth from each image to the position of the eye of the viewer is taken into consideration for each image. The angle φ is obtained. Note that FIG. 10 shows a case where the angle of the eye position on the Y axis with respect to the Z axis is obtained, but the angle of the eye position on the X axis with respect to the Z axis can be basically obtained in the same manner.

  As described above, in the second embodiment, considering the depth from the screen of the display unit 6 to the image, the angle from the image having the depth to the position of the viewer's eyes is set as the line-of-sight direction. It is possible to determine the angle (gaze direction) even for images with the same angle as in the act of looking at an actual object in real space while changing the gaze direction. It is possible to change to an image that will be in a correct state.

  Further, when displaying a plurality of images having depth from the screen of the display unit 6, the angle from each image having the depth to the position of the eye of the viewer is considered for each image. Therefore, the appearance changes for each image, and more realistic image display is possible.

  In each of the above-described embodiments, the angle of the eye position on the Y axis with respect to the Z axis is obtained, and the angle of the eye position on the X axis with respect to the Z axis is obtained. You may make it ask. In other words, it may be possible to look only in the vertical direction or to look only in the horizontal direction.

  Further, when the angle θ, φ of the eye position reaches a predetermined angle or more (for example, 70 ° or more), an image on the back side of the cuboid is generated and displayed, or the contents of the cuboid figure, for example, the cuboid is displayed. In the case of a house appearance model, an image inside the house may be generated and displayed.

  Further, in each of the above-described embodiments, the viewer's line-of-sight direction is specified with reference to the in-camera 8, and the three-dimensional model image is viewed from the position of the eyes. The model image is rotated and displayed. For example, in the state where the virtual camera is set to the eye position y and z of the viewer and the visual field of the virtual camera is directed to the screen of the display unit 6, the 3D technology A rendering display (for example, OpenGL / Direct3D) may be used to generate the image while calculating how the object looks.

  In each of the above-described embodiments, the case where the present invention is applied to a foldable mobile phone has been described. However, the present invention may be applied to a two-axis type mobile phone, and the type is not limited. Moreover, you may make it image | photograph a viewer using not only the in camera 8 but a camera (for example, external apparatus) different from a mobile telephone.

  Moreover, in each embodiment mentioned above, although the case where a viewer was image | photographed with the in-camera 8 and the direction of the line of sight was specified was illustrated, for example, an angular velocity sensor, an acceleration sensor, etc. are used, for example. You may make it generate | occur | produce the image of the state which is calculated | required from the direction by calculating | requiring a rotation angle.

  In each of the above-described embodiments, the case where the display device is applied to a mobile phone has been described. However, the display device may be a mobile terminal device such as a digital camera (compact camera), a PDA, a music player, or a game machine. The present invention is not limited to the terminal device, and can be similarly applied to a television receiver, a personal computer (for example, a notebook PC, a tablet PC, and a desktop PC).

  In addition, the “apparatus” and “unit” shown in the first and second embodiments described above may be separated into a plurality of cases by function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

A part or all of each of the above embodiments can be described as the following supplementary notes, but is not limited thereto.
Hereinafter, various aspects of the present invention will be collectively described as supplementary notes.
(Appendix 1)
FIG. 11 is a configuration diagram of the appendix 1 (functional block diagram of the present invention).
As shown in this figure, the invention described in appendix 1
Means 100 for displaying an image (display unit 6 in FIG. 1);
A line-of-sight direction determining unit 101 (in FIG. 1, the central control unit 1, the in-camera 8, and the program storage unit M2) that determines the line-of-sight direction of a viewer who faces the display unit 100 and views the screen;
An image generation unit 102 (in FIG. 1, the central control unit 1, the program storage unit M2, and the image storage unit M2) that generates an image in a state as seen from the line-of-sight direction determined by the line-of-sight direction determination unit 101;
Display control means 103 (in FIG. 1, the central control unit 1, the program storage unit M2, and the display unit 6) for displaying the image generated by the image generation unit 102 on the display unit 100;
A display device comprising:
According to Supplementary Note 1, since an image in a state as seen from the viewing direction of a viewer who faces the display unit 100 and looks at the screen is generated and displayed, an image corresponding to the viewing direction is displayed. Even if an expensive liquid crystal display device for 3D is not used, a viewer can see an image like a 3D display and can realize an image display rich in reality. It becomes possible.

(Appendix 2)
The image generation unit generates an image in a state as seen from the changed line-of-sight direction when the line-of-sight direction with respect to the displayed image is changed.
The display device according to attachment 1, wherein the display device is configured as described above.

(Appendix 3)
It further comprises imaging means for photographing the viewer facing the display means,
The line-of-sight direction determining means determines the line-of-sight direction of the viewer based on the image taken by the imaging means.
The display device according to appendix 1 or appendix 2, wherein the display device is configured as described above.

(Appendix 4)
The line-of-sight direction determining means identifies the position of the viewer's eyes by analyzing the image taken by the imaging means, and sets the direction perpendicular to the screen of the display means to the third in the three-dimensional coordinate system. In the case of an axis, the angle of the eye position on the second axis with respect to the third axis is determined as the viewing direction of the viewer.
The display device according to attachment 3, wherein the display device is configured as described above.

(Appendix 5)
The line-of-sight direction determining means identifies the position of the viewer's eyes by analyzing the image taken by the imaging means, and sets the direction perpendicular to the screen of the display means to the third in the three-dimensional coordinate system. In the case of an axis, the angle of the eye position on the first axis with respect to the third axis is determined as the viewing direction of the viewer.
The display device according to appendix 3 or appendix 3, wherein the display device is configured as described above.
(Appendix 6)
The image generation unit generates an image in a state as seen from the line-of-sight direction by rotating the image data of the three-dimensional model based on the line-of-sight direction determined by the line-of-sight direction determination unit.
The display device according to any one of appendix 1 to appendix 5, wherein the display device is configured as described above.

(Appendix 7)
Identifying means for identifying the position of the viewer's eyes based on the viewing direction of the viewer determined by the viewing direction determining means and the distance from the display means to the viewer;
Further comprising
The image generating means generates an image in a state as seen from the position of the eye of the viewer specified by the specifying means;
The display device according to any one of supplementary notes 1 to 6, wherein the display device is configured as described above.

(Appendix 8)
Determine the visual line direction of the viewer in consideration of the depth of the image from the screen of the visual line direction determination means and the display means,
The image generating means generates an image in a state as seen from the line-of-sight direction by the line-of-sight direction determining means;
The display device according to any one of appendix 1 to appendix 7, wherein the display device is configured as described above.

(Appendix 9)
The line-of-sight direction determining means determines the line-of-sight direction of the viewer for each image in consideration of the depth from the screen for each image when a plurality of images are displayed on the display means,
The image generation unit generates an image for each image as seen from the line-of-sight direction determined for each image by the line-of-sight direction determination unit.
The display device according to appendix 8, which is configured as described above.

(Appendix 10)
Determine the line-of-sight direction of the viewer who sees the screen facing the display means for displaying the image,
Generate an image as seen from the determined gaze direction,
Displaying the generated image on the display means;
A display method characterized by that.

(Appendix 11)
Against the computer,
A function for determining the line-of-sight direction of the viewer who sees the screen facing the display means for displaying an image;
A function of generating an image in a state as seen from the determined gaze direction;
A function of displaying the generated image on the display means;
A program to realize
According to Supplementary Note 10, the same effect as Supplementary Note 1 can be obtained, and the function of Supplementary Note 1 can be provided in the form of software (program).

DESCRIPTION OF SYMBOLS 1 Central control part 3 Memory | storage part 6 Display part 7 Operation part 8 In-camera 11 Operation part housing | casing 12 Display part housing | casing M1 Program storage part M2 Image storage part

Claims (9)

Display means for displaying an image;
Imaging means for photographing a viewer facing the display means;
Based on the position of the viewer in the captured image by the imaging means, and the line-of-sight direction determining means for determining a viewing direction of the viewer to see the screen and facing said display means,
The gaze direction of the viewer is corrected based on the deviation between the optical axis position of the imaging means and the center position of the display means, and image data of a three-dimensional model is obtained based on the corrected gaze direction of the viewer. An image generating means for generating an image in a state as seen from the viewing direction by rotating ;
Display control means for causing the display means to display an image generated by the image generation means;
A display device comprising: When the viewer's line-of-sight direction is changed, the image generating means rotates the image data of the three-dimensional model according to the changed line-of-sight direction so that the viewer can see the changed line-of-sight direction. To create an image that looks like
The display device according to claim 1, which is configured as described above. The line-of-sight direction determining means identifies the position of the viewer's eyes by analyzing the image taken by the imaging means, and sets the direction perpendicular to the screen of the display means to the third in the three-dimensional coordinate system. In the case of an axis, the angle of the eye position on the second axis with respect to the third axis is determined as the visual line direction of the viewer.
The display device according to claim 1 , wherein the display device is configured as described above. The line-of-sight direction determining means identifies the position of the viewer's eyes by analyzing the image taken by the imaging means, and sets the direction perpendicular to the screen of the display means to the third in the three-dimensional coordinate system. In the case of an axis, the angle of the eye position on the first axis with respect to the third axis is determined as the visual line direction of the viewer.
The display device according to claim 1 , wherein the display device is configured as described above. Identifying means for identifying the position of the viewer's eyes based on the viewing direction of the viewer determined by the viewing direction determining means and the distance from the display means to the viewer;
Further comprising
The image generation means corrects the position of the eye of the viewer specified by the specifying means based on a deviation between the optical axis position of the imaging means and the center position of the display means, and the corrected visual recognition By rotating the image data of the three-dimensional model based on the position of the eyes of the person, an image in a state as seen from the position of the eyes is generated.
Display device according to any one of claims 1 to 4, characterized in that the the like. The line-of-sight direction determining means determines the line-of-sight direction of the viewer in consideration of the depth of the image from the screen of the display means;
The image generation unit corrects the visual line direction of the viewer determined by the visual line direction determination unit based on a deviation between the optical axis position of the imaging unit and the center position of the display unit, and the corrected By rotating the image data of the three-dimensional model based on the visual line direction of the viewer, an image in a state as seen from the visual line direction is generated.
Display device according to any one of claims 1 to 5, characterized in that the the like. The line-of-sight direction determining means determines the line-of-sight direction of the viewer for each image in consideration of the depth from the screen for each image when a plurality of images are displayed on the display means,
The image generation unit corrects the visual line direction of the viewer determined for each image by the visual line direction determination unit based on a deviation between the optical axis position of the imaging unit and the center position of the display unit, and the correction By rotating the image data of the three-dimensional model based on the visual line direction of the viewer that has been generated, for each image to generate an image as seen from the visual line direction ,
The display device according to claim 6, which is configured as described above. The viewer who faces the display unit that displays the image and views the screen is imaged by the imaging unit, and the line-of-sight direction of the viewer is determined based on the captured image.
Correcting the visual line direction of the viewer based on the deviation of the optical axis position of the imaging means and the center position of the display means ;
By rotating the image data of the three-dimensional model based on the corrected gaze direction of the viewer, an image in a state as seen from the gaze direction is generated,
Displaying the generated image on the display means;
A display method characterized by that. Against the computer,
A function of capturing a viewer who views the screen in a face-to-face manner on a display unit that displays an image by the imaging unit, and determining the line-of-sight direction of the viewer based on the captured image;
A function of correcting the visual line direction of the viewer based on the deviation between the optical axis position of the imaging means and the center position of the display means ;
A function of generating an image in a state as seen from the gaze direction by rotating the image data of the three-dimensional model based on the gaze direction of the corrected viewer;
A function of displaying the generated image on the display means;
A program to realize

Images