JP2005077808A - Image display device, image display method, and image display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a user to easily fetch the image of a certain portion of a wide-viewing angle image, and to view the image. <P>SOLUTION: The image display device has a wide-viewing angle image output means 2 which can output the wide-viewing angle image picked up with an omnibearing camera 22; a display device 11 the position of which can be changed so as to match a user's line-of-sight direction; a position-detecting means 12 which detects the position of the display device 11 and outputs the result of the detection as the positional information; and a specific line-of-sight direction image generating means 3 which receives a positional information report from the position-detecting means 12, receives the wide-viewing angle image sent from the wide-viewing angle image output means 2 and generates the specific line-of-sight direction image, corresponding to the positional information from the wide-viewing angle image and the positional information. The image generated by the specific line-of-sight direction image generating means 3 is displayed on the display device 11. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image display device, an image display method, and an image display system that display an image according to the position of a display device.

  In a direct-view type display device having a display screen such as a CRT (Cathode Ray Tube) or a liquid crystal panel that is very commonly used, even if the position and orientation of the display device itself are changed, Regardless of the position and orientation of the display device, the transmitted image is displayed as it is.

  For example, when a face image viewed from the front of a person on the display screen of the display device is displayed, the displayed face image is viewed from the front even if the orientation of the display device is changed by 90 degrees horizontally, for example. Naturally, the face image is displayed as it is, and the side face is not displayed. Similarly, even if the display device is shifted to the left in order to center the image displayed near the left edge of the display screen in the display screen, the displayed image is displayed in the right direction on the display screen. Of course, it doesn't move to.

  On the other hand, there has conventionally been a display device that can change a rectangular display screen horizontally (horizontal direction) and vertically (vertically). Some display devices of this type can automatically determine whether the display screen is installed horizontally or vertically and switch the display mode according to the orientation of the display screen. .

  For example, “video display device” (referred to as prior art 1) described in Japanese Patent Laid-Open No. 2000-148122 is an example. This prior art 1 is, for example, when displaying images such as characters and photos displayed on a horizontally oriented display screen in a portrait orientation, when the display screen is rotated 90 degrees to a portrait orientation, Even if the display screen is rotated 90 degrees, the characters and pictures displayed on it are not rotated 90 degrees with the display screen and turned sideways, so that the top and bottom direction is displayed correctly. .

  Moreover, as an example of a technique for detecting the direction of the line of sight of a user viewing a display screen and displaying an image corresponding thereto, there is a “display system” (referred to as Prior Art 2) described in JP-A-11-27608.

In this prior art 2, a user wears a kind of head-mounted display called a line-of-sight input scouter like glasses, and projects an image according to line-of-sight position data obtained from the line-of-sight input scouter on a screen.
JP 2000-148122 A Japanese Patent Laid-Open No. 11-27608

  Prior art 1 described above detects whether the display screen is in landscape orientation or portrait orientation, and according to the detected orientation of either “vertical” or “horizontal”, characters, photos, etc. indicate the vertical direction. It is only displayed correctly.

  On the other hand, in the related art 2, an image corresponding to the user's line-of-sight direction is projected. However, in this system, an image linked to the line of sight always jumps into the eyes of the user. That is, since the user must see an image corresponding to the line-of-sight direction no matter where he / she looks, there is a problem that gives the user a feeling of fatigue and burden.

  Also, this type of projecting an image according to the user's line of sight has a reasonable effect for viewing only the image, but it can be used to compare it with materials at hand or other screens. There is also a problem that it is difficult.

  Therefore, the present invention provides an image display device, an image display method, and an image display system that can display an image according to the position of the display device itself by setting the position of the display device according to the user's line-of-sight direction. It is aimed.

  (1) The image display device of the present invention is an image display device capable of displaying an image when viewed from a viewing direction with a wide viewing angle image, and the position can be changed to match the viewing direction of the user. A display device; and position detection means for detecting the position of the display device and outputting the detection result as position information. The position information of the display device is output from the position detection means. The display device acquires a specific gaze direction image corresponding to the position information generated from the wide viewing angle image, and displays the acquired specific gaze direction image on the display device.

  As described above, the image display device of the present invention can display a specific line-of-sight image according to the position by setting the position of the display device itself in accordance with the line-of-sight direction of the user.

  As a result, when an image to be processed is viewed from a certain gaze direction, the image in the gaze direction can be displayed simply by moving the display device so that the desired image is displayed. It is possible to display an image of a position that the user wants to see (a specific line-of-sight direction image) on the display device with a very simple operation.

  Further, according to the present invention, when a specific gaze direction image is displayed on the display device, if the display device is left as it is, the position information of the display device remains fixed, so the image at that position is displayed. It will be the same as watching a normal display.

  Therefore, unlike conventional systems that display images in conjunction with the line-of-sight direction, the user does not always have to look at the displayed image. Can be the same. In addition, this can be used for a wide range of applications because the display image can sometimes be confirmed while referring to other materials.

(2) In the image display device according to (1), the position information output from the position detection unit may be a position in a three-dimensional space of a display coordinate system in the display device.
Thereby, the position where the display device is placed and the orientation of the display screen can be appropriately expressed, and an appropriate specific line-of-sight direction image can be displayed.

(3) In the image display device according to (2), the position of the display coordinate system in the three-dimensional space is the coordinates of the reference point of the display coordinate system on the three-dimensional coordinate or the reference point. It can be expressed by polar coordinates, a normal vector at a certain point in the display coordinate system, and a rotation angle in a rotation direction with the normal vector as a central axis.
Thereby, the position where the display device is placed and the orientation of the display screen can be appropriately represented.

(4) In the image display device of (2), the position of the display coordinate system in the three-dimensional space is a coordinate on a three-dimensional coordinate of a plurality of points of the display coordinate system or a polar coordinate of the plurality of points. It can also be expressed.
Accordingly, the position where the display device is placed and the orientation of the display screen can be appropriately represented with a small amount of information.

(5) In the image display device of (4), the plurality of points may be at least three corner points of the display area of the display device.
This makes it possible to appropriately represent the position where the display device is placed and the direction of the display screen with a smaller amount of information.

(6) In the image display device according to any one of (1) to (5), an image input by an image input unit can be used as the wide viewing angle image.
As a result, a wide viewing angle image can be obtained simply by attaching a wide angle lens or fisheye lens to a general digital camera or the like, and thus a wide viewing angle image can be easily obtained.

(7) In the image display device according to any one of (1) to (5), a three-dimensional computer graphics image can be used as the wide viewing angle image.
Thus, by using a three-dimensional computer graphics image, it is possible to apply to computer games, CAD (Computer Aided Design), and the like.

(8) In the image display device according to any one of (1) to (5), the wide viewing angle image may be a mixture of an image input by an image input unit and a three-dimensional computer graphics image. .
Thus, for example, a specific line-of-sight image for an image obtained with a digital camera and a specific line-of-sight image of an image obtained with 3DCG can be displayed, so that the present invention can be applied in a wide range of fields.

(9) In the image display device according to (6) or (8), the specific line-of-sight direction image displayed on the display device is an image in a predetermined range cut out from the wide viewing angle image based on the position information. On the other hand, the image is corrected so as to be an image suitable for display on the display area of the display device.
Thus, for example, when a part of a wide viewing angle image photographed with a wide-angle lens or a fish-eye lens is displayed on the display device as a specific line-of-sight direction image, a natural image that corrects distortion and does not give the user a sense of incongruity. It can be displayed on a display device.

(10) In the image display device according to (6) or (7), the specific line-of-sight direction image displayed on the display device is a line-of-sight direction from a certain viewpoint position of the three-dimensional computer graphics image based on the position information. The image obtained by calculation is determined.
As a result, simply moving the display device in the direction you want to see, you can display an image in that line-of-sight direction, providing a sense of reality in computer games, etc. It is possible to easily check and confirm the results when performing.

(11) In the image display device according to any one of (1) to (10), when the specific line-of-sight image is displayed on the display device, the wide viewing angle image is displayed on a part of the display area of the specific line-of-sight image. Can also be displayed.
In this way, by displaying a wide viewing angle image in a part of the display area displaying an image in a specific gaze direction, for example, it is possible to see a specific gaze direction image while checking the overall state. .

  (12) Further, the image display device of the present invention has a display device whose position can be changed to match the user's line-of-sight direction, and a position where the position of the display device is detected and the detection result is output as position information. And detecting the position information of the display device from the position detecting means, and the display device acquires a specific gaze direction image corresponding to the position information generated from the position information. The specific gaze direction image is displayed on the display device.

  This means that it is not always necessary to use a wide viewing angle image prepared from the beginning such as a wide viewing angle image taken by an omnidirectional camera or the like. As an example, it is also possible to create a virtual space on a computer using multiple images of people in different locations on the computer. It is also possible to create a specific line-of-sight direction image and display it on the display.

(13) In the image display device according to (12), the specific line-of-sight image displayed on the display device is calculated by determining a line-of-sight direction from a certain viewpoint position of the three-dimensional computer graphics image based on the position information. The obtained image can be obtained.
As a result, it is possible to easily create an image in a specific line-of-sight direction on a virtual space created on a computer.

Also in the image display device of (12), as in the image display device of (1), the position information output from the position detection means is within the three-dimensional space of the display coordinate system in the display device. Position.
Then, the position of the display coordinate system in the three-dimensional space is obtained by calculating the coordinates of the reference point of the display coordinate system on the three-dimensional coordinate or the polar coordinates of the reference point and a point in the display coordinate system. It can be expressed by a line vector and a rotation angle in the rotation direction with the normal vector as the central axis.
Similarly, the position in the three-dimensional space of the display coordinate system can also be expressed by coordinates on a three-dimensional coordinate of a plurality of points in the display coordinate system or polar coordinates of the plurality of points. Here, the plurality of points may be at least three corner points of the display area of the display device.

(14) The image display method of the present invention is an image display method capable of displaying an image when viewed from a line-of-sight direction with a wide viewing angle image on a display device, and detecting and outputting position information of the display device Then, using the position information and the wide viewing angle image, a specific gaze direction image corresponding to the position information is generated, the display device acquires the specific gaze direction image, and the acquired specific gaze direction image Is displayed.
As a result, the same effect as the above-described image display device can be obtained.

  (15) The image display system of the present invention is an image display system capable of displaying an image when viewed from a viewing direction with a wide viewing angle image, and has a wide viewing angle image output means capable of outputting a wide viewing angle image. A display device capable of changing the position so that it can be adjusted to the user's line of sight, position detection means for detecting the position of the display device and outputting the detection result as position information, and the position information A specification for receiving from the position detection means and receiving a wide viewing angle image generated by the wide viewing angle image output means and generating a specific gaze direction image corresponding to the position information from the position information and the wide viewing angle image And a line-of-sight image generation means.

  As a result, the same effect as the above-described image display device can be obtained. In addition, this image display system can be applied to a wide range of fields such as a teleconference system, various monitoring systems, computer games, CAD, and other architectural designs and mechanical designs.

Also in these image display method and image display system, the position information output from the position detection means is within the three-dimensional space of the display coordinate system in the display device, as in the image display device of (1). Position.
Then, the position of the display coordinate system in the three-dimensional space is obtained by calculating the coordinates of the reference point of the display coordinate system on the three-dimensional coordinate or the polar coordinates of the reference point and a point in the display coordinate system. It can be expressed by a line vector and a rotation angle in the rotation direction with the normal vector as the central axis.

  Further, the position of the display coordinate system in the three-dimensional space can be expressed by coordinates on the three-dimensional coordinates of a plurality of points in the display coordinate system or polar coordinates of the plurality of points. Here, the plurality of points may be at least three corner points of the display area of the display device.

The wide viewing angle image may be an image input by an image input means, and the wide viewing angle image may be a three-dimensional computer graphics image.
The wide viewing angle image may be a mixture of an image input by the image input means and a three-dimensional computer graphics image.

  The specific line-of-sight image displayed on the display device is suitable for displaying on a display area of the display device with respect to an image of a predetermined range cut out from the wide viewing angle image based on the position information. An image that has been corrected to be an image can be obtained.

  Further, the specific line-of-sight image displayed on the display device can be an image obtained by calculation by determining the line-of-sight direction from a certain viewpoint position of the three-dimensional computer graphics image based on the position information.

  In addition, when the specific gaze direction image is displayed on the display device, the wide viewing angle image may be displayed in a partial display area of the display area of the specific gaze direction image.

  Hereinafter, embodiments of the present invention will be described. The contents described in this embodiment include descriptions of the image display device, the image display method, and the image display system of the present invention.

  FIG. 1 is a block diagram illustrating an image display system according to the present invention, which includes an image display device 1, a wide viewing angle image output means 2, and a specific line-of-sight direction image generation means 3.

  The image display device 1 has a direct-view display screen, and a display device 11 whose position (which will be described later) can be changed so that it can be adjusted to the user's line-of-sight direction. And position detecting means 12 for detecting the position and outputting the detection result as position information.

  The wide viewing angle image output means 2 is capable of outputting a wide viewing angle image. As the wide viewing angle image used in this embodiment, for example, a wide viewing angle image acquired by an omnidirectional camera equipped with a fisheye lens, a wide angle lens, or the like is used. A viewing angle image, a three-dimensional computer graphics image (hereinafter referred to as a 3DCG image), and the like are used.

  As described above, the wide viewing angle image output means 2 outputs a wide viewing angle image captured by an omnidirectional camera equipped with a wide angle lens or a fisheye lens, or a three-dimensional image for generating a 3DCG image. In addition to outputting object information, images taken by a plurality of cameras may be output.

  The specific line-of-sight direction image generation means 3 receives the position information from the position detection means 12, and also receives the wide viewing angle image from the wide viewing angle image output means 2, and the position information is obtained from these position information and the wide viewing angle image. A specific line-of-sight image according to the above is generated.

  The direct-view display device 11 includes, for example, a liquid crystal display device, a CRT display device, etc. The size of the display screen 11a is not particularly limited. To those having a large display screen, such as a large television. However, it is required that the position can be easily changed in accordance with the user's line-of-sight direction. It is desirable that the user can easily change the position so that the user can change the position.

  The display device 11 used in this embodiment may be one in which the device main body and the display screen are integrated, or the device main body and the display screen may be separate.

  The position detection means 12 is, for example, an accelerometer, an ultrasonic sensor, a mechanical or optical position detection means, an angle sensor, and an image recognition process using a CCD camera, etc. Although it can be detected, a specific example thereof will be described later.

  As described above, the specific line-of-sight direction image generation unit 3 uses the position information of the display device 11 detected by the position detection unit 12 and the wide viewing angle image output from the wide viewing angle image output unit 2 to determine the position information. A specific line-of-sight image according to the above is generated.

  For example, if the wide viewing angle image output from the wide viewing angle image output means 2 is a wide viewing angle image from an omnidirectional camera, an image of a predetermined range is cut out from the wide viewing angle image based on position information. Alternatively, an image corresponding to the position and orientation of the display device 11 can be generated by performing image correction on the clipped image in the predetermined range.

  Also, if the wide viewing angle image is a 3DCG image, the viewpoint is determined, and by calculating what the image will look when a certain direction is viewed from the viewpoint, it is determined according to the position and orientation of the display device 11. An image can be generated.

  If the wide viewing angle image is image information from a plurality of cameras, an image corresponding to the position and orientation of the display device 11 can be generated by applying panorama synthesis processing or the like.

  Next, specific embodiments of the present invention will be described. In this embodiment, a case where the wide viewing angle image is a wide viewing angle image obtained by an omnidirectional camera and a case where the wide viewing angle image is a 3DCG image will be described. The wide viewing angle image is captured by an omnidirectional camera. A case where the image is a wide viewing angle image will be described as Embodiment 1, and a case where the wide viewing angle image is a 3DCG image will be described as Embodiment 2.

Embodiment 1
FIG. 2 is a diagram for explaining the first embodiment of the present invention, and is a configuration diagram when the present invention is applied to a remote conference system.

  In the remote conference system shown in FIG. 2, in one conference room 20 as an image sending side, a plurality of conference participants (in this figure, three conference participants A, B, and C) have a square table 21. Assume that the user sits in a square so as to surround the U-shape, and in the other conference room 30 on the image display side, the conference participant D is displayed on the display device 11 and participates in each conference. A situation is assumed in which participants A, B, and C participate in the conference in some form while watching the conference.

  The conference room 20 includes at least an omnidirectional camera 22 as an image input means for inputting a state in the conference room 20 as a wide viewing angle image, and a wide viewing angle image input by the omnidirectional camera 22. A wide viewing angle image output unit 2 for outputting and a network connection unit 23 capable of transmitting a wide viewing angle image output from the wide viewing angle image output unit 2 to the network 40 are provided.

  FIG. 3 shows an example of a wide viewing angle image obtained by taking an image with an omnidirectional camera. FIG. 3A shows an example of a wide viewing angle image taken using a wide angle lens. Some images of the persons A, B, and C and the table 21 are acquired. FIG. 3B is an example of a wide viewing angle image photographed using a fisheye lens, and the conference room 20 as well as the three conference participants A, B, and C and a part of the table 21 are shown. An image has been acquired.

  On the other hand, on the conference room 30 side, the image display device 1 described in FIG. 1 (having the display device 11 and the position detection unit 12), the specific gaze direction image generation unit 3, and the wide field of view from the conference room 20 side. A wide viewing angle image acquisition unit 31 that acquires a corner image (a wide viewing angle image in this embodiment) via the network 40 is provided.

  In the first embodiment, the specific line-of-sight direction image generation unit 3 can input a wide viewing angle image display request. When the wide viewing angle image display request is input, the wide viewing angle image acquisition unit 31 It is assumed that the acquired wide viewing angle image can be output to the display device 11 as it is (in this embodiment, since it is a wide viewing angle image, the wide viewing angle image is used as it is).

  Further, as described above, the position detection unit 12 uses the sensor unit 121 such as an acceleration sensor, an ultrasonic sensor, a mechanical or optical sensor, an angle sensor, and the output of the sensor unit 121 to output the display device 11. Position information calculation means 122 for calculating the position and its direction as position information is provided.

The position information from the position information calculation unit 122 can be a position in a three-dimensional space of a coordinate system (referred to as a display coordinate system) on the display screen 11a of the display device 11.
The position of the display coordinate system in the three-dimensional space refers to the coordinate on the three-dimensional coordinate of the reference point of the display coordinate system (for example, the base point of the two-dimensional coordinate on the display screen 11a) or the reference Can be expressed by the polar coordinates of the point, the normal vector of the display screen 11a, and the rotation angle in the rotation direction with the normal vector as the central axis. Alternatively, it can be represented by coordinates on a three-dimensional coordinate of a plurality of points in the display coordinate system or polar coordinates of the plurality of points.

In this embodiment (including Embodiment 1 and Embodiment 2 described later), the position information includes four corner points of the display area on the display screen 11a of the display device 11 (in this embodiment, the description is simplified). For this reason, it is assumed that the display area coincides with the display screen 11a, and the coordinates on the three-dimensional coordinates of the four corner points a1, a2, a3, and a4 of the display screen 11a are used.
However, since the shape of the display screen 11a of the display device 11 is actually determined (usually a rectangle), the coordinates of the three corner points (for example, the three corner points a1, a2, and a3 in FIG. 2) are determined. That's fine.

  Therefore, in this embodiment, the position information calculated by the position information calculating unit 122 is the coordinates on the three-dimensional coordinates of the three corner points a1, a2, and a3 of the display screen 11a of the display device 11, and the display device 11 The coordinates on the three-dimensional coordinates of the three corner points of the display screen 11a are sent to the specific line-of-sight direction image generation means 3 as position information.

  4 and 5 are diagrams for explaining a specific example of the position detecting means 12. The position detecting means 12 as shown in FIGS. 4 and 5 allows the points a1 at the three corners of the display screen 11a of the display device 11 to be pointed. The coordinates on the three-dimensional coordinates of a2 and a3) can be obtained as position information. A specific example of the position detection means 12 will be described with reference to FIGS. 4 and 5.

  First, in the display device 11 shown in FIG. 4, the display device main body 111 is supported by a support column 113 having a base 112, and the display device main body 111 is multi-directional with respect to the support column 113 (vertical and horizontal directions). It is supported so that the direction can be freely changed within a predetermined range.

  Further, the display device main body 111 rotates in the clockwise direction and the counterclockwise direction when the display screen 11a is viewed from the front with respect to the support column 113 (the rotation direction with the normal vector of the display screen 11a as the central axis). Rotation) is supported.

  An angle sensor (not shown) capable of detecting the orientation (angle) of the display device main body 111 with respect to the support column 113 such as the vertical direction and the horizontal direction, and the clockwise / counterclockwise rotation angle is provided. This angle sensor corresponds to the sensor unit 121 of FIG.

  Further, an optical reading device (not shown) is provided on the surface of the base 112 on the table 50 side, and the display device 11 is moved on the table 50 in the same manner as an optical mouse that is widely used in general. A movement vector based on a certain position can be calculated.

The display device 11 shown in FIG. 4 uses the movement information calculated based on the output from the optical reading device and the detection signals from the respective angle sensors described above to detect the position information ( The coordinates of the three corner points a1, a2 and a3 on the display screen 11a of the display device 11 are determined.
5 shows an example of the display device 11 having an articulated arm stand. FIG. 5 (a) is a perspective view and FIG. 5 (b) is a side view.
This articulated arm stand supports a column 62, a first arm 63, a second arm 64, and a display device main body 111 that are rotatably supported in the arrow bb ′ direction with respect to the base 61. The display device main body support member 65, the first joint mechanism 66 interposed between the support 62 and the first arm 63, and the second joint interposed between the first arm 63 and the second arm 64. The joint mechanism 67, the second arm 64, and the third joint mechanism 68 interposed between the display apparatus main body support member 65 are provided.

  The display device 11 having such an articulated arm stand can freely move in the vertical direction and in the front-rear direction within a predetermined range by the first, second, and third joint mechanisms 66, 67, and 68. In addition, the vertical and horizontal orientations of the display device main body 111 can be set within a predetermined range, and the column 62 that can be rotated in the direction of the arrow bb ′ can be freely rotated around the column 62. It has become a thing.

  Further, in the same manner as the display device 11 shown in FIG. 4, the display device 11 has a display device body 111 with respect to the display device body support member 65 in a predetermined range in multiple directions (vertical direction, horizontal direction, etc.). In the same manner as described above, the support 62 is rotated about the normal vector around the base point of the normal vector, that is, the display screen 11a is front-faced. It is supported so that it can be rotated clockwise and counterclockwise when viewed from above. It is supported so that it can be rotated clockwise or counterclockwise.

  Therefore, the display device 11 having the structure as shown in FIG. 5 includes the rotation angle of the support 62 with respect to the base 61, the bending angles of the first, second, and third joint mechanisms 66, 67, 68, and further the display device. The angle between the main body support member 65 is detected by each angle sensor provided, and the position information (display screen 11a of the display device 11) is detected by the position information calculation unit 122 using the angle detection signal from each angle sensor. 3) (coordinates on the three-dimensional coordinates of the points a1, a2, and a3 at the three corners) can be obtained.

  The structure of the articulated arm stand in the display device 11 shown in FIG. 5 is an example, and it is needless to say that the structure is not limited to such a structure.

  By the way, the wide viewing angle image output from the wide viewing angle image output unit 2 on the conference room 20 side is transmitted from the network connection unit 23 to the conference room 30 side via the network 40. On the conference room 30 side, the wide viewing angle image is acquired by the wide viewing angle image acquisition means 31.

  The wide viewing angle image acquired by the wide viewing angle image acquisition unit 31 is, for example, an image as shown in FIGS. 3A and 3B. In the first embodiment, FIG. A wide viewing angle image using a fisheye lens is used.

  Here, it is assumed that the conference participant D on the conference room 30 side looks at the conference room 20 side on the display device 11. At this time, a plurality of display devices 11 used by the conference participant D may exist, but here, it is assumed that there is one display device 11.

First, when the conference participant D wants to see the entire state of the conference room 20 side, a wide viewing angle image display request is given to the specific viewing direction image generation means. As a result, the wide viewing angle image (in this case, the wide viewing angle image) acquired by the wide viewing angle image acquisition unit 31 is displayed on the display device 11 via the network 40.
As a result, an image as shown in FIG. 3B is displayed on the display device 11 used by the conference participant D regardless of the position of the display device 11.

  Next, an operation for displaying the specific line-of-sight direction image on the display device 11 will be described with reference to FIG. In this case, it is assumed that the input of the wide viewing angle image display request to the specific line-of-sight direction image generation unit 3 has been canceled.

  FIG. 6 shows a case where one display device 11 is sequentially placed at a position P1, the next is a position P2, the next is a position P3, and the three display devices 11 are present. Not to do.

  For example, it is assumed that the display device 11 is now placed at a position P1 (a position in front of the conference participant D) on a table (not shown in FIG. 6). The line-of-sight direction of the conference participant D at this time is V1 in FIG. 6, and this line-of-sight direction V1 is a direction along the normal direction of the center coordinates of the display screen 11a of the display device 11.

  When the user (conference participant D in the first embodiment) views the image displayed on the display device 11, the user's line-of-sight direction is along the normal on the center coordinates of the display screen 11 a of the display device 11. The direction of the user's line of sight is not necessarily the direction of the normal line itself, but may be slightly deviated from the normal direction.

  Thus, when the display device 11 is placed at the position P1 by the conference participant D, the position information is sent from the position detection means 12 to the specific line-of-sight direction image generation means 3. Note that the position detection means 12 always detects the position of the display device 11 and can always output the current position information of the display device 11.

  The specific line-of-sight direction image generation unit 3 uses the position information from the position detection unit 12 and the wide viewing angle image acquired by the wide viewing angle image acquisition unit 31 to generate an image suitable for the line-of-sight direction of the conference participant D. I do.

  That is, in this case, the conference participant D faces the front direction (position P1 direction) on the table, and the display device 11 is placed so that the conference participant D and the display screen 11a of the display device 11 face each other. Is the case. Therefore, in this case, the specific line-of-sight direction image generation means 3 as shown in FIG. 7A based on the position information calculated by the position detection means 12 from the image photographed by the fisheye lens of FIG. Then, the image to be displayed is cut out. FIG. 7 (a) is the same image as FIG. 3 (b).

  In this case, first, an image surrounded by the thick line frame F1 in FIG. 7A (an image of the conference participant B sitting in the middle) is cut out. However, since the wide viewing angle image shown in FIG. 7A is an image taken with a fish-eye lens, if the image that has been cut out is displayed as it is on the display device 11 on the conference participant D side, The image is distorted and unnatural.

  Therefore, the specific gaze direction image generation unit 3 performs image correction to generate an image of the conference participant B that does not give the user a sense of incongruity by correcting the distortion as shown in FIG. As described above, the image of the conference participant B whose image has been corrected is sent to the display device 11.

  As a result, the image of the conference participant B is displayed on the display device 11 placed at the position P1 in front of the conference participant D so as to face the conference participant D.

  In this state, if the vertical angle of the display device 11 is changed slightly downward, the normal of the center coordinate of the display screen 11a of the display device 11 changes downward according to the rotation angle. When the specific line-of-sight direction image generation means 3 receives the corresponding position information, image generation according to the position information is performed.

  At this time, the image displayed on the display device 11 is the conference participant B. When the display device 11 is turned downward, the head of the conference participant B is positioned upward in the display screen 11a, and the table 21 is displayed. Since more parts are displayed, it is convenient to check materials placed on the table 21.

  Further, if the display device 11 is moved in the front-rear direction with respect to the conference participant D at the same position P1, zooming of the image is also possible. For example, if the display device 11 is moved backward (in the depth direction) at the position P1, the image of the conference participant B becomes smaller. Conversely, if the display device 11 is moved forward, the conference participant B It is also possible to display such that the image becomes larger.

  From this state (as shown in FIG. 6, the display device 11 is placed on P1), the conference participant D moves the display device 11 on the table in the direction of the arrow X in FIG. , It is assumed to be placed at the position P2. The line of sight of the conference participant D at this time is assumed to be V2 in FIG.

  Thus, when the display device 11 is placed at the position P2 by the conference participant D, the position information is sent from the position detection means 12 to the specific line-of-sight direction image generation means 3.

  The specific gaze direction image generation unit 3 uses the position information from the position detection unit 12 and the wide viewing angle image acquired by the wide viewing angle image acquisition unit 31 to provide an image suitable for the gaze direction V2 of the conference participant D. Generate.

  At this time, the line-of-sight direction of the conference participant D shifts from V1 to V2, and the display apparatus is placed so that the line-of-sight direction V1 and the normal direction of the center coordinates of the display screen 11a of the display apparatus 11 substantially coincide. Therefore, in the specific line-of-sight direction image generation means 3, based on the position information from the position detection means 12, the image surrounded by the thick line frame F2 in FIG. 7A (in the example of FIG. 7A, The image of the conference participant C sitting at the right end) is cut out, and as described above, the specific gaze direction image generation means 3 performs image correction to generate an image without distortion as shown in FIG.

  Then, the image of the conference participant C subjected to this image correction is sent to the display device 11. Accordingly, the image of the conference participant C is displayed on the display device 11 placed at the position P2 in FIG. 6 so as to face the conference participant D.

  As described above, since the position detection unit 12 always detects the position of the display device 11 and can always output the current position information of the display device 11, the user (the conference participant D in this example) intends. As long as the display state of the display device 11 is not turned off, the image at that time can be displayed even while the display device 11 is moving.

  For example, in the example of FIG. 6, when the display device 11 is moved from the position P1 to the position P2, the conference participant B gradually approaches the left end of the display area along with the movement, and the conference participant C Gradually appears, and at the intermediate point between the positions P1 and P2, both the conference participant B and the conference participant C are displayed on the display screen 11a. When the display device 11 reaches the position P2, the display screen 11a is displayed. Is in a state where the conference participant C is displayed in the middle.

  Further, from the state where the display device 11 is placed at the position P2 shown in FIG. 6, the conference participant D moves the display device 11 on the table in the direction of the arrow X ′ in FIG. 6 and places it at the position P3. Suppose. The line of sight of the conference participant D at this time is assumed to be V3 in FIG.

Thus, when the display device 11 is placed at the position P3 by the conference participant D, as described above, the image surrounded by the thick line frame F3 in FIG. 7A (in the example of FIG. 7A) The image of the conference participant A sitting at the left end) is cut out, and the specific gaze direction image generation means 3 performs image correction to obtain an image of the conference participant A without distortion as shown in FIG. Generate. Then, the image of the conference participant A for which the image correction has been performed is sent to the display device 11.
Accordingly, the image of the conference participant A is displayed on the display device 11 placed at the position P3 in FIG. 6 so as to face the conference participant D.

  Thus, the user (conference participant D in this example) can display an image he / she wants to see by changing the position of the display device 11 or changing the direction of the display device 11.

  In addition, when displaying the specific gaze direction image according to the positional information by changing the position or direction of the display device 11 described above, the specific gaze direction image displayed at each position is displayed. It is also possible to display a wide viewing angle image in part.

  FIG. 8 shows an example of the display. In FIG. 8, the display device 11 is placed at the position P1, and a wide viewing angle image 70 is displayed in a part of the display area where the image of the conference participant B is displayed. Is displayed.

  In order to realize this, for example, in the configuration of FIG. 2, the specific gaze direction image generated by the specific gaze direction image generation unit 3 and the wide viewing angle image acquired by the wide viewing angle image acquisition unit 31 are synthesized. By adding an image composition means (not shown) and giving an image composition request, for example, as shown in FIG. 8, a wide viewing angle image can be displayed on a part of the specific line-of-sight direction image.

  In this way, by displaying the wide viewing angle image in a part of the display area displaying an image in a specific line-of-sight direction, for example, in the example of FIG. 8, while confirming the state of other conference participants Some kind of conversation can be held with the conference participant D, the remote conference can be made more realistic, and the conference can proceed smoothly.

  FIG. 9 is a diagram for explaining a modification of the above-described first embodiment. In this FIG. 9, the specific gaze direction image generating means 3 is provided on the image sending side (the meeting room 20 side in this embodiment), and the image On the display side (the conference room 30 side in this embodiment), a position information transmission unit 32 that transmits the position information obtained by the position detection unit 12 and a display image acquisition unit that acquires a display image sent from the image transmission side. 33.

  On the other hand, on the conference room 20 side which is the image sending side, the network connection means 23 receives the positional information of the display device 11 on the conference room 30 side via the network 40 and outputs the positional information and the wide viewing angle image output means. The specific viewing direction image is generated using the wide viewing angle image, and the generated specific viewing direction image is transmitted to the image display side (meeting room 30 side) via the network 40.

  In the conference room 30 on the image display side, the display device 11 acquires the specific line-of-sight image transmitted from the conference room 20 on the image sending side, and the display device 11 displays the acquired specific line-of-sight image. Send to.

  In the case of FIG. 9 as well, the display device 11 may want to display only a wide viewing angle image if necessary. In this case, for example, it is possible to set to send a wide viewing angle image from the image sending side by giving a request for transmitting a wide viewing angle image not shown in FIG. It is.

  Further, the specific line-of-sight direction image generation means 3 may be installed at a place other than the image sending side (the meeting room 20 side in this embodiment) or the image display side (the meeting room 30 side in this embodiment). For example, if it is a remote conference system like this Embodiment 1, you may make it provide in the management server (not shown) which manages the remote conference system.

  In this way, when the specific visual line direction image generating means 3 is not provided on the image display side, the information transfer process between the image sending side and the image display side is somewhat complicated, as can be seen from FIG. It becomes. However, for example, when there are many image display sides, if a wide viewing angle image is sent to all the image display sides, the transmission load can be increased if the data amount of the wide viewing angle image is large. There is sex.

  On the other hand, in the method of FIG. 9, the image to be sent to the image display side is mainly an image obtained by cutting out a wide viewing angle image, so that there may be a case where the data amount is not so large. Therefore, the effect corresponding to each can be obtained by properly using the system to which the present invention is applied.

  In the above example, the example in which the present invention is applied to the remote conference system has been described. However, the present invention can be applied to a wide range of fields such as a monitoring system and astronomical observation. For example, when used as a monitoring system for a store or the like, the conference room 20 shown in FIG. 1 is set in the store to be monitored, and the conference room 30 is set as a monitoring center. The image at the desired position can be displayed.

Specifically, for example, a wide viewing angle image obtained by photographing one floor in the store is sent to a monitoring center or the like. On the other hand, in the monitoring center, when the monitor changes the angle or the position of the display device as the monitor as described above, the position information is output from the position detecting means 12 and transmitted from the position information and the store side. The specific viewing direction image generation means 3 generates an image corresponding to the position information using the wide viewing angle image, and displays it on the display device 11.
As a result, the monitoring person can display an image of the position to be monitored by changing the position where the display device 11 is placed or changing the angle.

When used for astronomical observation, for example, a wide viewing angle image obtained by photographing the whole sky is sent to the image display side. On the other hand, on the image display side, as described above, by changing the position where the display device 11 is placed or changing the angle, the specific visual line direction image is obtained from the position information and the wide viewing angle image sent from the image sending side. The generation unit 3 generates an image corresponding to the position information and displays it on the display device 11.
As a result, the constellation portion that the user wants to see can be freely displayed on the display device 11 from all over the sky.

  As described above, in the present invention, when a certain part of the wide viewing angle image acquired on the image sending side is to be viewed, only the display device 11 is moved so that the desired image is displayed. The image of the position that the user wants to see can be displayed on the display device 11 with a very simple operation. Further, when an image of the position desired to be viewed is displayed on the display device 11, if the display device is kept as it is, the position information of the display device 11 remains fixed. Will continue to be displayed and is the same as watching a display screen of a normal display device.

  Therefore, unlike conventional systems that display images in conjunction with the line-of-sight direction, the user does not need to always see the displayed image, so the fatigue and burden on the user is the display screen of a normal display device. Can be the same as watching. In addition, this can be used for a wide range of applications because the display image can sometimes be confirmed while referring to other materials.

  In this embodiment, the wide viewing angle image has been described as an example using a wide viewing angle image obtained by photographing with an omnidirectional camera or the like, but a wide viewing angle image generated with an omnidirectional camera or the like has been described. Is not necessarily used. For example, it is also possible to create a virtual space on a computer as if a plurality of people are in the same place by using images obtained by individually shooting a plurality of people in different places.

[Embodiment 2]
In the first embodiment described above, as an example of the wide viewing angle image to be sent to the image display side, for example, an image taken with a fisheye lens or the like has been shown. However, in this second embodiment, a 3DCG image is used as the wide viewing angle image. Use.

  FIG. 10 is a configuration diagram for explaining the second embodiment, and the image display device 1 is the same as the image display device 1 of FIG. 2, and thus the description thereof is omitted here.

  The wide viewing angle image in the case of the second embodiment is acquired from another computer or the like via local three-dimensional object information 51 or network 40 generated by calculation by application software such as a computer game or CAD, for example. 3D object information 52 is used. An object control program 53 is provided for the three-dimensional object information 51, and an object information receiving program 54 is provided for the three-dimensional object information 52 acquired from another computer or the like via a network. .

  Note that either one of the local three-dimensional object information 51 and the three-dimensional object information 52 obtained via the network can be used. These three-dimensional object information may be used in combination.

  FIG. 11 shows an example of a 3DCG image generated by the three-dimensional object information (herein, the local three-dimensional object information 51) input to the specific line-of-sight direction image generation means 3. Here, an intersection or a building is shown. Think of a cityscape with buildings such as mountains, mountains behind it, and a lake on the right.

  In such a scene, it is assumed that a portion marked with a circle (marked with P0) is a virtual viewpoint in the figure. Now, when the direction of the arrow V1 and the direction of the arrow V2 are set as the line-of-sight directions at the viewpoint P0, how they look from the respective line-of-sight directions V1 and V2 is considered.

  FIG. 12A is a plan view of FIG. 12 with the viewpoint P0 as the center. In FIG. 12A, only the periphery of the viewpoint P0 is shown, and mountains and lakes are omitted. In FIG. 12A, when the line-of-sight direction from the viewpoint P0 is changed to V1 and V2, it can be obtained by calculation, and an image obtained by calculating with V1 as the line-of-sight direction is obtained. In FIG. 12B, there is an intersection in the foreground, a road that extends straight from the building group behind it, and a landscape where mountains can be seen behind. In addition, an image obtained by calculating with the arrow V2 as the line-of-sight direction is FIG. 12C, which is a landscape where the lake can be seen behind the building.

  As described above, when the line-of-sight direction of the 3DCG image is determined, what kind of image can be seen when viewed from the line-of-sight direction can be obtained by calculation. By using this, the display device 11 can be placed at an arbitrary position, and by setting the orientation of the display screen 11a and the like, an image from the line-of-sight direction desired to be viewed can be displayed on the display device 11. it can.

  FIG. 13 shows an example of the case where the display device 11 is placed at the positions P1 and P2, and this is the same situation as when the display device 11 is placed at the positions P1 and P2 in FIG. To do.

  First, the display device 11 is placed at the position P1. The user's line-of-sight direction at this time is V1, which is the same state as when the display device 11 is placed at the position P1 in FIG.

  As a result, the specific line-of-sight image generation unit 3 uses the position information from the position detection unit 12 (as described above, the coordinates on the three-dimensional coordinates of the three corner points a1, a2, and a3 on the display screen 11a of the display device 11). ) Based on the three-dimensional object information 51, the display device 11 placed at this position P1 is generated by generating an image of the viewing direction V1 from the viewpoint P0 in the 3DCG image (see FIG. 11). An image similar to that shown in FIG. 12B is displayed.

  Next, from this state (the state in which the display device 11 is placed at P1 as shown in FIG. 13), the user moves the display device 11 in the direction of the arrow X in FIG. 13 and places it at the position P2. Suppose. The user's line-of-sight direction at this time is V2, which is the same as when the display device 11 is placed at the position P2 in FIG.

  As a result, the specific gaze direction image generation means 3 is based on the position information from the position detection means 12 when the gaze direction V2 is viewed from the viewpoint P0 in the 3DCG image (see FIG. 11) based on the three-dimensional object information 51. By generating an image, an image similar to that shown in FIG. 12C is displayed on the display device 11 placed at the position P2.

  In the case of the second embodiment as well, as in the first embodiment, the image at that time is displayed even while the display device 11 is moving unless the user intentionally turns off the display state of the display device 11. Is possible.

  For example, in the example of FIG. 13, when the display device 11 is moved from the position P1 to P2, the scenery displayed on the display device 11 with the movement is that the line-of-sight direction from the viewpoint P0 in FIG. When the display device 11 reaches the position P1, the scenery as shown in FIG. 12C is obtained.

  As described in the first embodiment, when the vertical angle of the display screen 11a of the display device 11 is changed while the display device 11 is in a certain position, an image corresponding to the change can be displayed. Of course.

  For example, when the display device 11 is placed at the position P1 and the display device 11 is slightly turned downward, the image displayed at that time cannot be seen in the sky and mountains in the image of FIG. The image is such that the intersection is located at the center of the display screen 11a in the vertical direction. On the other hand, if the display device 11 is slightly upward while the display device 11 is placed at the position P1, the image displayed on the display device 11 at that time is empty or empty in the image of FIG. Scenery such as ridgelines of mountains is the main image.

  Also in the case of the second embodiment, as in FIG. 1, if a function for synthesizing the image generated by the specific line-of-sight direction image generating means 3 and the wide viewing angle image as shown in FIG. At the position P1 shown in FIG. 13, a wide viewing angle image as shown in FIG. 11 can be displayed in a part of the display area where the specific line-of-sight direction image is displayed. According to this, it is possible to easily confirm from which direction of the entire image the currently viewed image is viewed.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, in each of the above-described embodiments, the display device 11 has been described as one unit, but a plurality of display devices 11 may be used.

  In the above-described embodiment, as the means for obtaining the position information, the example shown in FIGS. 4 and 5 is used as an example. However, the present invention is not limited to this. For example, as shown in FIG. The position information can also be obtained by an image recognition method using the camera 81 or the like. In this case, for example, four corners c1, c2, c3, and c4 of the outer frame of the display device 11 are extracted as feature points from the image obtained by photographing the display device 11, and 3 of the extracted feature points are extracted. Find coordinates on dimensional coordinates.

  Further, ultrasonic oscillators (not shown) are provided at the four corners c1, c2, c3, and c4 of the display device 11, and ultrasonic sensors (not shown) are provided at four corners such as a table on which the display device 11 is placed. ), And when the display device 11 is moved, the respective ultrasonic waves (ultrasonic waves of individual patterns) from the four ultrasonic oscillators of the display device reach the ultrasonic sensors at the four corners of the table. It is also possible to obtain the coordinates on the three-dimensional coordinates of the corners c1, c2, c3, c4 of the display device from the time difference.

  In each of the above-described embodiments, when the display coordinate system is represented as a position in the three-dimensional space, the coordinates on the three-dimensional coordinates are used. However, polar coordinates based on a certain point may be used.

  In addition, the wide viewing angle image used in the present invention can be used by mixing both an image input by an image input means such as a digital camera and a 3DCG image. As described above, even when both an image input by an image input unit such as a digital camera and a 3DCG image are used in combination, the respective images are specified as described in the first embodiment and the second embodiment. A line-of-sight image can be generated.

  Thus, for example, in the case of the above-described conference system, what is being discussed at the conference (for example, architectural design drawings) is expressed in 3DCG or the like and displayed in a superimposed manner. You can also

  Further, the present invention can create a processing program in which the processing procedure for realizing the present invention described above is described, and the processing program can be recorded on a recording medium such as a floppy disk, an optical disk, a hard disk, The present invention also includes a recording medium on which the processing program is recorded. Further, the processing program may be obtained from a network.

1 is a schematic configuration diagram of an image display system of the present invention. It is a figure explaining the structure of the remote conference system which is Embodiment 1 of this invention. It is a figure which shows an example of a wide viewing angle image, (a) is a figure which shows the example of a wide viewing angle image image | photographed using the wide angle lens, (b) is a figure which shows the example of a wide viewing angle image image | photographed using the fisheye lens. is there. It is a figure explaining an example of a means to detect position information. It is a figure explaining other examples of a means to detect position information. It is a figure explaining the example from which the image displayed changes with the position which puts a display apparatus. It is a figure explaining the image (specific gaze direction image) generation | occurrence | production displayed in each position shown in FIG. It is a figure explaining the example which displays a wide viewing angle image on a part of specific eyes | visual_axis direction image currently displayed with the display apparatus. It is a figure explaining the modification of Embodiment 1, and is a figure explaining the example which provided the specific gaze direction image generation means in the image sending side. It is a figure explaining the structure of the image display system in the case of using 3DCG which is Embodiment 2 of this invention as a wide viewing angle image. It is a figure which shows an example of a 3DCG image. It is a figure explaining what kind of image it becomes when it sees from a certain viewpoint in the 3DCG image of FIG. It is a figure explaining the example from which the image displayed according to the position which puts a display apparatus in Embodiment 2 differs. It is a figure explaining other examples of a means to detect position information.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image display apparatus, 2 Wide viewing angle image output means, 3 Specific gaze direction image generation means, 11 Display apparatus, 11a Display screen, 12 Position detection means, 22 Camera, 23 Network connection means, 31 Wide viewing angle image acquisition means, 32 position information transmission means, 33 display image acquisition means, 40 network, 51, 52 three-dimensional object information, 121 sensor unit, 122 position information calculation means, a1, a2, a3, a4 corner points of display area (display screen) , V1, V2, V3 viewing direction, P0 viewpoint, P1, P2, P3 position

Claims (15)

An image display device capable of displaying an image when viewed from a viewing direction with a wide viewing angle image,
A display device capable of changing a position so as to match a user's line-of-sight direction; and position detection means for detecting the position of the display device and outputting the detection result as position information. Is output from the position detection means, the display device acquires a specific gaze direction image corresponding to the position information generated from the position information and the wide viewing angle image, and the acquired specific gaze direction image is An image display device characterized by displaying on the display device.   2. The image display device according to claim 1, wherein the position information output from the position detection means is a position in a three-dimensional space of a display coordinate system in the display device.   The position in the three-dimensional space of the display coordinate system is the coordinates on the three-dimensional coordinates of the reference point of the display coordinate system or the polar coordinates of the reference point and the normal line at a point of the display coordinate system. The image display device according to claim 2, wherein the image display device is represented by a vector and a rotation angle in a rotation direction with the normal vector as a central axis.   The position of the display coordinate system in a three-dimensional space is represented by coordinates on a three-dimensional coordinate of a plurality of points in the display coordinate system or polar coordinates of the plurality of points. Image display device.   The image display device according to claim 4, wherein the plurality of points are at least three corner points of a display area of the display device.   The image display apparatus according to claim 1, wherein the wide viewing angle image is an image input by an image input unit.   The image display apparatus according to claim 1, wherein the wide viewing angle image is a three-dimensional computer graphics image.   6. The image display according to claim 1, wherein the wide viewing angle image can be used by mixing an image input by an image input means and a three-dimensional computer graphics image. apparatus.   The specific line-of-sight image displayed on the display device is an image suitable for displaying on a display area of the display device with respect to an image of a predetermined range cut out from the wide viewing angle image based on the position information. 9. The image display apparatus according to claim 6, wherein the image is corrected.   The specific gaze direction image displayed on the display device is an image obtained by calculating a gaze direction from a certain viewpoint position of the three-dimensional computer graphics image based on the position information. The image display device according to claim 7 or 8.   11. The wide viewing angle image is displayed in a display area that is a part of a display area of the specific line-of-sight image when the specific line-of-sight image is displayed on the display device. The image display device described.   A display device capable of changing a position so as to match a user's line-of-sight direction; and position detection means for detecting the position of the display device and outputting the detection result as position information. Is output from the position detecting means, the display device acquires a specific gaze direction image corresponding to the position information generated from the position information, and the acquired specific gaze direction image is displayed on the display device. An image display device characterized by the above.   The specific line-of-sight image displayed on the display device is an image obtained by calculating a line-of-sight direction from a certain viewpoint position of a three-dimensional computer graphics image based on the position information. Item 13. The image display device according to Item 12. An image display method capable of displaying an image when viewed from a viewing direction with a wide viewing angle image on a display device,
Position information of the display device is detected and output, and using the position information and the wide viewing angle image, a specific gaze direction image corresponding to the position information is generated, and the display device generates the specific gaze direction image. An image display method characterized by acquiring and displaying the acquired specific gaze direction image. An image display system capable of displaying an image when viewed from a viewing direction with a wide viewing angle image,
A wide viewing angle image output means capable of outputting a wide viewing angle image;
A display device capable of changing the position so that it can be adjusted to the user's line-of-sight direction;
Position detecting means for detecting the position of the display device and outputting the detection result as position information;
The position information is received from the position detection means, and a wide viewing angle image is received from the wide viewing angle image output means, and a specific gaze direction image corresponding to the position information is generated from the position information and the wide viewing angle image. Specific gaze direction image generating means for performing,
An image display system comprising:

Images