JP3289730B2 - I / O device for image communication - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video telephone conference system and an image communication input / output device having a high sense of presence in a video telephone system.

[0002]

2. Description of the Related Art With the development of image communication technology, a videophone conference system and a videophone capable of holding a meeting or a meeting while viewing a face image of a remote party at a remote place have been practically supplied.

[0003] By the way, the video conference system and the video telephone system require an imaging and display method that is natural, has a sense of realism, and is capable of reading facial expressions and the like without being conscious of a teleconference. Hereafter, as a technique for enhancing the sense of reality, an input / output device for image communication based on line-of-sight matching and 3
An input / output device for image communication based on dimensions will be described.

FIG. 6 is a diagram for explaining an image pickup and display method of a video conference system currently in practical use. (1) and (2) of FIG. 6 show terminals of the video conference system at remote locations, respectively.
Communication of the two terminal partner _{M 1 ~M 3, MT 1 ~MT} 3 is facing the camera 2 is positioned away from the display screen 1.
For example, three attendees M _{1 to} M ₃ , MT ₁ to
When the meeting has been advanced by MT _3, display a CRT
The most common method is to use a direct-view type device such as the one described above, but the projection type method using a CRT or a liquid crystal display panel is more likely to be used when the display size becomes large. Although this method has a very simple configuration, the communication partners M _{1 to} M ₃ and MT displayed on the display screen 1 of the display device are not shown.
_{For 1} to MT camera 2 away from the position of the face of the ₃ is installed, the line of sight with the communication partner is a problem that does not match.

In order to solve the above-mentioned problems, as a method of installing a camera in the line of sight of a person image without interrupting the display screen of the image display device, a line-of-sight display / image pickup device using a half mirror is used. It has been known.

FIG. 7 shows an example of the configuration of a line-of-sight type display / image pickup apparatus using a half mirror. An observer M observes a display device 4 such as a CRT via a half mirror 3. At the same time, since the face of the observer M is imaged by the imaging device 2 (camera), the line of sight of the observer M is directed toward the image capturing device 2 (camera), and a face image that matches the line of sight of the communication partner is transmitted. Will be done.

[0007] In the above-described conventional technology, a one-on-one telephone conversation can be realized with a realistic sense of sight with the same line of sight. Communication conference can not be realized. Next, the reason why the eyes do not match in a video conference in which a plurality of people are present will be described.

FIG. 8 is a diagram for explaining the line of sight mismatch. For example, assume a three-to-three communication conference. The display screen 5 of the display device displays MT ₁ , MT ₂ ,
Three of the video of the MT ₃ is displayed. Also, M ₁ ,
Assume that three persons M ₂ and M ₃ are present in the conference. The cameras 2, 2 'are display screens 5, which are display devices.
Assume that it is installed at the rear center of 5 '.

First, a description will be given of the fact that the lines of sight coincide when M ₂ talks with MT _{2 of the} communication partner. Since M ₂ is talking with MT ₂ displayed at the center of the display screen 5, the user sees the camera 2 installed at the rear center of the display screen 5 equivalently. The images of M ₁ , M ₂ , and M ₃ captured by the camera 2 are displayed on the display screen 5 ′ of the communication partner MT ₁ , MT ₂ , and MT ₃ . Since the image of the MT ₂ is displayed as an image facing the front in a central portion of the display screen 5 ', though, would be displayed as if looking at the MT _2. Similarly, M
Since the image of T ₂ is also displayed as a front image in the center of the display screen 5 of the _{M 1, M 2, M 3} , MT 2 of the communication partner and M ₂ may be a conversation that matches the line of sight.

Next, a description will be given of the fact that the lines of sight never coincide in the conversation between M ₁ and the communication partner MT ₁ . M ₁ is the MT displayed at the end of the display screen 5 for talking with MT ₁
You will turn your face to _one . Imaging the M ₁ image profile M ₁ from the camera 2 installed in the rear center of the display screen 5, the image is projected on the display screen 5 'of MT _1. Since MT ₁ observes the M ₁ image projected on the display screen 5 ′ with its eyes turned sideways, it is impossible to realize conversation in which the eyes are matched. Similarly, the combination of non-M ₂ and MT _2, in the configuration of the camera one display image one side, it is difficult to realize a matched conversation sight.

As a method of increasing the sense of reality, there is an attempt to use not a communication of a two-dimensional image but a three-dimensional image which a human normally perceives for communication. As an example, an apparatus using a multi-lens lenticular lens as a display screen will be described. FIG. 9 is a view showing the principle of a multi-view lenticular, in which 6 indicates a lenticular sheet. This means that a single lens displays striped images taken from multiple directions a ', b', c ',... F' at portions marked a, b, c,. . By the action of the lens plate of the lenticular sheet 6, the stripe images in each direction enter the left and right eyes E _L and E _R separately, and if the viewpoint is moved S _L and S _{R, a} horizontal stereoscopic image can be seen.

In this device, by arranging a large number of cameras continuously below the distance between eyes, a three-dimensional effect can be obtained as if the cameras existed in any direction. Therefore, by arranging a large number of cameras at about the caller's eye position and at a distance equal to or less than the distance between the eyes, the gaze can be adjusted with any party displayed on the display screen and simultaneously with any combination of the callers. Matched communication can be realized.

However, since this three-dimensional apparatus uses a large number of cameras, the amount of image information to be communicated is enormous. Further, it is uneconomical to use a large number of cameras regardless of the number of persons involved in communication. There is a real possibility in the range of handling small screens up to about 10 inches, but 100
When the screen becomes larger than an inch, it becomes impractical considering a huge amount of image information.

[0014]

As described above, the eye-gaze matching device and the three-dimensional device have been actively proposed as means for realizing a sense of reality, but the eye-gaze matching device is one-to-one.
The three-dimensional device is the ultimate solution for immersive communication, but not a realistic solution.

According to the present invention, in a conventional technology, in a video conference call in which a large number of people are present, it is not possible to economically converse with any communication partner in a state where their eyes are aligned. The purpose is to make it possible.

[0016]

According to the present invention, there is provided an image communication input system for a video telephone conference system including a display screen including a diffusion plate and a lenticular sheet, a plurality of imagers, and at least one projector. In the apparatus, the plurality of imaging devices are arranged inside or in front of the display screen and at positions facing a respective observation position where a plurality of attendees observe the self-display screen, and Are synthesized using captured images from the plurality of imagers, and the projected image is a first line image from the first imager and a first line image from the second imager. After arranging the line images in the same order from the respective imaging devices in the order of the second line image, arranging the line images in the same order from all the imaging devices, the line image in the next order is the same as above. Tile is synthesized, and the lenticular sheet set of stripes consisting of a lens 1 of the same order in line line image is projected to enter, and, in each of the observation position of the display screen, the other party The focal length, the curvature, and the lens pitch of the lens of the lenticular sheet are set so as to observe only the image of the image pickup device arranged opposite to each observation position.

[0017]

According to the present invention, in a video conference in which a plurality of people attend, video images captured by a plurality of cameras are transmitted as images of only the communication partners facing each attendee,
On the display screen of the communication partner, only the transmitted image is displayed by the lenticular lens used for the display screen, and the number of the display images corresponding to the images captured by the plurality of cameras can be displayed. this is,
A plurality of cameras corresponds to the fact that the camera is present in front of the communication partner instead of the attendee of the conference. Also,
The image is also displayed on the display screen as if the attendee were in front of the communication partner.

[0018]

FIG. 1 is a view showing a conference scene using an input / output device for image communication according to the present invention. M ₁ , M ₂ , and M ₃ are conference attendees, and MT ₁ , MT ₂ , and MT ₃ are communication partners. Since each attendee has a respective display screen, three screens are projected on the display screen 5.

FIG. 2 is a diagram showing the principle of imaging and display according to the present invention. Each attendee is observing each display screen. That is, attendees M ₁ is a display screen (1), the attendees M ₂ is a display screen (2), attendees M ₃ are observes a display screen (3). The imaging is performed by a camera (built-in to the display screen or disposed in front of the display screen) at a position equivalently facing each attendee. That is, the display screen (1) is a communication partner MT facing the attendees M ₁
An image captured by the camera C _3, which is placed in front of _one,
Similarly the display screen (2) is an image captured by the camera C ₂ or the display screen, (3) is an image captured by the camera C _1.

Therefore, each attendee can see an image as if he were in front of the communication partner. For example, MT ₁ is when viewing the M ₁ becomes equivalent Looking at the camera C ₃ for M _1, an image line of sight matches against M ₁ is imaged. Further, MT ₁ is when viewing the M ₃ represents, since the M ₃ is an image captured by the camera C ₁ are transmitted, an image line of sight matches against M ₃ is imaged. On the other hand, an image of MT ₁ whose sight line does not match is transmitted to M ₁ .

As described above, if a display screen having a plurality of display screens is used, even in a conference where a plurality of people attend,
Between the attendees M ₁ , M ₂ , and M ₃ and the communication partners MT ₁ , MT ₂ , and MT ₃ , communication in which the lines of sight match can be realized in any combination.

Next, the principle of creating a plurality of display screens according to the present invention will be described. FIG. 3 shows an embodiment of the present invention using a rear projection system using a projector. FIG. 4 shows a method of projecting stripe-shaped pixels viewed from the direction of the projector 8. The display screen 5 includes a lenticular sheet 6 and a diffusion plate 7 as minimum components.
The diffusion plate 7 functions as a display surface for projection light from a projector 8 (liquid crystal projector) disposed behind the display screen 5. C, B, and A in the figure are a set of pixels of one line of an image captured from different cameras, respectively.
The projection is performed so that the pixels of three lines C, B, and A fall within one line of the striped lenticular lens.

FIG. 5 is a diagram for explaining the setting of a lenticular lens so as to have respective display screens when there are three observers. In the figure, L is the width of the observer's observation area, S is the distance from the display screen to the observer, f '.
The focal length of the lens, P is the lens pitch, P _p is 1 pixel pitch, P 'is the pitch of pixels of 3 pixels as one unit. In order to divide the observation area into three, the pitch P of the lens is made narrower than the pitch P 'of the pixels. The image by the display pixel of A is projected to the observer (A), similarly, the image by the display pixel of B is projected to the observer (B), and the image by the display pixel of C is projected to the observer (C). Each observation area
Parameters such as the width (shown by oblique lines) and the lens pitch are determined from the following proportional relationships.

[0024]

F '/ P _p = S / L

[0025]

M / 2 × P ′: m / 2 × P = (S + f ′): S where m is the number of lenses. Next, the curvature of the lens is determined using a computer, and the lens is set.

[0026]

As described above, the input / output device for image communication according to the present invention can provide a screen depending on the angle at which the display screen is observed on one display screen, so that a video telephone conference attended by a large number of people can be provided. Thus, it is possible to realize a conference with a natural sense of realism that has a line of sight with any person of the communication partner.

[Brief description of the drawings]

FIG. 1 is a diagram showing a meeting scene using an image communication input / output device according to the present invention.

FIG. 2 is a principle diagram of imaging and display according to the present invention.

FIG. 3 is an embodiment using a rear projection system by the projector of the present invention.

FIG. 4 shows a method of projecting stripe-shaped pixels viewed from the direction of the projector.

FIG. 5 is a diagram illustrating setting of a lenticular lens for providing a display screen when there are three observers.

FIG. 6 is a diagram for explaining an imaging and display method of a video conference system currently in practical use.

FIG. 7 is a configuration example of a line-of-sight type display / imaging device using a half mirror.

FIG. 8 is a diagram for explaining line-of-sight mismatching.

FIG. 9 is a diagram showing the principle of a multi-view lenticular.

[Explanation of symbols]

2: Camera, 5, 5 ': Display screen, 6: Lenticular sheet, 7: Diffusion plate, 8: Projector.

────────────────────────────────────────────────── ─── Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI H04N 5/74 H04N 5/74 K (72) Inventor Kenji Akiyama 1-6-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Corporation (56) References JP-A-1-260987 (JP, A) JP-A-2-87792 (JP, A) JP-A-3-97390 (JP, A) JP-A-3-89794 (JP, A) JP-A-1-107247 (JP, A) JP-A-2-106735 (JP, A)

Claims (3)

(57) [Claims] 1. A display screen comprising a diffusion plate and a lenticular sheet; a plurality of imagers;
An input device for image communication of a video conference system including a plurality of projectors, wherein the plurality of imagers are inside or in front of the display screen, and a plurality of attendees observe the self-display screen. Arranged at a position facing each observation position, a projected image from the projector is synthesized using captured images from the plurality of imagers, and the projected image is After arranging the line images of the same order from the respective imaging devices in the order of the first line image, the first line image from the second imaging device, and arranging the line images of the same order from all the imaging devices, The line images in the next order are arranged and synthesized in the same manner as described above, and one set of stripes composed of the line images in the same order is included in one lens of the lenticular sheet. Is urchin projection, and, in each of the observation position of said display screen,
The input / output for image communication, wherein a focal length, a curvature and a lens pitch of a lens of the lenticular sheet are set so as to observe only an image of an image pickup device arranged opposite to each observation position of the other party. apparatus. 2. The input device for image communication according to claim 1, wherein a liquid crystal projector is used as the projector. 3. The image communication input / output device according to claim 1, wherein the projection onto the display screen is performed from a projector disposed behind the display screen.