JP3128018B2 - Image communication system - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image communication system, such as a video conference system, for holding a conference while viewing each other's face images. It relates to a system for use.

[0002]

2. Description of the Related Art There is an attended type video conference system that allows a user to hold a conference while simultaneously viewing a face of a user and a presented document with remote users by real-time communication of image information and audio information. In addition, each user can simultaneously write in a document presented in such a conference system,
There is a system that enables collaborative work between users distributed in remote places by adding a function to indicate the location of a document with a hand or a cursor ("Japanese group seeking a clue to popularization" Wear ", Nikkei Computer, March 11, 1991 issue).

[0003]

In such a conventional system, a face image of a user and an image of a hand for writing on a document or a document are displayed in separate windows as shown in FIG. It had been. for that reason,
These conventional systems have the following problems. (1) Two cameras, one for photographing a user's face image and one for photographing a document image on a desk, are required, which increases the cost of the terminal and complicates the configuration. (2) Since a face image and a document image are displayed in separate windows, a limited display screen space must be divided, and it is difficult to allocate sufficient space to each window. (3) Since images captured by the respective cameras are displayed in different windows on the display screen, a discontinuous boundary occurs between the face image and the document image, thereby increasing the user's cognitive load in terminal operation. I was

[0004] In communication while looking at each other's faces, there is already an apparatus that realizes the function of matching eyes using a half mirror or a liquid crystal shutter (Bill Bux).
tonand Tom Maron, “EUROPARC'S Integrated Intaracti
ve Intermedia Facility (IIIF): Early Experiences, Mul
ti-User Interfaces and Applications Elsevier Scien
ce Pubishers 1990, Shinichi Shiwa, Yuji Ohba, "Proposal of a new gaze matching display method", see the IEICE Autumn National Conference Papers, 1990). There is no such device.

The present invention has been made to solve the above problems, and an object of the present invention is to photograph an upper body image including a user's face and an image display surface with a single video camera.
By transmitting and displaying the image with the remote terminal at a remote location, the image on the drawing surface shared with the facial image of the remote partner is displayed as a continuous single image, and the entire display screen is viewed to see the facial image of the partner. Can also be used for collaborative drawing,
Another object of the present invention is to realize a communication means that can easily match the line of sight during the cooperative drawing.

[0006]

According to a first aspect of the present invention, there is provided an image communication system in which a user's face image and a document or a document are transmitted between image terminals connected by a communication line. An image communication system for performing a conference while simultaneously transmitting and receiving images such as charts and the like and simultaneously displaying the images and the like is provided by an image display means, an image display surface of the image display means within an imaging range, and a scene in front of the image display surface. An image photographing display comprising image photographing means for reflecting the image on the image display surface or photographing from behind through the image display surface, and display image photographing preventing means for preventing a display image of the image display means from being photographed by the image photographing means. Image control in which a device is a terminal and a scene in front of an image display surface photographed by an image photographing means of a partner terminal is displayed on the own terminal as a mirror image inverted left and right. It is those provided with the stage.

According to a second aspect of the present invention, there is provided an image communication system according to the first aspect, further comprising: a position detecting means for detecting a position on the image display surface designated by the designation means; A computer system for input processing the output of the position detecting means of the terminal, and an image synthesizing means for synthesizing the output image of the computer system and the output image of the photographing means are added, and the output image of the image synthesizing means is displayed on the image display means. Things.

An image communication system according to a third aspect of the present invention includes an image capturing and displaying device, an image display means, a half mirror superimposed on an image display surface of the image display means, and the half mirror. And an image photographing means for simultaneously photographing an image reflected by the surface and the half mirror, and a display image photographing preventing means for preventing a display image of the image display means from being photographed by the image photographing means.

Further, in the image communication system according to a fourth aspect of the present invention, the image capturing and displaying device is electrically controlled,
A liquid crystal screen which can be in a transparent or opaque state, an image projecting means located behind the screen for projecting an image on the screen, an image photographing means located behind the screen to fit the entire screen in a shooting range, and a transparent screen And control means for interlocking control so as to alternately take an opaque state and alternately take and project an image by the image taking means in synchronization with the state of the screen.

[0010]

According to the present invention, the image display means of the terminal device includes a drawing image drawn by the other user on the other terminal transmitted from the other terminal and a reflected image (for example, an image of the hand or face of the other user). Are displayed as a single image, and overlap with the drawn image that the user himself made on the image display surface of his / her own terminal device. From the user's position, the drawn image on the screen of each of the partner terminal and his / her own terminal is overlapped. Image and the image of the hand and face of the partner user appear as one image.

In the present invention, the photographing means covers the entire image display surface within the photographing range. However, since the display image photographing preventing means can prevent the display image from being photographed, the display image photographed by the image photographing means can be prevented. Is displayed on the partner terminal and is displayed again on the own terminal, so that the reflection of the screen does not repeat indefinitely.

When the image is taken through the image display surface, the image taken on the image display surface is a mirror image whose right and left are opposite to those seen from the position of the user. By controlling the display to be inverted, an image in the same direction as viewed from the position of the user is displayed.

Further, according to the present invention, the position on the display surface designated by the designation means is input to the computer system by the position detection means, subjected to arithmetic processing, and the output image of the computer system is taken by the photographing means of the partner terminal. Since the image is synthesized and displayed with the image, the computer system can be operated by giving an instruction on the display surface while watching the display surface, in addition to the above-described operation.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the principle of the present invention will be described with reference to FIGS. As shown in FIG. 2, an image communication system according to the first invention of the present invention transmits output images captured by image capturing means of each terminal to each other via a communication network, and displays image output means of a partner terminal. To be displayed. In FIG. 2, 10 is an image display means, 40 is a video camera as an image photographing means, 1
00 is an image communication terminal device, 200 is a communication network, 60 is a communication interface, and UA and UB are users.

When each user draws directly on the image display surface of the image display means 10 of the image communication terminal device 100 of the image communication system of FIG. 2 using a felt pen or the like, FIG.
As shown in (a), the image display means 10 of the image communication terminal device 100 displays on the image display surface the other party's video camera 40 taken by the video camera 40 through the image display surface or reflected on the image display surface. And the scene in front of the image display surface of the partner terminal, including the hand and face of the partner user, are displayed as one image P10.

Then, the image P10 and the user UA overlap with the drawn image P20 drawn on the surface of the image display means 10 of the user's own image communication terminal device 100, and from the position of the user UA, each of the partner terminal and his / her own terminal The image in which the drawn images on the image display surface overlap with each other and the image of the hand or face of the partner user appear as one image as shown in an image P30 in FIG. As a result, (1) an instruction operation using a finger or the like with respect to the drawn image of the other party, and (2) a comment operation with respect to the drawn image of the other party are instructed or written with respect to the image displayed on the own terminal. Accordingly, the work can be performed in real time, and the cooperative work using the drawn image on the screen can be performed. In addition, since the image of the user's face or hand is captured and displayed simultaneously with the drawn image formed on the image display surface through the image display surface or reflected on the image display surface, (3) the expression of the face and the body Communication using gestures can be realized. At that time, the image photographing means photographs the image passing through the image display surface or reflected on the display screen, so that when the user is facing the direction of the screen on which the other party's face is reflected, the user just faces the front. (4) Communication in line of sight is possible. According to the above-described means, face and hand images and drawing images can be displayed on one continuous screen, so that eye-to-eye communication by (3) and (4) is realized while performing cooperative work using (1) and (2). it can.

In the case where the image photographing means performs photographing by reflecting off the image display surface, the drawn image formed on the image display surface is photographed directly by the image photographing means. However, since the scene in front of the image display surface is once reflected and photographed, it is photographed as a mirror image in which the left and right are inverted from what the user has seen.

On the other hand, when the image photographing means photographs through the image display surface, the drawn image on the image display surface is photographed from the side opposite to the user's viewing side, so that it becomes a mirror image whose right and left are reversed, Scene becomes a right and left normal image. Thus, through the image display surface or reflected on the display surface,
When photographing the display surface and the scene in front of the display surface, one of the image of the front scene and the image of the display surface is a mirror image in which the left and right are reversed.

In order to perform a cooperative operation using the drawn image on the screen, a mirror image that is inverted left and right is inconvenient to read, for example, when writing characters, so that the drawn images viewed by each user are in the same direction. However, in the device of the present invention, by controlling the image control means so that the scene in front of the image display surface photographed by the image photographing means of the partner terminal is displayed on the own terminal as a mirror image inverted left and right. ,
With respect to the image on the display surface, each user can see an image in the same direction in which the left and right are not inverted.

[Embodiment 1] Next, an embodiment of the present invention will be described. FIG. 1 is a schematic diagram showing the configuration of the first embodiment of the present invention. This embodiment corresponds to the image communication system according to claim 1, wherein a CRT is used as an image display means, and a video camera installed above the image display means 10 as an image photographing means, and a video camera installed on the surface of the image display device. A case will be described in which a half mirror is used, and polarizing plates on the photographing side and the display side are used as display image photographing prevention means.

In FIG. 1, reference numeral 40 denotes a video camera;
Denotes a CRT, 31 denotes a display-side polarizing plate provided on the image display surface of the CRT 11, and 32 denotes a polarizing axis direction orthogonal to the polarizing plate 31 attached to the lens unit of the video camera 40 (transmitting through the polarizing plate). The direction of the electric field oscillation of the generated light) is a polarizing plate on the photographing side, and the display image photographing preventing means 30 is constituted by both polarizing plates 31 and 32. Reference numeral 20 denotes a half mirror installed on the polarizing plate. The video camera 40 and the half mirror 20 cover the entire image display surface of the image display means within a shooting range and display an image of a scene in front of the image display surface 10A. Image capturing means for capturing an image by reflecting the image on a surface is configured. 50 is an image transformation device.

In this embodiment, as shown in FIG.
The image display surface of the RT 11 is installed at an angle of about 45 degrees with respect to the vertical direction, and the video camera 40 is installed above the CRT 11 so that the entire image display surface 10A is within the shooting range while the scene in front of the terminal (user (Including the face, arms, etc.) as an image reflected on the half mirror 20 and transmitted to the partner terminal. The image transmitted from the partner terminal is displayed on the CRT 11. If each user UA, UB draws on the half mirror 20 using a felt-tip pen or the like, the user sees the face image in the image, and communicates with his / her eyes while drawing, and the drawn image in the same image. Or half-mirror 2
By drawing on 0, cooperative drawing essential for cooperative work can be performed. The same effect can be obtained even if the image transformation device 50 is inserted in a stage preceding the CRT 1 on the image receiving side.

In order to perform such cooperative drawing, it is necessary that the drawn image drawn on the half mirror 20 be displayed in the same shape and the same size on the partner terminal. Since the upper side of the drawn image on the half mirror 20 is closer to the video camera 40, the upper side of the drawn image is photographed larger than the lower side of the drawn image, and the length in the vertical direction is also reduced. As a result, FIG. The drawn image on the half mirror 20 having the shape shown in FIG. 4 is photographed in a distorted shape as shown in FIG. In the apparatus of the present embodiment, the output image of the video camera 40 is reduced by the image deforming device 50 at the upper side of the image, and the lower side of the image is enlarged and expanded in the vertical direction to remove the above-described distortion, thereby reducing the half mirror 20. It enables the drawing image to be reproduced on the partner terminal in the same shape and size.

When the video image taken by the video camera 40 is displayed on the CRT 11, the image is displayed on the CRT 11 of the partner terminal.
Is displayed on the video camera 40 of the partner terminal,
It is displayed again on the CRT 11 of its own terminal, and the reflection of the screen is repeated indefinitely. In this embodiment, in order to prevent the display image of the CRT 11 from being photographed by the video camera 40 and to prevent reflection, the image display surface 10A of the CRT 11 is used in the present embodiment.
A polarizing plate 31 is provided on the upper side, and a light beam output from the CRT 11 is filtered into only a specific polarization component. A polarizing plate 32 having a polarization axis direction perpendicular to the polarization plate 31 has only a polarization component orthogonal to the polarization plate. The image of the CRT 11 is prevented from being captured by the video camera 40 by utilizing the property of not transmitting the light beam filtered at 31. On the other hand, the drawn image on the half mirror 20 or the image reflected on the half mirror 20 is not filtered by the polarizing plate 31 into only a specific polarized light component.
The video is taken by the video camera 40 without being interrupted by the video camera 40.

In this embodiment, the scene in front of the image display surface is
The scene in front of the image display surface 10A is displayed as a mirror image inverted left and right by reflecting the image on the half mirror 20 on the image display surface 10A, and the image is shot by the video camera 40 of the partner terminal. The image control means for controlling the scene in front of the image display surface 10A so as to be displayed on the terminal as a mirror image inverted left and right is satisfied.

[Embodiment 2] FIG. 5 is a schematic diagram showing the configuration of Embodiment 2 of the present invention. In this embodiment, a liquid crystal shutter is used as the "display image photographing prevention means 30" instead of the polarizing plates 31 and 32 of the first embodiment.

In FIG. 5, reference numeral 33 denotes a display-side liquid crystal shutter provided on the CRT 11, reference numeral 34 denotes a photographing-side liquid crystal shutter provided on a lens portion of the video camera 40, and reference numeral 35 denotes drive control of the liquid crystal shutters. Device
These constitute the display image photographing prevention means 30. However, FIG. 5 shows only the terminal device of one user.
Other parts are the same as those in FIG.

In the apparatus of this embodiment, both liquid crystal shutters 33 and 34, which are constituted by liquid crystal panels which can be electrically controlled to be in a transparent or opaque state, are provided on the image display surface 10A of the CRT 11 and the video camera 40. The liquid crystal shutter 33 on the display surface side and the liquid crystal shutter 34 on the photographing side are mounted on the lens unit and synchronized with the photographing / display cycle of the video camera 40 and the CRT 11 by the drive control device 35.
By making the image transparent or opaque according to the time chart shown in FIG. 6, the image is not taken when the image is displayed on the CRT 11 (period T2 in the time chart of FIG. 6), but taken by the video camera 40. Is not displayed (the period T1 in the time chart of FIG. 6), and further, during the display period T2, the image is always transmitted from the partner terminal. For this purpose, the CRT 11 is synchronized with the partner terminal or stored in an image storage device so that it can be taken out and displayed at any time.
Is prevented from being photographed by the video camera 40.

[Embodiment 3] FIG. 7 is a schematic diagram showing the configuration of Embodiment 3 of the present invention. In this embodiment, a video projector and a translucent screen are used as the image display means instead of the CRT 11 of the first and second embodiments.

In FIG. 7, reference numeral 12 denotes a video projector, and 13 denotes a translucent screen for rear projection for forming a projection image of the video projector 12, and the other components are the same as those in FIG. However, in the present embodiment, an example is shown in which the polarizing plates 31 and 32 are used as the display image photographing prevention means 30. In this embodiment, the video projector 12 used for rear projection is used. However, when the video projector 12 is used for front photographing, it is needless to say that the image needs to be inverted left and right for projection.

In this embodiment, an image is displayed by projecting an image from the video projector 12 onto the translucent screen 13. Video camera 4 installed above
0 is approximately 4 diagonally as in the case of the first and second embodiments.
The front scene of the terminal reflected on the half mirror 20 on the image display surface 10A inclined at 5 degrees is photographed. In the case of the first and second embodiments, the video camera 40 has the half mirror 2
0 The upper part of the picture drawn on the upper part is photographed large and the lower part is photographed small, and contraction distortion occurs in the vertical direction, so the upper part of the image is reduced, the lower part of the image is enlarged, and the image is expanded in the vertical direction. In this embodiment, it is necessary to remove the distortion.
This need does not arise.
That is, by projecting from the video projector 12 whose optical axis is set in the horizontal direction to the translucent screen 13 inclined at 45 degrees to the vertical direction, the upper side of the screen is reduced because the video projector 12 and the translucent screen 13 are close to each other. On the other hand, the lower part of the screen is magnified due to being far away, and forms an image. And can be reproduced in size.

[Embodiment 4] FIG. 8 is a schematic diagram showing the configuration of Embodiment 4 of the present invention. This embodiment corresponds to the image communication system according to claim 2, wherein a CRT is used as image display means, an electromagnetic digitizer is used as the electromagnetic position detection means, and a plurality of input interfaces are provided for the digitizer. This is a case where a computer system including a single computer provided is used.

In FIG. 8, 90 is a digitizer, PA and PB are pen-shaped cursors, 70 is an image synthesizing means, 81A and 81B are input interfaces, 82 is an image output interface, 83 is an arithmetic processing unit, and 80 is the 81A. , 81B, 82, 83. Other parts are the same as those in FIG.

The electromagnetic digitizer 9 used in this embodiment
0 is a current flowing through a coil embedded in a transparent glass plate, and a pen-shaped cursor PA, used by the user as an indicator.
By interaction with the coil embedded in the PB,
The position of the pen-shaped cursor on the digitizer 90 is detected. The detected position information is stored in the computer system 80.
Is transmitted to the arithmetic processing unit 83 via the input interfaces 81A and 81B, and the arithmetic processing unit 83 performs arithmetic processing based on the input information, and outputs the result as an image signal via the image output interface 82.

The image synthesizing device 70 synthesizes the output image P40 of the computer system 80 and the image P50 transmitted from the partner terminal via the video network.
The CRT 11 displays an output image of the image combining device 70.

In the computer system 80,
By executing a cooperative drawing program for performing drawing based on information from a plurality of digitizers 90, cooperative drawing using a computer function is possible while observing the expression and behavior of the other party.

FIG. 9 shows an embodiment of the cooperative drawing program executed by the computer system 80. The cooperative drawing program used when the computer system of the fourth embodiment is constituted by two computers connected via a network. It is a flowchart which shows the processing procedure of a program. When the program is started, first, the state of the switches of the pen-shaped cursors PA and PB of the terminals operated by the user UA and the user UB is checked. When the switch of the pen-shaped cursor PA or PB is ON (that is, when it is determined that the user is performing drawing), the X and Y coordinates on the digitizer 90 of the pen-shaped cursor PA and PB are acquired. This is coordinate-transformed into the coordinate system of a computer display.

In the coordinate conversion, when an arbitrary coordinate of the digitizer 90 is (X, Y) and a mapping from the coordinate system of the digitizer 90 to the coordinate system of the computer display is f, a coordinate corresponding to (X, Y) is displayed on the display. (X ′, Y ′) is given by (X ′, Y ′) ← (X, Y).

Next, the present cooperative drawing program draws a point on the coordinates (X ', Y') on the computer display. By repeating the above, the users UA and U
B enables simultaneous and cooperative drawing.

If the image P50 transmitted from the partner terminal is synthesized by the chroma key method or the video burley method with the output image P40 of the computer executing the drawing program, the image is drawn in the same manner as the drawing by the pen in the first embodiment. 1) It is possible to perform cooperative drawing. 2) It is possible to obtain the effect that face-to-face dialogue with gaze coincidence is possible.

In this embodiment, the computer system 80 has been described as being constituted by one computer, but the computer system can be constituted by a plurality of computers connected via a network. Needless to say.

FIG. 10 is a flowchart showing a processing procedure in the case of the embodiment of the cooperative drawing program when the computer system is constituted by two computers connected via a network. In the present embodiment, the operation of the program on the user UA side will be described. When the program is started, first, the state of the switch of the pen-shaped cursor PA of the terminal operated by the user UA is checked. When the switch of the pen-shaped cursor PA is ON, X, X on the digitizer of the pen-shaped cursor PA
The Y coordinate is obtained, and the coordinate (X, Y) is transmitted to the computer on the user UB side connected via the communication line. Next, the coordinates (X, Y) are converted into a coordinate system of a computer display to obtain coordinates (X ′, Y ′),
A point is drawn at the coordinates (X ′, Y ′).

Next, the input from the terminal of the user UB connected via the communication line is checked, and if the input is the pen-shaped cursor PB, the X and Y of the pen-shaped cursor PB of the user UB are checked.
The coordinates are obtained, the coordinates are converted to the coordinate system of the computer display, (X ′, Y ′) is obtained, and a point is drawn at the coordinates (X ′, Y ′). By repeating the above, it is possible for the user UA and the user UB to simultaneously and cooperatively draw between the two terminals connected via the network.

As the position detecting means, in addition to the electromagnetic type digitizer 90 shown in the present embodiment, a pressure-sensitive type digitizer, a pen point is extracted by image processing from an image inputted by a video camera, and the position is detected. Needless to say, various devices for detecting coordinates, such as a device for performing such operations, can be used.

FIG. 11 shows the CRT 1 of the terminal device of this embodiment.
1 shows a configuration of a screen when a video projector and a translucent screen for rear projection are used instead of 1.

[Fifth Embodiment] FIG. 12 is a schematic diagram showing the structure of the fifth embodiment. In the present embodiment, the CRT 11 and the image synthesizing unit 70, which are the image display units of the fourth embodiment, are composed of a translucent screen 13 for rear projection and two video projectors 14 and 15. In FIG.
14 is a video projector on the video side, and 15 is a video projector on the computer side. In addition, FIG.
The terminal device on the user UA side is shown, and the terminal device on the partner user UB side is omitted. Other parts are the same as those in FIG.

The video projector 14 on the video side converts the image P50 transmitted from the partner terminal via the video network into the video projector 15 on the computer side.
Projects the output image P40 of the computer system onto the translucent screen 13. As a result, a combined image in which the images P40 and P50 overlap each other is displayed on the translucent screen 13. However, in this configuration, since light is additively mixed, when one of the video projectors 14 and 15 projects an image with high luminance, the other image is erased. To prevent this, the image P50 on the video side
The image P40 on the computer side needs to be arranged such that characters and lines are light and the background is dark in the image P40 on the computer side.

[Embodiment 6] FIG. 13 is a schematic diagram showing the configuration of Embodiment 6. In the sixth embodiment, an image signal switch device that selects and outputs one image signal from two image signals input as image combining means is used.

In FIG. 13, P50 is an image transmitted from the partner terminal via a communication line, P40 is an image output from the computer system, and 71 is the image P40, P
This is an image signal switch device that selects and outputs one of the signals 50 while switching at a high speed. FIG. 13 shows only the user UA side and omits the partner user UB side. Other parts are the same as those in FIG.

[Embodiment 6-1] In this embodiment, the user U
FIG. 1 shows image signal switching devices on the A side and the partner user UB side.
While switching as shown in FIG. 4A, two types of images are displayed alternately. In a period T1 in FIG. 14A, the image communication terminal device on the user UA side displays an output image P40 of the computer system. The video camera on the user UA side captures an image in which the displayed image P40 and the scene in front of the display surface overlap, and the image is transmitted to the user UB side and displayed.

In a period T2 of FIG. 14A, the image P50 transmitted from the user UB is displayed on the image display surface of the user UA. The video camera 40 of the user UA captures an image in which the image P50 and the scene in front of the display surface overlap, but in the period T2, the user UB displays the image of the computer system 80, so that this image is not displayed. Absent.

As described above, (1) the image P40 of the computer system 80 (2) the image P50 transmitted from the partner terminal (the image in which the output image P40 of the computer system 80 and the scene in front of the image display surface of the partner terminal overlap) Are displayed while alternately switching at high speed, these images appear to the user's eyes as a mixture, and the images can be combined.

In this embodiment, while one terminal is displaying the video image transmitted from the other terminal, the other terminal displays the image P40 of the computer system 80. A display image photographing prevention means for preventing infinite repetition of photographing and displaying again at the partner terminal and photographing is realized.

[Embodiment 6-2] Also, as shown in FIG. 14B, the operation of the image signal switch device 71 and the operation of the video camera 40 are controlled in conjunction with each other to synthesize images. Can be.

In a period T1 in FIG. 14B, an image P40 of the computer system 80 is displayed and the video camera 4
0 captures an image P60 in which the image P40 of the computer system 80 and the scene in front of the image display surface of the partner terminal overlap. In the subsequent period T2, the image P50 transmitted from the partner terminal is displayed, and the video camera 40 continues to output the image P60 in the previous period T1 without taking an image in this period. As described above, (1) the image P40 of the computer system 80 and (2) the image P50 transmitted from the partner terminal (the image in which the image of the computer system 80 and the scene in front of the image display surface of the partner terminal overlap) are alternately displayed at high speed. , The images appear to the user's eyes as a mixture, and the images can be combined.

By controlling in this way, the image transmitted to the partner terminal is always an image in which the image P40 of the computer system 80 and the scene in front of the image display surface reflected by the half mirror are overlapped, and the image transmitted from the partner terminal is transmitted. Since the displayed image is not taken, the display image taking prevention means is realized.

However, Examples 6-1 and 6-
In the second device, (1) the image P40 of the computer system 80 (2) the image P40 transmitted from the partner terminal includes the output image of the computer, so that the image P40 of the computer system 80 included in both is included. If the shape, size, and position of are not correctly matched, the image becomes unclear.

In the apparatus of this embodiment, the display image photographing prevention means is realized as shown in the embodiments 6-1 and 6-2.
A clearer image can be obtained by adding the display image capturing prevention means as shown in FIG.

FIG. 13 shows that the computer system 80,
Unless the coordinate detecting means is removed and an image is not input to the input of the image signal switch device 71 on the image P40 side of the computer system 80, the device of this configuration has a system such as the first embodiment of FIG. Function as the display image photographing prevention means 30 in FIG.

[Embodiment 7] FIG. 15 is a schematic diagram showing the configuration of Embodiment 7 of the present invention. This embodiment corresponds to the first to sixth embodiments.
In the following, a description will be given of a configuration method of the apparatus when the angle of the display surface of the image display means 10 is set to an angle other than 45 degrees.

In FIG. 15, reference numeral 10A denotes an image display surface on which a half mirror 20 is superimposed. P10 is a partner user image displayed on the image display surface 10A, UA is a user, and θ is an angle formed by the image display surface 10A with a horizontal plane. Point A is the position of the user's eyes, point B is the viewpoint of the user looking at the other user's image, point C is the position of the video camera, and point D is a point on the straight line AB that is on the opposite side of point B with respect to point A. It is.

The image display surface 10A is rectangular, and is installed so that the bottom is horizontal. Both ends of a line segment on the image display surface 10A passing through the point B and perpendicular to the bottom of the image display surface 10A are defined as points E and F as shown in FIG. Also,
One point on the normal line of the image display surface 10A passing through the point B is shown in FIG.
As shown in FIG.

In order to capture the drawn image written on the image display surface 10A and, at the same time, realize a conversation in which the line of sight coincides with the communication partner, the partner user image P10 on the image display surface 10A is required.
The video camera 40 needs to be positioned on the extension of the user's line of sight (line segment AB) (for example, at the position of point D in FIG. 15). In this embodiment and the first to sixth embodiments, instead of placing the video camera 40 at the position of the point D, the video camera is placed at the point C which is a plane symmetric position with respect to the image display surface 10A. An image equivalent to a mirror image of the video camera placed on D is displayed on the image display surface 10.
A can be taken as an image reflected on the half mirror 20 superimposed on A.

First, a case will be described in which the apparatus is configured so that the user's line of sight (line segment AB) is horizontal. Angle of incidence ΔAB of the line of sight of user UA with respect to image display surface 10A
Let E be θ2. Since ∠DBF and ∠ABE are diagonal to each other, ∠DBF = ∠ABE = θ2. Since the video camera may be placed at a position (point C) symmetrical to point D with respect to the display surface, the position is determined at a point such that ∠CBF = θ2 on a plane including a normal BG passing through point B. , So that the entire display surface can be photographed. When the user's line of sight (line segment AB) is horizontal, if the magnitude of the tilt angle of the image display surface 10A is θ1, then θ2 = θ1 (relationship between complex angles).

FIG. 16 is a schematic diagram of a side surface of the apparatus of this embodiment in which the inclination θ1 of the display surface is variously changed in order to show the range of possible values of θ1 in this embodiment. If the video camera is placed at the point C as described above, the drawn image written on the image display surface 10A can be photographed, and at the same time, the dialogue with the communication partner can be realized, but the inclination of the image display surface 10A can be realized. When θ1 is large as shown in FIG. 16A, the portion of the image display surface 10A that cannot be photographed by the video camera 40 due to the shadow of the user is unsuitable.

If the inclination θ1 of the image display surface 10A is small and equal to or less than 0 °, the image display surface 10A cannot be photographed due to the shadow of the display surface, which is inappropriate. Even when θ is 0 degree or more, the value is small, and FIG.
In such a case, the distortion of the image on the image display surface 10A taken by the video camera 40 becomes large, so that the image quality deteriorates and is inappropriate. The value of θ1 needs to be a value that is not so large as to cause the inconvenience as shown in FIG. 16A and not so small as to cause the inconvenience as shown in FIG. 17B.

Next, when the apparatus is configured so that the user's line of sight (line segment AB) is not horizontal, for example, FIG.
The case where the display surface is arranged so that the user looks down at the screen as shown in FIG. In this case, the relationship of θ1 = θ2 does not hold, and both can be set independently.

As in the case of FIGS. 16 (a) and 16 (b), the angle formed horizontally by the image display surface 10A is set to the above θ1, and the viewpoint A of the user and the eye position B of the other user in the image are connected. The angle ∠ABE between the straight line and the image display surface 10A is θ2. In order to capture the drawn image written on the image display surface 10A and, at the same time, realize a dialogue with the communication partner with the same gaze, as in the case of FIGS. 16A and 16B described above,
The video camera 40 needs to be at the position of point C in FIG.

In this configuration, by increasing θ2, distortion of the image on the image display surface 10A captured by the video camera 40 can be reduced. When both θ1 and θ2 are large, as shown in FIG. 16A, the portion of the image display surface 10A that cannot be photographed by the video camera 40 due to the shadow of the user's body becomes large, which is inappropriate. Is smaller and θ2 is larger, the user cannot advance his body to the area in front of the lower end of the device shown by the shaded dots in the figure, so that the image display surface 10A is shaded by the camera by the user's body. Therefore, it is possible to adopt a configuration in which the distortion of the photographed image on the image display surface 10A is suppressed to be small, and at the same time, the image display surface 10A is prevented from being behind the body of the user.

In the case where cooperative drawing is realized by superimposing a drawing image in an image transmitted from a partner terminal and a drawing image drawn on the own terminal as in the first embodiment, The image on the image display surface 10A of the terminal had to be accurately reproduced on the image display means 10 of the terminal itself. However, when realizing cooperative drawing by the computer system 80 as in the fourth embodiment, a felt pen or the like was used. When the used drawing is not performed, since the coordinate information is accurately input and transmitted by the position detecting means, the image transmitted from the partner terminal and the image P40 of the computer system 80 do not necessarily have to overlap exactly. , Some errors are allowed. Therefore, by adopting a configuration in which θ1 is small and θ2 is large as shown in FIG. 16C, if distortion of the image on the image display surface 10A to be photographed is suppressed to a small value, the fourth to sixth embodiments can be realized.
Then, the necessary image transformation device 50 can be omitted.

[Embodiment 8] FIG. 17 is a schematic diagram showing the configuration of an embodiment 8 of the present invention, and corresponds to claim 4.
In this embodiment, a liquid crystal screen which can be in a transparent or opaque state by being electrically controlled by a video projector is used as the image display means 10, and the entire liquid crystal screen is placed behind the liquid crystal screen as an image photographing means. This is a case where a video camera set to fit in a video camera is used.

In FIG. 17, 12 is a video projector, 40 is a video camera, 16 is a liquid crystal screen, 17
Is the video projector 12 and the video camera 40
And a liquid crystal screen drive control device that controls the liquid crystal screen 16 in an interlocked manner. Other parts are the same as those in FIG.

In the present embodiment, shooting and display are performed in synchronization with alternately making the liquid crystal screen 16 transparent or opaque according to a time chart as shown in FIG. During the period T1 when the liquid crystal screen 16 is transparent, the video camera 40 captures a scene in front of the liquid crystal screen 16 as an image display surface through the screen surface.

During the period T2 when the liquid crystal screen 16 is opaque, the video projector 12
Is projected onto the liquid crystal screen 16 to form and display an image on the liquid crystal screen 16. Thus, (1) shooting through the image display surface and (2) display of an image are alternately performed.

In the present embodiment, the liquid crystal screen 16 is made transparent during the period T1 during which the video camera 40 takes a picture, so that the display image is prevented from being taken by the video camera 40, and the display picture taking prevention means is provided. Has been realized.

In the first to seventh embodiments, the image display surface 1
Since the image is photographed obliquely with respect to 0A, the image display surface 10A is photographed with a trapezoidal distortion.
However, in the apparatus according to the present embodiment, since the photographing is performed in the direction perpendicular to the image display surface, no distortion occurs and the image deforming apparatus 50 is unnecessary.

In this embodiment, the system including the computer system 80 has been described.
The method of this embodiment using the video projector 12 and
Obviously, the present invention is also applicable as a means for image display and photographing in a system as shown in FIG.

[Embodiment 9] FIG. 19 is a block diagram showing a configuration of a ninth embodiment of the present invention. This embodiment describes a case where three or more persons use the image communication system of the present invention.

In FIG. 19, 100-1, 100-
2, 100-3 are image communication terminal devices of the present invention, P1, P
2 and P3 are output images of each image communication terminal device 100;
Reference numerals -1, 70-2, and 70-3 denote image synthesizing devices that superimpose two images translucently, P11, P12, and P13 denote input images to the respective image terminals, and U1, U2, and U3 denote users.

In this embodiment, in order to display a plurality of opponents on the image display means, images of the plurality of opponents are displayed on the image synthesizing device
The images are synthesized in a semi-transparent shape by 0-1, 70-2, and 70-3, and are superimposed and displayed. Referring to FIG.
For example, the input image P11 displayed on the image display means 10 of the user U1 is obtained by combining output images P2 and P3 of two other users (users U2 and U3) in a semi-transparent manner.

In this embodiment, a video effector used in a broadcasting station or the like is used as an apparatus for synthesizing in a semi-transparent state. For example, optical synthesis is performed using two video projectors. Needless to say, various methods are possible.

By displaying the image synthesized in this way, the image including the faces and bodies of a plurality of users and the drawn image drawn on the half mirror 20 of the terminal are superimposed and displayed on the image display means. Therefore, it is possible to perform a cooperative drawing and a line-of-sight dialog with a plurality of users. Although FIG. 19 particularly shows the case of three persons, it is needless to say that the image communication to which the present invention is applied is also possible in the case where the number of persons is more than three by synthesizing the image of the communication partner.

[0083]

As described above, according to the present invention,
The distributed users are shared on the screen, can cooperate while performing an instructing operation and a comment operation on the drawn image in real time, and can realize communication in line of sight.

Further, since a picture image and a face image on the screen can be simultaneously photographed as one continuous image by one video camera, the terminal can be constructed at low cost and includes a face image and a hand image. The cognitive load of the user at the time of terminal operation that existed when the drawn image is displayed on a separate screen can be removed, and the entire display screen can be used to see the face of the other party and to perform collaborative drawing, so it is limited Space can be used effectively.

Further, since the present invention uses the computer system for input processing of the output of the position detection of each terminal, it is possible to perform a cooperative operation using a computer in addition to the above effects.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing a configuration of a first embodiment according to the present invention.

FIG. 2 is a diagram schematically illustrating an image communication system for explaining the principle of the present invention.

FIG. 3 is a diagram for explaining the principle of the present invention, and is a schematic diagram for explaining that an image transmitted from a partner terminal and a drawn image made by a user on a screen of his / her own terminal appear to overlap; is there.

FIG. 4 is a diagram showing how a drawn image is distorted when captured by a video camera in the first embodiment of FIG. 1;

FIG. 5 is a schematic diagram illustrating a configuration of a second exemplary embodiment of the present invention.

FIG. 6 is a driving time chart of a liquid crystal shutter according to a second embodiment of the present invention.

FIG. 7 is a schematic diagram illustrating a configuration of a third embodiment of the present invention.

FIG. 8 is a schematic diagram illustrating a configuration of a fourth embodiment of the present invention.

FIG. 9 illustrates a computer system according to a fourth embodiment of the present invention.
11 is a flowchart illustrating a processing procedure of a cooperative drawing program used when configured by one computer that receives inputs from two digitizers.

FIG. 10 is a flowchart illustrating a processing procedure of a cooperative drawing program used when the computer system according to the fourth embodiment of the present invention is configured by two computers connected via a network.

FIG. 11 is a diagram showing a configuration of a screen when a video projector and a translucent screen for rear projection are used instead of a CRT in Embodiment 4 of the present invention.

FIG. 12 is a schematic diagram illustrating a configuration of a fifth embodiment of the present invention.

FIG. 13 is a schematic diagram illustrating a configuration of a sixth embodiment of the present invention.

FIG. 14 is a time chart illustrating operations of a video switch and a video camera according to Embodiment 6 of the present invention.

FIG. 15 is a schematic diagram showing a configuration of a seventh embodiment of the present invention.

FIG. 16 is a diagram illustrating an appropriate range of the inclination angle of the display surface according to the seventh embodiment of the present invention.

FIG. 17 is a schematic diagram illustrating a configuration of an eighth embodiment of the present invention.

FIG. 18 is a time chart showing the operation of the liquid crystal screen of the eighth embodiment of the present invention.

FIG. 19 is a block diagram showing a configuration of a ninth embodiment of the present invention when used by three persons.

FIG. 20 is a diagram illustrating an example of a screen of a conventional image terminal.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 10 Image display apparatus 10A Image display surface 11 CRT 12 Video projector 13 Translucent screen 14 Video projector 15 Computer video projector 16 Liquid crystal screen 17 Liquid crystal screen drive control circuit 20 Half mirror 30 Display image taking prevention means 31 Polarizing plate on display surface 32 Polarizing plate on photographing side 33 Liquid crystal shutter on display surface 34 Liquid crystal shutter on photographing side 35 Drive control device 40 Video camera 50 Image transformation device 60 Communication interface 70 Image synthesizing device 71 Image signal switch device 72 Image Synchronous control device between signal switch device and video camera 80 Computer system 81A Input interface unit of computer system 81B Input interface unit of computer system 82 Computer system Arithmetic processor systems out of the image output interface unit 83 computer system 90 Digitizer 100 image communication terminal apparatus 200 network

Continuation of the front page (56) References JP-A-2-228893 (JP, A) JP-A-4-81090 (JP, A) JP-A-4-81089 (JP, A) JP-A-2-63393 (JP) JP-A-4-321395 (JP, A) JP-A-4-94268 (JP, A) JP-A-4-145788 (JP, A) JP-A-1-276892 (JP, A) 2-192382 (JP, A) JP-A-4-323974 (JP, A) JP-A-4-86161 (JP, A) JP-A-63-102483 (JP, A) (58) Fields investigated (Int. H04N 7/10 H04N ^7/ 14-7/173 H04N ^7/ 20-7/22

Claims (4)

(57) [Claims] In an image communication system in which an image is mutually transmitted and received between image terminals connected by a communication line and a conference is held while simultaneously displaying images, an image display means and an entire image display surface of the image display means are photographed. Image capturing means for capturing the scene in the range and reflecting the scene in front of the image display surface to the image display surface or from behind through the image display surface; and a display image of the image display means being captured by the image capturing means. An image capturing and displaying device comprising a display image capturing preventing means for preventing the terminal from being displayed, so that a scene in front of the image display surface captured by the image capturing means of the partner terminal is displayed on the own terminal as a mirror image inverted left and right. An image communication system, comprising: image control means for controlling. 2. The image communication system according to claim 1, wherein a position detecting means for detecting a position on the image display surface specified by the specifying means, and an input processing of an output of the position detecting means of each terminal. And an image synthesizing means for synthesizing the output image of the computer system and the output image of the photographing means, and displaying the output image of the image synthesizing means on the image display means. 3. An image communication system according to claim 1, wherein said image capturing and displaying device comprises: an image display means; a half mirror superimposed on an image display surface of said image display means; An image capturing means for simultaneously capturing the surface of the mirror and an image reflected by the half mirror, and a display image capturing preventing means for preventing a display image of the image display means from being captured by the image capturing means. Image communication system. 4. An image communication system according to claim 1, wherein the image capturing and displaying device is electrically controlled, and can be in a transparent or opaque state.
Image projection means for projecting an image on the screen positioned behind the screen, and image photographing means for positioning the whole screen within a photographing range behind the screen, wherein the image control means makes the screen transparent and opaque. An image communication system, wherein the image communication system is controlled so as to alternately take a state, and is interlocked so as to alternately take and project an image by the image taking means in synchronization with the state of the screen.