JPH0494268A - Image pickup device in common use for display device and picture communication system - Google Patents

Abstract

PURPOSE:To attain conversation with a high presence in which visual lines between conversation parties are coincident with each other by allowing each of plural image pickup devices provided to pick up all the region where plural conversation parties are in existence opposite to each other with a display screen inbetween in the light transmission state. CONSTITUTION:A synchronizing circuit 55 generates a synchronizing signal 2 whose period is T1 with a delay T3 with respect to a synchronizing signal 1, where T2 is equal to a light transmission time of a display screen 52 and T3 is equal to a delay with respect to the synchronizing signal 1. The synchronizing signal 2 is fed to an image pickup device 53, which is operated with conditions of a period T1, a delay T3 and a light transmission time T2 synchronously with the operation of the screen 52. As a result, since the device 53 fetches a picture when the screen 52 is always in the light transmission state, a picture is obtained as if no display screen were in existence. Thus, the conversation of the conversation parties toward the screen 52 is equal to that toward the device 53. Thus, the conversation facing the display in which the visual lines of both the parties are matched with each other is realized.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to a display/photographing device for image communication such as a video telephone or a video conference, and an image communication method using the display/photographing device.

(Prior Art) Image communication devices such as videophones and video conferences are equipped with a display/photographing device for transmitting and receiving images in addition to a voice transmitting/receiving device, making it possible to create a communication device with a high sense of realism. .

Conventionally, the image display/capturing part of such communication devices includes:
A configuration in which a display device such as a CRT display and a photographing device such as a television camera are arranged closely together has been used.

(Problem to be Solved by the Invention) However, images sent and received by such communication devices do not match the shooting direction of the camera because the viewing interlocutor's line of sight is directed toward the display surface of the display device, and as a result, the images sent and received by the communication partner are The drawback was that it made it difficult to make eye contact and gave an unnatural feeling. Furthermore, when there are multiple interlocutors, it is difficult to make eye contact with each interlocutor.

Among these drawbacks, a small image communication device having a configuration as illustrated in FIG. 9, for example, has been proposed as a means for aligning the eyes of the former. This is done by installing a half-mirror-IA between the photographing device 2A (camera) and the display device 3A, and performing display and photographing from the same direction. ) to align the lines of sight.

However, this requires a large space because the half mirror IA, which is about the same size or larger than the display surface of the display device 3A, is installed at an angle, and therefore has the disadvantage that the device becomes large.

(Object of the Invention) The present invention solves such conventional drawbacks, and provides a display and photographing device in which the lines of sight of each of the plurality of interlocutors and a specific person match even when there are a plurality of interlocutors, and the display and photographing device. The purpose of the present invention is to provide an image communication method using a dual-camera imaging device.

(Means for Solving the Problems) In order to solve the above problems and achieve the objects, the invention described in claim (1) includes a projector that projects an image, and a combination of projection light from the projector. In an image display device consisting of a display screen that displays an image, a plurality of photographing devices are installed at positions facing the interlocutor who holds the display screen and observes the image, and a state of light transmission to the display screen is determined by electrical signals. a control device, the control device alternately switches the display screen between a light transmission state and a light non-transmission state, and when the display screen is in the light transmission state, each of the plurality of photographing devices is installed on the opposite side through the display screen. This display and photographing device is characterized in that it photographs the entire range where a plurality of interlocutors are present.

Furthermore, the invention as claimed in claim (2) provides a configuration in which the display/photographing device according to claim (1) is connected to each other via a communication line, and some interlocutors photographed with the display/photographing device. This is an image communication method characterized by combining other images into a display signal.

(Function) The invention described in claim (1) uses a display screen in which a light transmitting state and a light non-transmitting (scattering) state are alternately switched by an electric signal, and a plurality of display screens that observe an image by sandwiching the display screen. A plurality of photographing devices provided at positions facing the interlocutors photograph all the interlocutors when the display screen is in a light transmitting state. As a result, only the projected display image can be observed from the interlocutor, and from the imaging device it appears as if there is no display screen, and an image is obtained in which the lines of sight of multiple interlocutors and a specific person are aligned. .

In addition, the invention described in claim (2) cuts out a specific interlocutor from among the images of all interlocutors photographed by the display and photographing device,
By compositing them on the screen, it is possible to eliminate the mismatch in line of sight between multiple interlocutors and the unnatural feeling of the direction of line of sight.

(Embodiment) FIG. 1 shows a system configuration diagram of an embodiment of the present invention, in which display and photographing devices are installed facing each other. This is for ease of understanding, and in practice, a plurality of them will be installed as described later (FIG. 4).

First, in the figure, 4A and 4B are interlocutors who observe a display screen, and 5A and 5B are display and photographing devices of the present invention, and image projectors 5 that project the other interlocutor onto display screens 52A and 52B, respectively.] A.

51B, photographing devices 53A and 53B that photograph the interlocutor, drive circuits 54A and 54B that control the light transmitting state and non-light transmitting state of the display screens 52A and 52B, and photographing device 53A
, 53B and drive circuits 54A, 54B, a plurality of photographing devices (here, 53A,
It consists of image synthesis devices 56A and 56B that cut out a part of an image of a single device 53B, but there are a plurality of devices installed as shown in FIG. 4, and synthesize them.

Here, the image projector 51A is connected to the photographing device 53B, and the image projector 51B is connected to the photographing device 53A via screen compositing devices 56B and 56A, respectively, and the display screens 52A and 52B are connected to each other's dialogue. Persons 4B and 4A are displayed.

In the above-mentioned display screens 52A and 52B, display screens that can generate a light transmitting state and a non-transmitting state include a capsule-shaped liquid crystal cell in which a cell is formed by sandwiching a liquid crystal layer between transparent electrodes, a dispersed liquid crystal cell, and a liquid crystal cell in a liquid crystal cell. Examples include liquid crystal cells such as a scattering type liquid crystal cell. Other display screens that can generate light transmission and non-transmission states include PLZ
An example is a T-element in which a comb-shaped electrode is formed on an inorganic material having an electro-optical effect.

Dynamically scattering liquid crystal is a known material and will not be specifically explained, but it has recently been developed as a capsule-type liquid crystal, a dispersion-type liquid crystal plate, and a glass plate with a dimming function. , Nippon Sheet Glass's "instant dimming glass"
, "light control glass LCWJ" manufactured by Asahi Glass, and "light control liquid crystal sheet (ACT)" manufactured by Ajinomoto. In addition, Dainippon Ink Co., Ltd.'s Japan Display 'dispersed liquid crystal'
One example is rPN-LCDJ, which was presented at the 1989 International Conference. Further, some PLZT elements can undergo a phase change by applying a voltage to form a light scattering state.

FIG. 2 shows a basic configuration device 100 of the display and photographing device shown in FIG.
It shows a three-dimensional view of the image projector 51 and the display screen 52.
, a photographing device 53, a drive circuit 54, and a synchronization circuit 55, and the screen composition device 56 is omitted.

Next, the operation will be explained using the timing chart shown in FIG.

First, the synchronization circuit 55 generates a synchronization signal 1 with a period (T1) that does not cause the display screen 52 to flicker, and sends it to the drive circuit 54. The period T1 of the synchronization circuit 55 may be generated by the synchronization circuit itself, but may be generated by the frame period of the image (176
0 seconds, etc.) is more convenient.

Next, the drive circuit 55 generates a drive signal synchronized with the synchronization signal 1 to operate the display screen 52.

As a result, the display screen repeats a light transmitting state and a non-transmitting (scattering) state at a period T1. In this case, if the time in the light-opaque state is longer than the time in the light-transmissive state by more than a certain value, the display screen 52 enters the light-opaque state from the perspective of the interlocutor 4.

Next, it is assumed that the light transmission time of the display screen 52 is T2, and its delay with respect to the synchronization signal 1 is T3.

At this time, the synchronization circuit 55 generates a synchronization signal 2 having a delay of T3 with respect to the synchronization signal 1 and having the same period <TI. This synchronization signal 2 is sent to the photographing device 53, and the photographing device operates in synchronization with the operation of the display screen 52 with a period T1, a delay T3, and a light transmission time T2. As a result, since the photographing device 53 captures an image when the display screen 52 is always in a light-transmitting state, an image that appears as if the display screen does not exist is obtained. The opening/closing operation of this imaging device is
This can be done using mechanical and electrical/electronic shutters.

As a result, when the interlocutor faces the display screen 52 for dialogue, it means that he or she faces the photographing device 53, so that the dialogue can be performed while facing the display with the line of sight.

In this example, the image projector is operated continuously, and the projected image is turned on/off in synchronization with the display screen.
By performing the synchronized operation of problem is solved.

FIG. 4 is a three-dimensional diagram of a device configuration 200 in which four sets of basic component devices +00 explained in FIG. 2 are arranged. This device is the third
It operates in the same manner as the operation timing shown in the figure. Further, this device is designed for a case in which the number of interlocutors 4A and 4B in FIG. 1 is four or less. When this device is used, one interlocutor 4 can be set for one set of basic component devices 100 on one side of the device, as shown in FIG. As long as the user speaks facing the display screen 52, the user can maintain eye contact and have a conversation.

However, in this device configuration 200, as shown in FIG. 6, one of the interlocutors 4 (right end) interacts with a display screen 52 (left end) other than the front display screen 52, and the other party also interacts with the corresponding display screen ( (not shown), the images facing sideways to each other are transferred to the display screen 52, creating the problem that the lines of sight are no longer aligned.

This problem has been solved by the invention recited in claim (2) of the present invention, and FIG. 7 shows a basic layout configuration diagram thereof. The configuration of this device is basically the same as the device 200 in FIG. 4, but only the photographing device 53, display screen 52, and interlocutor 4 are shown, and the other devices shown in FIG. 1 and FIG. It is omitted. In this device 300, each photographing device 53 photographs all the interlocutors 4 within the range shown by the broken line, and the display screen 52
The photographing device 53 is installed relatively close to.

When this device 300 is used, when a particular person among the interlocutors speaks and that person is moved onto the display screen 52, all the persons on the side of the conversation will see the display screen. This state is shown by interlocutor 4 in FIG.
The image of the speaker is displayed on the right end of the display screen 52. In this case, when the image taken by the photographing device 53' (right end) is transmitted to the other party, the line of sight of all people will be aligned with the interlocutor 4 on the receiving side, as shown in FIG. In this way, by switching the screen to the image capturing device corresponding to the speaker and transmitting the image to the other party, it is possible to have a conversation with a good sense of realism. Methods for switching the screen of this photographing device include a manual method in which the speaker presses a switch before speaking, and a method in which the speaker's voice is detected and the screen is automatically switched.

In the explanation so far, we have assumed that the interlocutors are separated by 2 meters, but even if the interlocutors are separated in three or more locations, it will feel like all the interlocutors are in the same venue.
Furthermore, since line of sight matching is possible, the sense of realism is further improved. In this case, it is desirable that as many people as possible be shown on the screen by placing interlocutors from other locations in the empty seats, rather than transmitting the entire screen of the other party in one location as is.

In order to display such a screen, it is preferable to create a screen by cutting out only the human images of the participants in the conversation and composing the images of all the people on the screen. As a method for cutting out this human image, there are a method using a chromakey device and a method in which a background image is stored in advance and a newly added image is extracted as a human image. This is a method of synthesizing the person obtained through these methods with a suitable background image and transmitting it to the other party. These series of processes are then performed by the screen composition devices 56A and 56B in FIG.
will do.

The resulting image allows interlocutors in different locations to appear as if they are in the same location, as in FIG. 8, and allows for a highly realistic dialogue in which the speaker's line of sight coincides.

(Specific Example 1) A display/photographing device 200 having the configuration shown in FIG. 4 was manufactured using the following members.

The image projector 51 is a commercially available liquid crystal projector, the display screen 52 is a glass screen (manufactured by Nippon Sheet Glass, trade name: "Instant light control glass") in which a liquid crystal sheet is sandwiched between glasses, and the photographing device 53 is equipped with a wide-angle electronic shutter. A television camera (manufactured by Matsushita, model: WV-E73) was used to construct the image communication system shown in FIG.

The photographing and projection frequencies of the screen and the switching frequency of light transmission/non-transmission of the display screen were set to 60.

Regarding the drive signals for the display screens 52A and 52B,
The applied voltage was normally set to Ov to make the display screen non-light transmitting, and a pulse voltage of 100 V at a ratio of 60 for 1 to 5 milliseconds was applied to instantaneously create a light transmitting state.

Next, the delay between the synchronization signal of the drive signal and the synchronization signal of the television camera was adjusted so that the electronic shutter of the television camera was opened only during the time when the display screen transmitted the most light after the pulse signal was applied.

As a result, the front images of the four people on the other side were shown on the display screen, and when the screen shot by the camera after the display screen showing both people in the conversation was displayed, the two people in the conversation were able to see the same line of sight. It was possible to have a highly emotional dialogue.

(Specific Example 2) The same device used in Specific Example 1 is used at three or more locations, and the background of the interlocutor is set to, for example, blue, which is suitable for person extraction using chroma key. In this example, when the speaker is in the location where the government office is located, the speaker selects the camera so that the speaker's line of sight is fixed, and at other locations, the speaker selects the camera installed behind the display screen where the speaker is reflected, and inputs the image. . The necessary person image is cut out from these images and the image is displayed on the other party's display screen. As a result, the line of sight of the speaker coincides with the line of sight of the speaker, and even interlocutors located in other locations can see the speaker, making it possible to have a conversation with a high sense of realism.

(Effects of the Invention) As explained above, the display and photographing device according to claim (1) of the present invention switches between light transmission and light non-transmission of the display screen in synchronization with the television camera, so it is possible to communicate with the other party while displaying the image. This has the advantage that the line of sight can be made to match, and even in a situation where there are a plurality of interlocutors, the line of sight of the speaker can be selectively aligned, and an excellent sense of presence can be obtained.

Further, the image communication method according to claim (2) of the present invention cuts out a specific interlocutor from among the images of all the interlocutors taken by the display and photographing device, and synthesizes them on the screen. It is possible to eliminate mismatched gazes between interlocutors and unnatural gaze directions.

[Brief explanation of drawings]

FIG. 1 is a system configuration diagram of an embodiment of the present invention, FIG. 2 is a three-dimensional diagram of the basic components of the display and photographing device shown in FIG. 1, and FIG.
The figure is a timing chart explaining the operation of figure 2, and figure 4.
The figure is a three-dimensional view of the configuration device in which four display/photographing devices shown in FIG. FIG. 7 is a diagram showing the basic arrangement of the interlocutor, the display screen, and the photographing device according to the invention according to claim (2), and FIG. 9 is a diagram showing an example of the configuration of a conventional small-sized image communication device. 4.4A, 4B...interlocutor, 5A, 5B. 100, 200...display and photographing device, 51.51A
. 51B... Image projector, 52.52A, 52B.
・Display screen, 53.53A, 53B...Photographing device, 54.54A, 54B...Drive circuit, 5
5, 55A, 55B...Synchronous circuit, 56A. 56B...Screen composition device, 300...Basic arrangement of photographing device, display screen, and interlocutor. Patent applicant Nippon Telegraph and Telephone Corporation

Claims (2)

[Claims] (1) In an image display device consisting of a projector that projects an image and a display screen that forms an image of the projection light from the projector, a display device located at a position facing the interlocutor who holds the display screen in between and observes the image. A plurality of photographing devices are installed, and a device for controlling a light transmission state of the display screen by an electric signal, and the control device alternately switches the display screen between a light transmission state and a light non-transmission state. A display/photographing device characterized in that, in a light transmitting state, each of the plurality of photographing devices installed shoots an image of the entire range where a plurality of interlocutors on the opposite side are present through a display screen. (2) The display/photographing device according to claim (1) is configured to be interconnected via a communication line, and a part of the interlocutor and other images photographed by the display/photographing device are synthesized and displayed. An image communication method characterized by using a signal for communication.