JPH11289523A - Bidirectional interactive system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an ideal counseling environment by providing a reader reading a medium recording prescribed items required for personal authenticating of a terminal user so as to realize detailed interactive environment and secrecy between interacting persons through the use of background music and character graphic information in addition to the video sound of the interacting persons. SOLUTION: The secrecy of interaction is assured by recording reservation information, information for personal authentication of the user of a second terminal being an interaction opposite party and information shared only between respective interacting persons such as an interacting condition as the prescribed items in an external storage medium 289 in the shape of a card with at least IC. A reader 271 reads these information and after line-connecting with a second terminal retrieved through a communication network based on interactive reservation data, bi-directional communication is started between first/second terminals to exchange data. Psychological preparation to the introduction of interaction is arranged by utilizing information of a picture, sound, background music, characteristic graphic data, etc., within a terminal equipment 26 with each other.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a two-way interactive system using transmission means, and more specifically, to a one-to-one correspondence, such as a relationship between a doctor and a patient at a considerable distance, a relationship between a counselor and a client, and the like. The present invention relates to a system for facilitating the start, progress, and end of a two-way dialogue.

[0002]

2. Description of the Related Art Conventional counseling in the fields of education, medical care, psychology, and the like is generally performed in a situation where interlocutors face each other directly, while building a close mutual trust between them. However, those who have counseling skills with those who need counseling due to natural disasters, remote places, etc. (Counselors)
There are situations where it is difficult to share the same place and time with the same. Thus, with the development of today's communication technology, the need for a system for realizing counseling between persons in remote locations has increased.

In order to construct such a counseling system using communication technology, in order to establish a close relationship and an ideal counseling environment between interlocutors as in the conventional face-to-face method, First, it is important to match the eyes of the interlocutors.

In view of the above, a conventional image pickup apparatus that matches the line of sight between interlocutors via a transmission line is disclosed in
-269128 (the first prior art). In the first prior art, a half mirror provided between the interlocutor and the image capturing apparatus is provided, so that light emitted from the interlocutor serving as a subject is transmitted to the image capturing apparatus and the image display apparatus. Are separated. Therefore, according to the device configuration of the first prior art, the imaging device is arranged at a position avoiding the display screen of the image display device viewed by the interlocutor, and the input optical axis of the imaging device and the line of sight of the interlocutor. And match.

Secondly, Japanese Patent Application Laid-Open No. Hei 4-213287 (second prior art) uses a special display device capable of switching between reflection and transmission of light, and a rear portion of the display device (this display device is referred to as a second display device). An imaging device having a configuration in which an imaging device is arranged on the side opposite to an interlocutor) is disclosed. According to this device configuration, the line of sight of the interlocutor is directed to the imaging device arranged behind the display device and therefore behind the display screen of the display device viewed by the interlocutor. It is possible to match eyes.

Thirdly, Japanese Patent Application Laid-Open No. 4-977663 (third prior art) discloses that a small mirror or a small half mirror is mounted on the upper part of a display device, and a small camera whose optical axis is directed to the mirror is used as an interlocutor. There is disclosed an imaging apparatus configured to capture images. In this device configuration, the image of the interlocutor reflected by the mirror is captured by a small camera arranged at a position avoiding the display screen of the display device seen by the interlocutor. Therefore, the elevation angle of the line of sight of the talker is suppressed within 10 ° which is the allowable range of the line of sight coincidence in the vertical direction, and the line of sight of the talker is achieved.

[0007]

As described above, the conventional two-way interactive system has a variety of problems. However, the two-way interactive system is an intimate two-way interactive system, that is, as an element for realizing an ideal counseling environment. It is a system that allows the eyes of the person to match. However, it is empirically clear that it is extremely difficult to achieve an ideal counseling environment and obtain a sufficient psychological effect using only such hardware elements.

[0008] Although transmission means such as a public telephone line may be used as a medium for transmitting information between interlocutors, maintenance of security (protection of privacy) is completely considered in the above two-way interactive system. Absent.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and integrally processes audio (including background music (BGM)), character and graphic information, etc., in addition to the video of the interlocutor. The goal is to create an organic relationship between the interlocutors (realize close communication) and to provide an ideal counseling environment.

[0010]

According to the present invention, there is provided a first terminal installed at a first point where a first interlocutor is located, and a first terminal which is located at a considerable distance from the first interlocutor. A second terminal device installed at a second location, and at least two-way communication of image / audio data of the first and second interlocutors between the first and second terminal devices. The invention relates to a two-way interactive system with a transmission means for enabling it.

In the bidirectional system according to the present invention, the first
Before the start of the dialogue, the terminal device and / or the second terminal device include a reading device for reading the predetermined items from an external recording medium in which predetermined items including information necessary for personal authentication of the terminal user are recorded in advance. Prepare.

The external recording medium is preferably a card-shaped external storage medium provided with an IC or the like that can be carried by the interlocutor. For example, an optical card, IC card, optical disk, or magneto-optical disk can be used. It is. In addition, the external storage medium includes, as predetermined items, reservation information, information for performing personal authentication of a user of the second terminal device as a conversation partner, a reserved conversation start time, and a reserved conversation start time. More preferably, conversation conditions such as time and conversation contents are recorded. Such information shared only between each interlocutor is stored in the form of an external recording medium by each user (interlocutor).
Keeps the confidentiality of the dialogue.

[0013] Also, in the interactive interactive system according to the present invention, at least the first terminal device may be provided from an external recording medium in which predetermined items including a reserved conversation start time and conversation reservation data relating to a conversation partner are recorded in advance. A reading device for reading the predetermined item is provided, and after being connected to the second terminal device searched through a communication network as a transmission unit based on the dialog reservation data, the second terminal device is connected to the second terminal device. The configuration may be such that two-way communication is started between them.

In general, on a telephone or the like, a conversation between the two parties cannot be established unless the other party is located or a specific telephone is designated. However, with the above-described configuration, a two-way conversation can be realized based on information shared only between each of the interlocutors without specifying the location of the interlocutor and the terminal to be used.

Further, in the interactive interactive system according to the present invention, at least the first terminal device includes audio information, characters,
A reading device for reading a predetermined item from an external recording medium such as a CD-ROM or an optical card, in which a predetermined item including at least one of a graphic and an image is recorded, is recorded on the external recording medium. Part of the information, the second
The configuration may be such that the information is displayed on the monitor of the first terminal device at least at any stage before, during, and after the interaction with the interlocutor.

Such information such as characters, figures, and images generally includes information that can provide a relaxation effect,
Information such as corporate advertisements is also included. In other words, the same information is provided every time, so that the interlocutor can prepare for the introduction of the dialogue before the start of the dialogue, even if his location and psychology are different from those of the previous dialogue. This is because the same mental state can be led every time, and after the dialogue, psychological preparation for returning to the normal activity state can be made, and a psychological relaxation effect can be obtained.

Further, the relaxation effect may be acoustic information composed of at least one of background music and voice, in addition to the visual information as described above. This acoustic information is also recorded on an external storage medium such as a CD-ROM or an optical card. A part of the acoustic information recorded on the external recording medium is transmitted to at least the terminal user at least at any stage before, during, and after the conversation with the second interlocutor. Is output.

The above-described visual information and acoustic information are used as control information to be displayed and output on the second terminal device, at least before the start of the dialogue with the second interlocutor, during the dialogue, and after the end of the dialogue. At any stage, the data may be transmitted to the second terminal device via the transmission unit.

In addition, the two-way interactive system according to the present invention also has a structure for matching the eyes of the interlocutors via a transmission line in order to realize a smooth two-way dialog. That is, in this system, at least the first terminal device defines a display unit for displaying an image of the second interlocutor received on the monitor via the transmission unit, and at least a distance between the first interlocutor and the display unit. For this purpose, the apparatus comprises a position defining means for defining the position of the first interlocutor, and an imaging device for transmitting the photographed image of the first interlocutor to the second terminal device via the transmitting means.

Here, the image pickup apparatus includes at least an image pickup unit for picking up an image of the first interlocutor; A support mechanism for setting the imaging unit at a predetermined position between the monitor and the monitor. In particular, the imaging unit is a monitor on which the imaging direction of the imaging unit is directly directed to the first interlocutor, and a monitor image of the second interlocutor is displayed when viewed from the first interlocutor. The monitor is installed so as to be located within the image display area and above the position of the eyes of the monitor image. The second
The terminal device may have a similar configuration, and the first terminal device is configured to be able to remotely control the second terminal device. Furthermore, each terminal device is provided with a sound absorbing member which is arranged around the interlocutor and has a shielding property against external sounds. Therefore, the psychological obstacle to the leakage of the contents of the dialogue to the outside is reduced, and the sound quality is improved.

As described above, the system in which the image pickup unit is installed so as to block a part of the monitor image in the above-described line of sight between the interlocutor and the monitor is configured by the interlocutor. Even when the imaging unit is installed so as to block a part of the human image displayed on the monitor, the presence of the imaging unit is bidirectional if the size of the imaging unit is suppressed to a certain value or less. This is because the inventor has confirmed through experiments that it has little effect on the dialogue.

In the above two-way interactive system, the parallax that matches the line of sight of the interlocutor (first interlocutor) with the line of sight of the person (second interlocutor) displayed on the monitor (monitor image). The angle is 4.5 ° or less in the left-right direction of the monitor screen and 1 ° in the direction directly above the monitor screen with respect to the line of sight of the interlocutor.
It is preferable that the direction of 2 ° or less and the direction directly below is 8 ° or less. Further, the parallax angle is set such that the horizontal direction and the vertical direction of the monitor screen are each 3 ° or less with respect to the line of sight of the interlocutor when viewed from the interlocutor. Is more preferable. In particular, as shown in FIG. 8, the angle formed by a line perpendicular to the photoelectric conversion surface 208 of the imaging device 205 and passing through the photoelectric conversion surface 208 and the first interlocutor is within the range of the parallax angle described above. It is.

It is preferable that the line-of-sight matching area in which the imaging unit is installed is an area higher than the line-of-sight of the photographer. In particular, as a result of experiments by the inventor, the imaging unit that blocks between the interlocutor and the monitor is preferably located near the top of the human image displayed on the monitor as viewed from the interlocutor as the installation position of the imaging unit. Was confirmed.

A distance from the interlocutor specified by the position defining means to the monitor, a predetermined parallax angle at which it is determined that the line of sight of the interlocutor matches the line of sight of the person image displayed on the monitor; The means for installing the imaging unit in the area defined by
Means for moving the imaging unit itself within the line-of-sight area between the interlocutor and the person image on the monitor, and secondly, moving the interlocutor facing the monitor to form a person image on the monitor. Means for moving the line-of-sight matching area itself to relatively change the position of the imaging unit and the line-of-sight matching area. Third, moving the person image on the monitor, and similarly, the line of sight of the interlocutor facing the monitor. And means for moving the line-of-sight matching region itself of the human image on the monitor (this third means is effective as an algorithm for moving the video display area where the human image is displayed when the display control unit divides the screen. ).

The first means includes a processing unit for recognizing a position of a human image displayed on a monitor on the monitor in the terminal device. This can be realized by providing a drive mechanism for relatively moving the human image whose position is recognized by the processing unit. As an embodiment, the first means is performed for each terminal device, and the position of the monitor image of the second interlocutor is recognized by the processing unit of the first terminal device. The second
In some cases, the first terminal device remotely controls the second terminal device by transmitting it to the terminal device.

The second means is provided with a processing unit for recognizing a position on the monitor of the person image displayed on the monitor in the terminal device. This can be realized by providing a mechanism for adjusting the line of sight of the interlocutor photographed with respect to the line of sight in the position defining unit. As an embodiment, the first means is performed for each terminal device, and the position of the monitor image of the second interlocutor is recognized by the processing unit of the first terminal device. The second
In some cases, the first terminal device remotely controls the second terminal device by transmitting it to the terminal device.

Further, in order to realize the third means,
The terminal device includes a processing unit that adjusts a display position of a human image displayed on a monitor. The processing unit specifies a virtual position on the monitor image where the imaging unit should originally exist as a reference point based on the human image received and displayed on the monitor, and then the imaging unit actually Based on the coordinates on the monitor corresponding to the installed position and the coordinates of the obtained reference point on the monitor, the displacement of the human image displayed on the monitor is detected. If it is determined that this person image is deviated from the display position on the monitor where it should be displayed, the coordinates of the reference point on the monitor are used as the starting point, and the monitor corresponding to the installation position of the imaging unit is used. A vector having the above coordinates as the end point is calculated, and the shifted human image is translated on the monitor based on the obtained vector. As an embodiment, a case in which the process is performed on a monitor image of the dialogue person (initial setting before the start of the dialogue)
May be performed on the monitor image of the second interlocutor. In particular, during a screen division operation (including an enlargement / reduction operation of a displayed person image) by a display control unit to be described later, the display size of the person image (the monitor image of the first interlocutor or the second interlocutor) is arbitrarily set. When changed, the area for displaying the image can be set at an arbitrary position on the monitor screen. Therefore, when the sight line between the arbitrarily changed human image and the interlocutor is matched, The third means is particularly effective.

The above-mentioned reference point is used to detect the respective display positions on the monitor of both eyes of the human image received and displayed on the monitor, and from the center of both eyes to above the monitor. The position on the monitor is separated by a predetermined distance (for example, about 10 cm). Further, the reference point is configured to binarize the human image received and displayed on the monitor, detect the outline of the human image, and specify the outline as a position on the monitor corresponding to the top of the human image. Is also good.

The drive mechanism for moving the image pickup unit once recognizes the position of the interlocutor in the processing unit, and sets the angle of the image pickup direction of the image pickup unit based on the recognized position of the interlocutor and the position of the image pickup unit. It may be configured to adjust. By adjusting the angle of the imaging unit by this drive mechanism, the imaging direction of the imaging unit is automatically directed to the line of sight of the interlocutor whose position has been recognized by the processing unit. Further, the drive mechanism for moving the imaging unit detects an average match in the line-of-sight direction, and when moving and aligning, the moving speed is equal to or less than a predetermined value, and the centering operation is not performed when the fluctuation in the line-of-sight direction is within a predetermined range. (This determination is made by a processing unit or the like). In addition, when the imaging unit moves, the image output from the imaging unit shakes. However, the movement speed is limited, and the movement is not performed frequently and the movement is suppressed to the minimum necessary. .

Here, in the above-described drive mechanism for moving the imaging unit, it is preferable that a part of a member located between the interlocutor and the display unit is formed of a transparent material.

Next, the position defining means for defining the position of the interlocutor includes a height-adjustable chair (chair provided with a seat height adjusting mechanism) installed at a predetermined distance from the display unit. Also,
A table provided between the interlocutor and the display unit also functions as a position defining unit. In this system, the position defining means is used to set the distance between the photographed interlocutor and the display unit by 0.5 m or more.
m or less. With such a configuration, the movement of the interlocutor in the front-rear and left-right directions can be sufficiently restricted by the position defining means. This is because vertical movement adjustment is most effective.

Note that the imaging unit adapted to the system may be independent of the display unit, for example, as shown in FIG. Further, as shown in FIG. 7, the display unit may be attached to a part of the display unit by a support mechanism.

The two-way interactive system according to the present invention comprises:
Since it is aimed at a counseling system between distant interlocutors, it has various configurations and functions in order to collect useful counseling data and realize an ideal counseling environment.

For example, a display control section for controlling display on the display section has a function of enlarging / reducing an image displayed on the display section. Since the display size has a great effect on the psychological effect of the interlocutor, the display control unit can freely set the size of the video display area on the monitor. Environment can be realized. Since the display position of the reduced person image (monitor image) on the monitor is adjusted by the above-described third line-of-sight matching means, the line-of-sight matching state between the first interlocutor and the second interlocutor is maintained. Is done.

The transmission information may be character / graphic information in addition to the image and voice of the interlocutor. This makes it possible for the display unit to display a character or a figure together with a person image based on the character / graphic information received via the transmission unit. That is, by displaying not only images and sounds, but also characters and figures, the dialogue can be further facilitated.

As a method of displaying characters or figures, the characters or graphics may be displayed so as to be superimposed on the displayed human image. Further, the monitor screen may separate an area for displaying characters or figures from an area for displaying a human image. As a mode of this screen separation, (1) the monitor screen is divided in the vertical direction, and a person image is displayed on the upper side, or (2) the monitor screen is divided in the horizontal direction, and the monitor screen is divided in one direction side or in the center. A mode such as displaying a human image is preferable. Note that the inventor has confirmed that the eye-gaze coincidence between the interlocutor and the human image does not deteriorate even in the modes (1) and (2). As an application example of this aspect, the display unit may have a plurality of configurations, such as including a first monitor for displaying a person image and a second monitor for displaying characters, graphics, and the like.

Further, the system includes a character / graphic input device as a man-machine interface. With this configuration, the display unit can display a character or graphic input from the character / graphic input device via the character / graphic input control unit, and can display a character or graphic corresponding to the character or graphic thus taken into the device. It is also possible to output information or graphic information to the terminal device of the other party. Note that such a character / graphic input device may be provided only in one of the terminal devices at the first point and the second point where the interlocutor is located.

As described above, since the system is provided with the character / graphic input device, it is possible to carry out a conversation with characters and / or figures, thereby enabling a smoother conversation. In addition, as such a character / graphic input device, a keyboard device, a pen input device (touch panel) arranged near an interlocutor can be used.
Can be adopted. Further, by providing both such a keyboard device and a pen input device so that the keyboard device and the pen input device can be appropriately selected, a smoother conversation can be realized. Further, the character / graphic input device may include a pointing device for inputting a pointer position pointing to a displayed character or graphic. The position indication by the pointer makes it possible to specify the conversation target at each time. In addition, by using such a character / graphic input device or the like, by recognizing the voice of the input interlocutor and converting it into character information and displaying it, it is effective for establishing a smooth dialogue (or counseling).

The two-way interactive system includes a speaker arranged as a voice output unit, which is located on the side of the interlocutor and near the ear. This is because being whispered in the ear improves intimacy. A typical example of the audio output unit is a headphone.

Next, the system includes a telephoto microphone as an audio input unit. A microphone with a table or attached to the chest can be used as the microphone, but a microphone with a table may impair consciousness of conversation. Also, if the chest microphone is too far away, the cord may be damaged. Therefore, it is desirable to arrange a telephoto microphone which does not require the presence of the telephoto microphone at the top of the display unit or to incorporate the microphone into the display unit. In particular, since the gaze is already recognized,
By directing the sound collection direction to the mouth, the directivity can be increased.

The audio / sound data processing unit for taking in an audio signal from the audio input unit has an audio input amplification adjustment function for controlling a sound collection volume. Here, the voice / sound data processing unit automatically adjusts the output volume of the collected voice to a predetermined level. The voice / sound data processing unit is configured to detect an average level of the collected sound volume in a predetermined time and to automatically adjust the sound volume when a difference between the collected sound volume and the average level is equal to or more than a predetermined value. May be.

At this time, the display unit displays the output volume (volume of the interlocutor output to the other party) in the voice / sound data processing unit. If you are unfamiliar with talking to a person on the screen without being able to see the microphone directly, you will be confused as to what volume to use if you are unfamiliar with it. It is planned. In addition, by displaying the output volume of the interlocutor on the monitor screen and allowing the interlocutor to recognize the output volume, it is possible to feed back to the interlocutor whether the output volume is appropriate for transmission.

On the other hand, the audio / sound output control unit for controlling the audio output unit outputs background music (BGM),
It has a function of detecting the conversation volume and adjusting the output volume of the background music. Here, the voice / sound output control unit detects that the conversation volume is substantially zero for a predetermined time, and based on the detection result, at least starts the background music, adjusts the volume of the background music, and adjusts the background music. Adjust the volume of music. Further, the voice / sound output control unit detects the volume of the voice of the interlocutor and adjusts the volume of the background music so as not to hinder the hearing of the interlocutor's voice, thereby impressing the interlocutor's heart. The calming background music may be superimposed on the voice of the other party's interlocutor and output from the voice output unit. In particular, if background music is started or the volume is changed when silence continues, the tension of the interlocutor can be released and the interlocutor can be calmed down.

Further, in the two-way interactive system,
Each terminal device includes a printer for printing a display screen of a monitor, and a storage unit for storing information on the display screen. The storage means is a card-shaped recording medium provided with an IC (integrated circuit), for example, an optical card, an IC card, an optical disk, or a magneto-optical disk can be used.
It is possible to save information during the dialogue, particularly information on characters and figures used during the dialogue.

In order to maintain the security of the system, it is preferable that each photographing device includes a person authentication unit for confirming the person of the interlocutor. This is because the security of the system can be sufficiently maintained by starting the interactive operation after the person confirmation. Here, the person authentication unit may be configured to read the authentication data from a storage medium in which the authentication data is written from a dedicated reading device, and perform the person confirmation based on the read result. Note that the storage medium is
A card-shaped recording medium provided with an IC or the like, for example, an optical card, an IC card, or a magnetic card can be used. Furthermore, this person authentication unit performs (1) person confirmation based on the analysis result of the face of the interlocutor, (2) person confirmation based on the voice analysis result of the voice of the interlocutor, or (3)
The person confirmation may be performed based on the character analysis result of the signer's signature character, and (4) the person confirmation may be performed based on the sentence analysis result of the message text of the conversation person. Such person authentication makes it possible to specify the interlocutor.
It is preferable to use the authentication method properly according to the confidential content. The above-described text analysis of the message text means, for example, that the text “X is Y.” is compared with the characters put in X and Y and the database of the person or the encryption database of the specified person. Refers to collation analysis.

Further, by using these card-shaped external recording media such as IC cards, reservations of dialogues, authentication of a designated partner's interlocutor, reserved dialogue time, preset dialogue time, etc. Authentication and control of conversation conditions may be performed. Of course, it can be used as a means for electronic payment of fees. In addition, if the recording medium is used for recording the contents of dialogue and homework items, it is convenient for the next counseling and communication. If data such as the conversation start time and the conversation partner (partner party) is input in advance to a recording medium such as an IC card, a search for the partner terminal device on a network such as an electronic bulletin board system is performed based on the data. Can be performed, and based on the search result, the first and second interlocutors are connected to each other at any of the first and second points no matter where they are located, and the first and second interlocutors are connected. It is possible to build a dialogue environment between interlocutors. Further, the name, face (image data), and conversation items (the subject of the conversation, etc.) of the other party's interlocutor are input to the recording medium (IC card, etc.), and these information are stored for a predetermined period (before the start of the dialogue). By displaying the information on the monitor of the interlocutor for only (for example, about 30 seconds), the interlocutor can be mentally prepared for the dialogue of the interlocutor, and can easily enter the subject of the dialogue. At this time, if BGM, voice, and the like are output in conjunction with each other, it is possible to provide a more comfortable conversation environment for the user. It is effective that the above-mentioned monitor display is performed during and / or after the dialogue as necessary. In particular, by performing the above-described monitor display after the end of the dialogue, the interlocutor can smoothly proceed with summarizing the dialogue and the like (the time of the monitor display after the end of the dialogue is different for the interlocutor from the end of the dialogue). It is a mental preparation period until work).

Next, the above-mentioned table as the position defining means is arranged between the interlocutor and the display unit such that one end faces the interlocutor and the other end substantially contacts the display unit. .
Here, the table is set to a height where (1) the interlocutor is higher than the table above the chest when sitting in a chair,
(2) The height above the chest may be set to be higher than the table with the interlocutor standing. With such a configuration, a sense of continuity is generated via the table, so that anxiety caused by losing the relative distance (psychological distance) from the other party is reduced. In this case, it is desirable that the display unit and the table are in contact with each other, but there may be a gap (for example, several tens cm or less) that does not cause a psychological obstacle.

Next, in the two-way interactive system, the terminal device includes an interactive processing unit for statistically analyzing the interactive time of the interactive user. Here, the interaction processing unit calculates the total interaction time, the average interaction time, and the number of interactions, and displays the total interaction time, the average interaction time, the number of interactions, and the distribution of the number of interaction times on the display unit. You may do it. Furthermore, the voice / sound data processing unit in the terminal device performs statistical analysis processing on the voice spectrum of the interlocutor or the voice spectrum corresponding to the transmitted voice information. Here, the statistical analysis processing refers to analyzing the voice spectrum of the interlocutor or the voice spectrum corresponding to the transmitted voice information with time, and detecting an average spectrum and a voice spectrum that has changed by a predetermined value or more. .

Next, each terminal device of the system is provided with a dialogue processing unit for detecting a predetermined term issued by the interlocutor or a predetermined term in the voice corresponding to the transmitted voice information. In addition, the terminal device may include a position change analysis unit that analyzes a movement trajectory, a movement change amount, a movement speed, or a movement acceleration of a head, an upper body, or a hand of a person or a transmitted person image ( This analysis is performed by a processing unit or the like).

When counseling or the like is performed, a dialogue state analysis such as a dialogue time analysis, a voice spectrum analysis, a predetermined term analysis, or a body motion analysis can be performed to quantify the dialogue state.

In the transmission means in the interactive interactive system, a plurality of transmission paths independent of each other are secured between a first point and a second point separated from each other, and at least one of the plurality of transmission paths is provided. Is an audio transmission path for transmitting audio information. Here, the transmission means includes an already laid network such as a telephone line network, and furthermore, an electronic bulletin board system (BBS: Bulletin Board S) provided with various databases.
ystem). In addition, the system includes a backup system that detects a failure in a transmission line that is being used interactively and switches the transmission line to be used. Therefore, when a failure occurs in a transmission path during a conversation, the conversation can be continued using another transmission path. In such a case, it is preferable to secure at least the transmission of voice in another transmission path. In general, it is considered that a line abnormality is caused by an abnormality of a coaxial cable, an optical fiber cable, or a transmission device.

[0052]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a two-way interactive system according to the present invention will be described below with reference to FIGS.

FIG. 1 is a logical configuration diagram showing the entire interactive interactive system according to the present invention. This system uses A
Interlocutors 3A (first interlocutor) and B at point (first spot)
This is a system that realizes a two-way conversation with the interlocutor 3B (second interlocutor) at the point (second point). In particular, since this system is oriented toward a counseling system that establishes a close relationship of trust and achieves in-depth communication even with remote interlocutors, at least one interlocutor is a teacher, doctor, or psychologist. Such counselors are assumed. Therefore, this system has various functions and configurations so that these counselors can provide information necessary for counseling and an ideal counseling environment.

More specifically, at point A, (a) a chair 4A on which a talker 3A (for example, a counselor) sits;
A table 2A to which an interlocutor 3A goes;
7 and a terminal device 20A provided with a mechanism for matching the line of sight between the interlocutors. On the other hand, at the point B, (a) a chair 4B on which the interlocutor 3B (for example, a client who receives counseling) sits; (b) a table 2B to which the interlocutor 3B goes; And a terminal device 20B provided with a mechanism for matching the line of sight of the user. Here, the transmission unit 27 includes an optical fiber transmission line 31 that enables large-capacity transmission in order to enable transmission and reception of image information and audio information between the terminal device 20A at the point A and the terminal device 20B at the point B. (Main transmission path)
And a satellite communication transmission line 32 (backup transmission line). The transmission means 27 may be wired or wireless. The transmission means includes an already laid network such as a telephone line network and an electronic bulletin board system (BBS: Bulletin Board) provided with various databases.
System).

At the point A, the chair 4A described above functions to define the position of each interlocutor 3A. This chair 4A
Is preferably fixed to maintain a constant distance from the display unit 21A of the terminal device 20A. However, even when the chair 4A is not fixed, the table 2A is installed on the interlocutor 3A and the display unit 21A, so that the chair 4A is not fixed.
It is possible to keep the distance between A and the display unit 21A constant (an experiment by the inventor has confirmed that the existence of the table 2A serves as a reference for the psychological distance of the interlocutor 3A).
The same applies to the configuration of the terminal device 20B at the point B where the other party's interlocutor 3B (for example, a person receiving counseling) is located.

At the point A, the terminal device 20A (1)
A monitor TV1 arranged on the table 2A for displaying an image of the interlocutor 3B based on image information transmitted from the terminal device 20B at the point B via the transmission means 27.
A display section 21A provided with A, and (2) an image of the interlocutor 3A captured by the CCD camera 5A is captured as image information.
The imaging unit 6A for transmission to the terminal device 20B at the point, and (3) the voice of the interlocutor 3B based on the voice information transmitted from the terminal device 20B at the point B via the transmission means 27. Output unit 23A (speaker) and (4) a voice input unit 24A (microphone) for collecting the voice of the interlocutor 3A as voice information and transmitting it to the terminal device 20B at the point B. Furthermore, in order to realize more detailed communication as a counseling system, (5) the interlocutor 3A inputs characters and figures, and the input characters and figures are displayed on the display unit 21A.
And a character / graphic input unit 25 for transmitting the character / graphic information to the terminal device 20B at the point B.
A (an interface such as a keyboard, a pointing device, a touch panel, etc.) and (6) a processing unit 26 that is provided between each of the above components and the transmission unit 27 and performs signal processing and transmission control. The terminal device 20B at the point B is configured similarly to the terminal device 20A at the point A described above.

Further, the terminal device 20 at the point A and the point B
In A and 20B, the configuration of the imaging devices 600A and 600B will be described. For convenience of description, when referring to components common to the terminal devices 20A and 20B at the point A and the point B, for example, as in the imaging device 600,
The description will be made by omitting letters A and B for distinguishing each point. In addition, unless otherwise described, it is assumed that the terminal device 20A at the point A is described in principle,
The duplicate description of the terminal device 20B at the point B having the common configuration is omitted.

The imaging apparatus 600 includes an ultra-small CCD camera 5 (5A) as an imaging section, and a support mechanism 10 (a first mechanism) for supporting the CCD camera 5 at a predetermined position while supporting the CCD camera 5.
An embodiment is shown in FIG. 2), and an imaging unit 6 (6A) for controlling the CCD camera 5. The monitor TV1 (1A) is placed on the table 2 (2A), and the interlocutor 3 (3A) moves a distance L from the monitor TV1.
(M) Sitting on a chair 4 (4A) provided with a height adjustment mechanism 40 (40A) placed away. This distance L (LA)
Is set to 0.5 m or more and 5 m or less. An outer diameter of 20 mm or less is set between the person 3 to be photographed and the monitor TV 1.
A cylindrical microminiature CCD camera 5 having a length of about 100 mm is provided. The CCD camera 5 is installed so that its imaging direction is directed to the person 3 to be photographed. Then, the image of the interlocutor 3 captured by the camera 5 is transmitted from the imaging unit 6 to the other interlocutor side as image information (transmitted to the terminal device 20B at the point B via the transmission means 27). Note that the interval between the display unit 21 and the interlocutor may be slightly shifted depending on the interlocutor. Therefore, in order to cope with such a situation, it is preferable to select a CCD camera 5 having a large depth of focus.

Next, a specific installation position of the micro CCD camera 5 will be described with reference to FIG.

First, the image pickup unit 6 of the image pickup apparatus 600 is installed on the display section 21 for displaying the transmitted image of the other interlocutor. The main body of the imaging unit 6 displays an image of the interlocutor photographed by the camera 5 (taken into the imaging unit 6 via the camera cable 50) and a transmitted image of the other interlocutor 3B. In addition to the changeover switch 6a for switching, various adjustment switches 6b are provided. Switch 2
Reference numeral 10 includes a power switch for turning on / off the power of the monitor TV1. The CCD camera 5 is installed by a support mechanism 10 (robot arm) at a predetermined position in front of a monitor screen 8 on which the transmitted image 9 of the other talker 3B is displayed. The monitor screen 8 is 4 inches or more and 4
The screen is about 0 inches.

More specifically, the CCD camera 5 is located at a position w (cm) ahead of the monitor screen 8 and
Is arranged in the vicinity of the head 9a of the interlocutor image 9 displayed in.
The central axis of the cylindrical CCD camera 5 is located at a position h (cm) above the eye position of the interlocutor image 9 indicated by a dotted line.

As described above, since the ultra-small CCD camera 5 is located near the head above the position of the eye of the interlocutor image 9 displayed on the screen 8 of the large monitor TV 1, it is specially used for two-way conversation. No trouble occurs. For example, the CCD camera 5 is provided at about h (about 10 cm) above the eye position (the position shown by the dotted line in the figure) of the interlocutor image 9 displayed on the screen 8, and is provided between the monitor TV 1 and the interlocutor 3. In the case of a system configuration in which the distance L is set to about 2.5 (m), the parallax angle is 2.3 °.
(The distance w between the monitor screen 8 and the CCD camera 5 is 10 (c).
m) does not particularly affect the fluctuation of the parallax angle). In other words, as a result of the experiment, if the opponent's eyes (the eyes of the image of the other interlocutor 3B whose monitor is displayed on the surface 8) are clearly visible at the time of eye gaze coincidence, the microscopic camera Even if 5 was provided, it was confirmed that if the screen 8 of the monitor TV 1 was large, there would be little hindrance to the two-way conversation. The size of this screen 8 is 35
It has been confirmed that a good two-way conversation can be realized if the size is about 26 cm or 26 cm or more. If the mutual interlocutors are acquaintances, there is also obtained a result that the size of the screen is small but the psychological obstacle is also small, and the screen size may be appropriately used depending on the application.

Next, the structure of the robot arm which is the camera support mechanism 10 of the image pickup apparatus 600 shown in FIG. 1 will be described with reference to FIG.

This robot arm comprises a first rod 101
a, a second rod 102a, and a holder 103a that holds the camera 5. The main body of the imaging unit 6 and the first rod 101a are connected to the first joint 10
0, the first rod 101a and the second rod 102a
Is connected to the second joint portion 101, and the second rod 102a and the holder portion 103a are connected to the third joint portion 10.
2 are rotatably connected. In particular, the third joint portion 102 includes a protrusion 103 directly rotatably connected to the holder portion 103a to enable special rotation. The first joint part 100 is rotatable in a direction indicated by an arrow 100b, and the first rod 1
The movement in the direction indicated by arrow 100c of 01a is enabled. The second joint portion 101 supports the second rod 102a and enables the second rod 102a to move in a direction indicated by an arrow 101b. Third joint part 1
02 enables the projection 103 to move in the direction indicated by the arrow 102b. Further, the protrusion 103 and the holder 103
a can change the angle in the direction indicated by the arrow 103b. The above configuration of the robot arm (first embodiment of the support mechanism) enables the camera 5 to be installed at a desired position, and also allows the angle of the camera 5 to be adjusted. Therefore, the setting position of the CCD camera 5 and the imaging direction 5a can be freely set by the robot arm.

Next, as an application example of the image pickup apparatus 600, the structure of an image pickup apparatus 601 provided with another support mechanism (second embodiment) will be described with reference to FIGS.

The camera supporting mechanism of the image pickup apparatus is the talker 3B.
Is displayed in front of the surface 8, it is preferable to use a very inexpensive light-transmitting material such as a transparent thread, a tape, a transparent plate of glass, acrylic or the like. This is because the display of the monitor screen 8 is not hindered.

The image pickup device 601 shown in FIG.
A support member 110 made of a light-transmissive material for fixing and supporting the camera 5 at one end and installing the camera 5 at a predetermined position in front of the monitor screen 8;
And a pedestal portion 61 for gripping the other end of the transparent support member 110 and supporting the entire transparent support member 110. In particular, the pedestal 61 is provided with the imaging device 601.
The circuit board 6c which is housed inside the imaging unit 6 and realizes functions such as a data processing unit is also housed in the imaging unit 6. In the drawing, reference numeral 62 denotes an electric signal for driving and controlling the pedestal 61 itself from the circuit board 60 for moving the installation position of the CCD camera 5 from the circuit board 60 (FIGS. 5 and 6).
Are a plurality of wires (ribbons) to be communicated to.

The horizontal and vertical movements of the CCD camera 5 are realized by the mechanism shown in FIG. That is,
The pedestal 61 supporting the transparent support member 110 is installed on the first moving table 63a, and the position in the horizontal direction is defined by the wire 64. Accordingly, when the first motor 65 pulls the wire 64 in either direction, the pedestal 61 moves in the direction indicated by the arrow 63c. Further, the first moving table 63a is also the first moving table 63a.
It is installed on the second moving table 63b together with the motor 65, and the position in the vertical direction is defined by the wire 67. Therefore, the second motor 66 is connected to the wire 67
Is pulled in either direction, so that the entire first moving table 63a including the first motor 65
Move in the direction indicated by d.

On the other hand, the angle of the CCD camera 5 is adjusted as shown in FIG.
Is realized by the mechanism shown in FIG. That is, the base 6
Reference numeral 1 denotes a pair of plate members 68a and 68b and a distance adjusting unit 69 for defining a distance between these plate members. At this time, the transparent support member 110 is pushed up by the spring member 70 in the direction shown by the arrow 70a. In such a configuration, according to a control signal (electric signal) output from the circuit board 60 via the wiring 62 connected by the connector 62a, the interval adjusting unit 69 moves the plate member 68a in the direction indicated by the arrow 69a, By adjusting the interval of 68b,
The angle of the transparent support member 110 is adjusted in a direction indicated by an arrow 110a. In other words, by adjusting the angle of the transparent support member 110 in the direction indicated by the arrow 110a,
The angle of the imaging direction 5b of the CCD camera 5 can be changed.

The display unit 21 may employ a plasma display or a liquid crystal display instead of the monitor TV. When such a display device is used, FIG.
As shown in (1), a structure in which the CCD camera 200 is incorporated in the housing 210 of the display device in advance is preferable. 7 is a monitor screen 22 supported by a housing 210.
0, a through hole 230 is provided at a predetermined position.
A structure is provided in which the tip of the D camera 200 is directly penetrated, and the tip is located in front of the monitor screen 220.

FIG. 8 shows a specific structure. FIG. 8 is a diagram showing a cross-sectional structure of the display device taken along line XX of FIG. As shown in the figure, the barrel 2 of the CCD camera 200
01 is provided with a spherical projection 202 so that it can be attached to the housing 210. An imaging device 205 and an imaging optical system 203 are incorporated in the lens barrel 201, and an electric signal detected by the imaging device 205 is transmitted from the electronic circuit unit 206 to an external data processing unit via the camera cable 500. Can be Also, the lens barrel 201
Has a structure in which at least one of the lenses constituting the imaging optical system 203 can be moved in the axial direction of the lens barrel 201. The focus of the CCD camera 200 is controlled by an adjustment knob 204. Will be

In the two-way interactive system according to the present invention, a line 5e that is perpendicular to the photoelectric conversion surface 208 of the imaging device 205, which is an imaging unit, and passes through the photoelectric conversion surface 208, Is within the range of the parallax angle described above. This line segment 5e is
Substantially coincides with the imaging direction. Then, the camera 200 attached to the display unit centers on a predetermined portion of the lens barrel 201 in a direction indicated by an arrow 207 in the figure in a direction indicated by an arrow 207 (one line 5e in the figure). Is moved, the imaging direction of the camera 200 (line segment 5 in the figure) is changed.
e).

The CCD camera 5 having the above structure
Is a gripping member 2 directly attached to the housing 210.
40 and 250 are attached by gripping the projection 202 of the CCD camera 5. Since the projection 202 of the lens barrel 201 has a spherical shape, the angle of the CCD camera 200 can be freely changed.
The imaging direction 5c of the camera 200 can be directed to a desired direction. FIG. 9 is a view showing an assembly process for realizing the structure shown in FIG. In this figure,
The grip members 240 and 250 are bolts 252, 253 and 25
4 fixed. Further, in the drawing, reference numeral 255 denotes a through hole for bolt insertion provided in the gripping member 250.

FIG. 10 is a diagram showing a terminal device in which the CCD camera 5 and the display unit are integrally formed. FIG.
The terminal device has a camera driving mechanism 2 that drives the CCD camera 200 in the direction (up, down, left, and right) indicated by the arrow 260a.
60 are provided. With this camera driving mechanism 260, the imaging direction 5d of the CCD camera 200 can be changed to the direction shown by the dotted line in the figure. The image captured by the CCD camera 200 is captured by the captured data processing unit 60 and sent to the other terminal device 20B (point B) via the transmission unit 27. In this embodiment, the control of the camera driving mechanism 260 is performed by the imaging data processing unit 6.
Performed at 0. Furthermore, in this embodiment, the imaging data processing unit 60 is configured in the processing unit 26,
It may be configured independently as shown in FIG.

Next, the parallax angle will be described with reference to FIG. In this specification, the parallax angle is, as shown in FIG. It refers to the angle θ between the line of sight of the interlocutor 3. In other words, it means a shift angle of the installation position of the CCD camera 5 with respect to the line of sight of the interlocutor 3. In addition, the line-of-sight coincidence referred to here means a joint conference of the Institute of Electrical Engineers of Japan in 1972 (No. 19).
98), it means below the allowable limit of unnaturalness due to the installation position of the CCD camera 5. Quantitatively, the parallax angle is 4.5 ° or less in the left-right direction (image 9).
Of the image 9 with respect to the center A of the image 9), 12 ° directly above (the head 9b side of the image 9 with respect to the image center A), 8 ° directly below (with respect to the image center A). Nineth body side) and below are within the line-of-sight matching range. Of course, the smaller the parallax angle, the better, and the detection limit is 3 ° or less in the left-right and up-down directions. Therefore, the area where the CCD camera 5 can be installed is
13 is the conical area shown in FIG. Note that this conical area corresponds to the eye 30 of the interlocutor 3 and the image 9 displayed on the monitor screen 8.
(The center of both eyes of the image 9 is defined as the center point A of the image 9 in this embodiment).
The eye-gaze matching area 9b on the monitor screen 8 at a predetermined distance L (m) away from the eye gaze 31
0 position.

On the other hand, when the CCD camera 5 is set within a certain area with respect to the eye position (eg, image center A) of the image 9 of the interlocutor 3B at the point B displayed on the monitor screen 8, for example,
There are the following three means for placing the image 9 about 10 cm above the eye position.

The first means is to move the CCD camera 5 itself with respect to the monitor screen 8. In order to realize the first means, the interlocutor 3 displayed on the monitor screen 8
An imaging data processing unit 60 (for example, as shown in FIG. 10) that performs image recognition processing on the position of the image 9 of the interlocutor 3A or 3B at the point A or the point B in the monitor screen 8. A driving mechanism (for example, a driving mechanism for moving the CCD camera 5 so that the CCD camera 5 is set in the above-described predetermined area in the monitor screen 8 with respect to the position-recognized image 9)
The robot arm shown in FIG. 3 and the mechanism shown in FIGS. 5, 6, and 8) are provided in the imaging unit 6.

The second means is to move the person 3 to be photographed with respect to the monitor screen 8. This second
Means is that the imaging data processing unit 60 operates the monitor screen 8
The position of the image 9 of the interlocutor 3 displayed on the monitor screen 8 in the monitor screen 8 is subjected to image recognition processing, and the interlocutor 3 photographed with respect to the position-recognized image 9 of the interlocutor 3 itself
This can be realized by providing a person position defining means for adjusting the position of the person. As the person position defining means, for example, a chair 4 on which the interlocutor 3 sits is preferable. Adjustment of the seat height for each interlocutor is possible by adjusting the seat height adjustment mechanism 40 provided in the chair 4. This is because the installation position of the chair 4 (distance from the monitor screen 8 and the like) is fixed in advance, so that once the interlocutor 3 sits on the chair 4, it is not necessary to adjust the position of the interlocutor 3 in the left-right direction. . In other words, the only uncertain factor for determining the position of the interlocutor is the vertical displacement caused by the physique difference between the interlocutors. Therefore, by adjusting the sitting height for each interlocutor, the second Means can be realized.

The third means is to copy the image 9 of the interlocutor 3 (which may be any one of the interlocutors 3A and 3B at the point A and the point B) displayed on the monitor screen 8 to a CC whose installation position is fixed in advance.
It is moved relatively to the D camera 5. This third means, like the above means, first recognizes the position of the image 9 in the monitor screen 8 by the imaging data processing unit 60 provided in the imaging unit 6, and then displays the image 9 The position needs to be moved. This allows
The CCD camera 5 can be relatively present in the fixed area.

Next, using the flowchart of FIG.
The initial setting operation of the CD camera 5 (initial setting operation for eye-gaze matching) will be described. FIG. 15 is a diagram for conceptually explaining this setting operation.

First, the interlocutor 3 causes the CCD camera 5 to photograph itself (step ST1), and displays the self image 9 photographed by the CCD camera 5 on the monitor screen 8 of the display unit 21 (step ST2). At this time, if the installation position of the CCD camera 5 matches the line of sight of the interlocutor 3 (line connecting the eyes of the interlocutor and the center point A of the image 9) (the parallax angle is 0).
゜), the presence of the CCD camera 5 causes a dialogue obstacle.

Then, the CCD camera 5 is moved upward by Δh.
(Step ST3), and then re-install the CCD camera 5 in a position that does not cause a dialogue obstruction and within a line-of-sight matching allowable region (for example, the region shown in FIG. 12). That is, the CCD camera 5 is installed so as to overlap the hair portion 9 b of the image 9 as viewed from the interlocutor 3.

As described above, when the installation position of the CCD camera 5 is moved by Δh, the imaging direction of the CCD camera 5 cannot be directed to the actual talker 3. Therefore, the installation angle of the CCD camera 5 is corrected downward by Δθ in accordance with the movement of the CCD camera 5 (step ST4). As described above, by adjusting the installation position and the set angle of the CCD camera 5 in advance so that the parallax angle determined from the relationship between the distance L between the interlocutor 3 and the monitor screen 8 and the movement amount Δh is within 3 °. , The eye-gaze matching state can be maintained.

Specifically, when the interlocutor 3 is a short child, the sitting height of the child is adjusted by the sitting height adjusting mechanism 40 of the chair 4 and the angle of the CCD camera 5 is corrected downward. It is preferable to move the image 9 itself displayed on the monitor screen 8.

On the other hand, the talker 3 displayed on the monitor screen 8
The operation of moving the image 9 of the interlocutor 3A or 3B at the point A or the point B will be described with reference to the flowchart of FIG. FIG. 16 is a diagram for conceptually explaining the image moving process. This image movement processing is performed by the imaging data processing unit 60 of the imaging unit 6 or the processing unit 26 (display control unit 285).

First, a reference point is specified from the received image 9 displayed on the monitor screen 8 of the other party or the user (step ST5). This reference point is a point 10 cm above the center (point A) of the eye position (indicated by a dotted line in the figure) of the image 9 or the contour of the image 9 easily obtained by binarization processing. 9 (CC)
(The position where the D camera 5 should originally exist).

Subsequently, a position 81 corresponding to the actual installation position of the CCD camera 5 is obtained from the absolute coordinates 80 on the monitor screen 8, and the deviation amount between the already obtained reference point position 82 and the actual position 81 is obtained. Is within the allowable range (step ST6). If the amount of shading exceeds the allowable range at this stage, it is determined that the display position of the image 9 is deviated, and the coordinates 82 of the reference point are set as a starting point, and the actual position 81 is determined.
Is calculated as an end point (step ST
7). Then, the vector 8 obtained as shown in FIG.
Based on 3, the displaced image 9 is translated (step ST <b> 8).

For the purpose of the present invention, any of the above-described first to third means may be used as the means for matching the line of sight between the interlocutors via the transmission means, and these means may be combined. May be used. In the two-way conversation, the distance L of each chair 4 from both monitors 4 to the monitor TV1 is fixed within a certain range, and the height of each chair 4 is adjusted vertically by the seat height adjustment mechanism 40 by the above-described second means. Then, the position of the face of the image 9 of the interlocutor 3 displayed on the monitor screen 8 is present in a substantially constant area. In addition, as a result of an experiment, the inventor has confirmed that the range in which fine adjustment of the face size, eye position, and the like for each interlocutor is within several cm to several tens cm. Therefore, for this fine adjustment, it is effective to move the display position of the image 9 of the interlocutor 3 displayed on the monitor screen 8 by the above-described second means. This display movement causes the seat height adjustment mechanism 40 of the chair 4 to move.
It is possible to easily achieve mutual gaze matching between the interlocutors simply by adjusting. Further, it is not necessary to provide a complicated moving mechanism and an optical processing device on the side of the CCD camera 5. In the above-described embodiment, the installation position of the CCD camera 5 is set at 10
(Cm), the distance h can be easily changed according to the image display movement by the third means.

The photographing device 600 of this embodiment can be retrofitted to a TV device that is usually sold on the market.
Therefore, there is no need to develop a special-purpose device as in the above-described second related art using a display device that switches between reflection and transmission, and the above-described imaging device can be realized at low cost. Further, there is no need for special optical components such as a half mirror and a reflection mirror as in the first and third prior arts described above. Therefore, the apparatus configuration is simplified, and the image of the talker is not darkened by the mirror.
Also, as a result of enlarging the mirror to obtain a wide angle of view,
The problem that the screen display is disturbed does not occur in the imaging device according to the above-described embodiment. Therefore, according to the two-way interactive system according to the present invention, two-way interactive using the image display device having a large screen can be easily realized.

FIGS. 17 and 18 are diagrams showing the configuration of an imaging device 600 that can be retrofitted to a commercially available TV device. As shown in FIGS. 1 and 2 and the like, the imaging apparatus 600 includes a CCD camera 5 for imaging an interlocutor, supports the CCD camera 5, and sets the CCD camera 5 at a desired position. Support mechanism 10 (robot arm) for
Processing unit 26 including imaging data processing unit 60 described above
As a circuit board.

In the imaging device 600, the CCD camera 5
From the terminal 51 via the camera cable 50
Is sent to the processing unit 26 which is electrically connected. The voice of the interlocutor 3 is captured by the microphone 24,
The signal is sent as an electric signal to the processing unit 26 electrically connected to the terminal 241 via the cable 240. on the other hand,
The image data and audio data of the interlocutor sent from the transmission means 27 are taken into the processing unit 26 which is electrically connected to the terminal 611 via the cable 610. Then, the image data and the audio data of the other interlocutor (for example, the interlocutor 3B at the point B) once taken into the processing unit 26 are sent to a commercially available TV device via the cable 650. The terminals 601a (audio terminal) and 601b (image terminal) of the cable 650 are electrically connected to the processing unit 26, while the terminal 602a (audio terminal),
The terminal 602b (image terminal) is connected to a predetermined connector of the TV device.

FIG. 18 is a diagram showing a specific configuration of a connector provided on the back surface of the imaging device 600. As can be seen from this figure, the terminal 51 of the CCD camera 5 is inserted and fixed to the image input connector 52, the terminal 241 of the speaker 24 is inserted and fixed to the audio input connector 242, and sends image and audio data to the TV device. The terminals 601a and 601b of the cable 650 are inserted and fixed in the audio output connector 603a and the image output connector 603b, respectively, and the terminal 611 of the cable 610 for transmitting and receiving data to and from the transmission means 27 is connected to the terminal 611. The connector 612 is inserted and fixed. Note that the image input connector 620 and the audio input connector 630 are spare connectors for transmitting and receiving data to and from the transmission unit 27.

Next, a logical configuration diagram of the terminal devices 20A and 20B in the interactive system according to the present invention will be described with reference to FIG. In addition, the system is aimed at a system that realizes realistic counseling between interlocutors in a remote place. In addition to a mechanism that matches the gaze between interlocutors, the system is designed to realize an ideal counseling environment. It has various functions. The configuration described below is a configuration common to both the terminal devices at the point A and the point B.

First, this terminal device is connected to the processing unit 26.
Is an integrated device including an imaging data processing unit 60. Here, the imaging data processing unit 60 takes in the imaging data from the CCD camera 5 and controls a data processing unit 600a for performing predetermined processing and a driving mechanism 600c for adjusting the set position and the set angle of the CCD camera 5. And a drive control unit 600b.

The main control section 273 of the processing unit 26 is a card-shaped recording medium 289 provided with at least a person authentication section 274 (IC (integrated circuit)) for managing the security of the system.
The information read from the card or the like by the reading device 271 is also used.)
An interactive processing unit 275 for recording, a backup system 276 for performing crisis management of the entire system
(For example, an abnormality of the transmission / reception system is detected, and the main transmission / reception system 288 such as an optical fiber is switched to the wireless system 287). External memory management unit 277 for outputting to the recording medium 289, reception, keyword information, consultation item information guidance,
An integrated information processing system 278 for comprehensively managing counselor information, fee information, and the like with integrated information D / B (data base) 267 is provided. In addition, the main control unit 27
In step 3, the position of the human image (including the monitor images of the first and second interlocutors) displayed on the monitor is also recognized.

The recording medium 289 includes, in addition to the information necessary for the person authentication, reservation information, information for performing the person authentication of the user of the terminal device of the conversation partner, and the reserved conversation start time. It is more preferable that conversation conditions such as reserved conversation time and conversation contents are recorded. This means that information shared only between such interlocutors can be stored on the recording medium 28.
This is because the confidentiality of the dialogue is ensured by each user (interactor) possessing it in the form of No. 9. Further, it is preferable that predetermined items including the reserved dialog start time and the dialog reservation data relating to the conversation partner are recorded in advance in such a recording medium 289, and the transmission unit 27 is stored based on the dialog reservation data. A configuration may be adopted in which, after being connected to the partner terminal device searched via the line, two-way communication is started with the partner terminal device. In general, on the phone etc., the conversation between the two parties will not be established unless the other party is located or a specific phone is specified, but with the above configuration, without specifying the location of the interlocutor and the terminal used, This is because a two-way dialog can be realized based on information shared only between the parties.

The processing unit 26 further includes a voice / sound data processing unit 279 (for example, spectrum analysis of voice information) for processing voice information taken in from the microphone 24, and adjusts the installation position of the microphone 24. And a sound / sound output unit 282 (the sound / sound output unit 282) for causing the speaker 23 to output predetermined sound or sound information.
Utilizes dedicated voice / sound information D / B 265 to realize a volume control function 283 and a BGM output function 284 for providing an ideal counseling environment. ), A display control unit 285 (provided with processing functions such as an image enlargement / reduction function) for displaying image data, character data and the like on the monitor 1, and a character / graphic input device 272 as a man-machine interface. And a character / figure input control unit 286 for performing the operation. It should be noted that a plurality of monitors 1 may be provided, and the first and second monitors 1a, 1a,
The display control unit 285 may perform control to display different information for each of b (for example, display image information on the first monitor 1a and display character / graphic information on the second monitor 1b).

The processing units 26 are individually provided with dedicated D / Bs (for example, image information D / B 261, audio / sound information D / B 262, character information D / B 263, backup D / B 264, BGM information D / B 265). , Integrated information D
/ B267, display information D / B266, etc.). The character / graphic input device 272 includes, for example,
There are keyboards, pointing devices, touch panels and special pens.

Hereinafter, the operation of the interactive interactive system as a counseling system will be described in detail with reference to the configuration of FIG. FIG. 20 shows the audio output unit 23
FIG. 3 is a diagram showing the configuration of FIG. By using the speaker arranged on the side of the interlocutor 3 and in the vicinity of the ear, an environment in which the user whispers at the ear is realized, and the intimacy is further improved. The headphone is representative of the audio output unit 23, but the ear independent speaker 12 fixed to the chair 4 by the flexible joint 13 shown in FIG. Note that the speaker may be attached to a surrounding wall (for example, the sound absorbing members 22A and 22B shown in FIG. 1).

The audio output unit 23 has a function 284 for outputting background music (BGM) and a sound / sound output control function 283 for automatically detecting the conversation volume and adjusting the output volume of the background music. The output is controlled by the unit 282. When detecting that the conversation volume is substantially zero for a predetermined time, the voice / sound output control unit 282 starts outputting background music, increases the volume of background music, or increases background music based on the detection result. Adjust volume reduction.
In addition, the voice / sound output control unit 282 outputs the background music that calms the mind of the interlocutor 3 on the voice of the interlocutor 3 without hindering the hearing of the interlocutor 3's voice. be able to. In particular, when the silence continues, the background music is started or the volume is changed.
Is released, and the interlocutor 3 can be calmed down.

The acoustic information such as the background music and voice can be recorded on a recording medium 269 such as a CD-ROM or an optical card. Then, this recording medium 26
A part of the acoustic information recorded in 9 is output at least at any stage before the start of the dialogue, during the dialogue, and after the end of the dialogue, so that the psychological relaxation effect can be more effectively obtained. In addition, the acoustic information is used as control information to be output to the partner terminal device via the transmission unit 27 at least at any stage before the start of the dialog, during the dialog, and after the end of the dialog. It may be transmitted to the device.

The voice input unit 24 is preferably a telephoto microphone. Generally, a microphone with a table or attached to the chest can be used, but a microphone with a table may impair consciousness during counseling. Also, if the microphone on the chest is too far away, the code may be broken. Therefore, a telescopic microphone that does not need to be conscious of its existence is placed on the upper part of the display unit 21 or
It is preferable to incorporate them into Since the line of sight of the interlocutor 3 has already been recognized, the directivity can be improved by directing the sound collection direction to the mouth.

The voice input unit 24 has a voice input amplification adjustment function for controlling the volume of the collected sound, automatically converts the output volume of the collected voice to a predetermined level, and also controls the volume of the collected voice at a predetermined time. The average level is detected, and is controlled by a voice / sound data processing unit 279 having a volume adjustment function for automatically converting the volume when the difference between the collected sound volume and the average level is equal to or more than a predetermined value. The display unit 21 displays the volume to be transmitted from the voice / sound data processing unit 279 to the other interlocutor via the transmission unit 27 under the control of the main control unit 273 of the processing unit 26. . The act of speaking to a person (an image of an interlocutor) on the monitor screen 8 in a state where the microphone 24 is not directly visible makes it difficult to judge how much the user's own volume should be if not used. Therefore, by adjusting the volume of speaking, smooth dialogue is facilitated. In addition, by displaying the output volume (the volume of the audio information sent to the other party) on the screen, it is possible to feed back to the interlocutor whether the output volume is appropriate for transmission.

On the other hand, the character or graphic information input from the character / graphic input device 272 is
6, the character information or graphic information corresponding to the character or graphic input from the character / graphic input device 272 is transmitted and output to the terminal device 20B at the point B. Therefore, the transmission information transmitted on the transmission path further includes character information or graphic information (text data) in addition to human image information (image data) and audio information (audio data). The display unit 21 displays a character or a graphic together with the image 9 of the other party 3 based on the transmitted character information or graphic information.
In this way, by displaying characters or figures together with a human image, counseling is facilitated.

Furthermore, such characters, figures, etc.
The information may be recorded on a recording medium 269 such as a ROM or an optical card. A part of the information recorded on the recording medium 269 may be read before, during, and after the conversation with the other party. At least one of the stages may be configured to be displayed on the display unit 21. It should be noted that such information such as characters, figures, images, and the like generally includes information such as a corporate advertisement in addition to information that can provide a relaxation effect. In other words, the same information is provided every time, so that the interlocutor can prepare for the introduction of the dialogue before the start of the dialogue, even if his location and psychology are different from those of the previous dialogue. This is because the same mental state can be led every time, and after the dialogue, psychological preparation for returning to the normal activity state can be made, and a psychological relaxation effect can be obtained. Then, the above-mentioned visual information and acoustic information are used as control information to be displayed and output on the partner terminal device at least at any stage before the start of the dialogue, during the dialogue, and after the end of the dialogue. May be transmitted to the terminal device of the other party via the.

Here, the characters or figures displayed on the monitor TV1 of the display unit 21 may be displayed so as to be superimposed on the image of the interlocutor 3, but as shown in FIGS. Preferably, the display area and the display area of the image 9 of the interlocutor are separated. In particular, when a human image is displayed on a monitor while changing the display size, the human image is moved on the monitor by the algorithm shown in FIG. This can be realized by using any of the monitor images of the interlocutors), and the gazes are matched between the first interlocutor and the second interlocutor. When the display area 8 (monitor screen) of the monitor TV1 of the display unit 21 is vertically divided, the image 9 of the interlocutor is preferably displayed on the upper side (see FIG. 21). Alternatively, the monitor TV of the display unit 21
It is also possible to display the image of the interlocutor on one side or the center of the first display area 8 in the left-right direction (see FIGS. 22 and 23). In particular, when the screen is divided into right and left, it is preferable that a character or a graphic on the right side is displayed toward a person who looks at the monitor screen, and an image of the other person is displayed on the left side. In general, humans perform artistic and subjective thinking in the right brain, and perform computational and logical thinking in the left brain. However, visually, the movements of the right and left brains are reversed. Information from the right eye is mainly processed by the left brain, and information from the left eye is mainly processed by the right brain. Therefore, when displaying two types of characters / graphics (logical) and the image of the interlocutor (subjective) simultaneously, it is preferable to display the characters / graphics on the right side and the human figure on the left side.

Further, as shown in FIG.
An image of the interlocutor may be displayed in the upper central portion of the first display area 8. In this case, the display control unit 285 performs image processing so as to reduce the interlocutor image 9.

The display control unit 285 is capable of enlarging or reducing the interlocutor image 9 (which may be either the first interlocutor or the second interlocutor) displayed on the monitor TV1. At this time, a part of the image corresponding to the position where the camera 5 is actually installed is detected near the top of the dialogue image 9 and the dialogue is performed in a state where the detected image part is fixed. Enlarge or reduce the whole person image. At this time, a predetermined area on the monitor TV 1
And at least one of a character and a graphic is displayed in the area on the monitor excluding the area where is displayed.

According to the above-mentioned separation method, the eye-gaze coincidence does not deteriorate. A display device for displaying an image of the interlocutor;
It is of course possible to separate the display device for displaying characters or graphics (for example, as shown in FIG. 19, the display control unit 285 has the first monitor TV1a and the second monitor TV
1b can be independently displayed and controlled). Further, the character / graphic input device 272 may be arranged at one of the terminal devices 20A and 20B at the point A or the point B.

[0110] Smooth dialogue is possible by performing counseling with characters and figures.
It is most preferable that the voice of the interlocutor can be input, recognized, converted into character information and displayed (this function is controlled by the main control unit 273 under the control of the voice / sound output). Unit 282).

FIG. 1 shows a character / graphic input device 272.
As shown in FIG. 9, a keyboard device, a pointing device,
A device, and further, a pen input device (touch panel and pen) arranged near the interlocutor can be adopted. Further, by providing both the keyboard device and the pen input device as appropriate, the counseling can proceed more smoothly.

The system aims to enable ideal counseling between interlocutors at remote locations. Therefore, in order to preserve such a special environment,
The system includes, as a security function, a person authentication unit (a person authentication unit 274 in the processing unit 26) for confirming the person of the interlocutor, and the counseling operation is started after the person is confirmed.

In this case, the personal authentication section 274 may read the authentication data from the storage medium 289 in which the authentication data has been written, and may perform a person confirmation based on the read result. The storage medium 289 is a recording medium provided with an IC or the like, and for example, an optical card, an IC card, or a magnetic card can be used. 271
Prepare).

The person authentication section 274 may perform the person confirmation based on the image analysis result of the face of the interlocutor, or may perform the person confirmation based on the voice analysis result of the voice of the interlocutor. It is preferable that the confirmation of the person is performed based on the character analysis result of the sign character of the interlocutor. Further, the person confirmation may be performed based on the result of the text analysis of the message text of the interlocutor. Such person authentication makes it possible to specify the interlocutor. It is preferable to use the authentication method properly according to the confidential content.
The above-mentioned message sentence analysis refers to, for example, comparing a sentence "X is Y." with the characters put in X and Y against a database of the person or a cipher database of the designated person. Refers to analysis.

Next, the table 2 is set at a height where the height above the chest is higher than the table when the interlocutor 3 is sitting on the chair 4. However, the height of the table 2 may be set so that the height above the chest is higher than the table when the interlocutor 3 is standing. This is because by performing the conversation while standing, it is possible to expect the effect of the active conversation and the shortening of the conversation. Further, since a sense of continuity is generated between the interlocutor 3 and the monitor screen 8 by using the table 2, a feeling of anxiety caused by losing the relative distance (psychological distance) from the other party is reduced. In this case, the display unit 2
It is desirable that the table 1 and the table 2 are in contact with each other (the configuration in FIG. 1).
Below).

In the case of counseling or the like, it is preferable that the processing unit 26 conducts dialogue state analysis such as dialogue time analysis, voice spectrum analysis, predetermined term analysis, or body movement analysis. As a result, it becomes possible to quantitatively analyze a psychological change or the like of the interlocutor who is receiving counseling (it also becomes a counselor's data).

[0117] At least a part of the surroundings of the interlocutor is provided with a sound absorbing member 22 having a property of blocking external sounds (see FIG. 1). As a result, the psychological obstacle to the leakage of the conversation contents (counseling contents) to the outside is reduced, and the sound quality is improved.

In FIG. 1, two independent transmission paths are prepared as the transmission means 27, but three or more transmission paths may be provided. In addition, a coaxial cable transmission line or another wireless transmission line may be used as the transmission line in addition to the optical fiber transmission line 31 and the satellite communication transmission line 32.

Further, the two-way interactive system according to the present invention is provided with a backup system for detecting a failure of a transmission line during interactive use and automatically switching the transmission line to be used. By connecting via a plurality of transmission paths, if a transmission path failure occurs during a dialogue (or during counseling), the dialogue can be continued using another transmission path. In such a case, it is preferable to secure at least the transmission of voice in another transmission path. Generally, line abnormalities include coaxial cables, optical fiber cables,
Alternatively, it is considered that the cause is an abnormality of the transmission device, and therefore, it is preferable that the other transmission paths be wireless.

Next, a description will be given of a result of an experiment of a two-way image communication dialogue performed using the two-way dialogue type system having the eye-gaze matching mechanism according to the above-described embodiment (a detailed system configuration is shown in FIG. 1). reference).

The monitor TV1 for the experiment has a screen diagonal line 10
A 3.4 cm 40 inch large projection monitor was used. In addition, a micro camera having a camera head having a diameter φ of 12 mm was used as the experimental CCD camera 5.
Each monitor TV1 was installed in each adjacent room, and the distance between the monitors was about 20 m. The distance L between the monitor TV1 and the interlocutor 3 was about 2 m, the upper body of the interlocutor 3 was photographed within a range of 88 cm in width and 66 cm in height, and the image was transmitted to the monitor TV1 of the other party. . Also, CCD
The camera 5 was fixed at a position h = 10 cm above the line of sight of the interlocutor 3, and the height of the chair 4 was adjusted according to the sitting height of each interlocutor 3.

In the two-way conversation performed in such a system configuration, the parallax angle is in the range of about 2.86 ° in the basic posture in which the interlocutor 3 sits on the chair 4 in a normal posture. However, in practice, the head and upper body of the interlocutor 3 move when speaking. When this movement was actually measured for six interlocutors, the maximum was about ± 15 cm in the horizontal direction and about ± 10 cm in the vertical direction. Therefore,
When the parallax angle including the movement of the interlocutor 3 is calculated in the above system configuration, a maximum of 4.3 is obtained in the horizontal direction of the screen of the monitor TV1.
°, up to 5.7 ° vertically. For this reason, according to the present embodiment, the parallax angle is within a range of 4.5 ° or less in the left-right direction, 12 ° directly above, and 8 ° or less directly below, which are regarded as the allowable range of the eye-gaze coincidence by the Joint Conference of the Institute of Electrical Engineers of Japan. Fits. In addition,
The average value of the movement of the interlocutor 3 is about ± 7 cm in the horizontal direction and about ± 4 cm in the vertical direction, each of which is less than half of the maximum value. Therefore, the parallax angle obtained by the present system is considered to have no problem in actual dialogue. Also, in the present experimental results, the interlocutor did not feel any special discomfort regarding the eye-gaze matching, and the two-way conversation was smoothly performed.

According to this experiment, by matching the eyes of the interlocutors via the transmission means 27, it is possible to realize emotional exchange between the interlocutors even on the theme relating to the trouble, and more than 80% of the subjects It was confirmed that counseling was also possible with the system. In addition, by using characters or figures together, it is possible to conduct a "character and figure recognition dialogue" in which the dialogue is advanced while staring at the letters or figures.
It was confirmed that the conversation could be organized.

In order to further improve the system, the distance L between the monitor TV 1 and the interlocutor 3 is increased to about 2.5 m, and the display of the interlocutor image 9 displayed on the monitor screen by the display control unit 285 is performed. It is preferable that the position is finely adjusted and controlled so that the line of sight of the interlocutor 3 always falls within the parallax angle of 3 ° with respect to the CCD camera 5. According to such a configuration, even when the interlocutor 3 moves slightly, proper gaze matching can be maintained, and it becomes possible to achieve more realistic gaze matching between interlocutors.

The above experiment was conducted by using the large monitor TV1 and setting a certain distance between the photographer 3 and the monitor TV1. When a small monitor, for example, a 10-inch monitor TV is used as the monitor TV1, the image of the interlocutor displayed on the monitor screen 8 naturally becomes smaller.
Therefore, the parallax angle can also be reduced by combining the small monitor TV with an ultra-small CCD image pickup device and setting the size of the image pickup device on the small monitor screen to a certain ratio or less. In this case, the distance L between the interlocutor 3 and the monitor TV1 can be easily reduced to 1 m or less, which is advantageous in saving space. The inventor uses a 14-inch monitor TV to set h =
A counseling experiment was performed under the conditions of 5 (cm) and L = 1.2 (m). As a result, 83% of 18 clients positively evaluated counseling in this manner.

According to the above-described embodiment, it can be retrofitted to the monitor TV1 which is normally shipped, does not require any special optical components such as a half mirror and a reflection mirror, and can be handled only by the CCD camera 5. In addition, if an image processing function is added, an imaging apparatus having a line-of-sight matching mechanism having a line-of-sight matching function can be realized (FIGS. 17 and 18).
reference). In the above-described embodiment, the configuration in which the imaging device is retrofitted to the existing TV device has been described. That is, a device configuration in which an image pickup unit such as a CCD camera is embedded in a part of a display device such as a liquid crystal television or a plasma display, or a device in which the image pickup unit is previously formed as a part of a device in a partial area of the display device. It may be configured (see FIGS. 7 to 10). In this case, it is necessary to develop a special-purpose machine, but the device configuration is extremely effective for reducing the thickness of the monitor in the future.

The present invention is not limited to the above embodiment, but can be modified. For example, an image printing device (printer 270) that prints a display screen, or a storage unit that stores display screen information (for example, display information D / B
266, a backup D / B 264, an IC card 269). As the storage means, an optical card, an IC card, an optical disk, or a magneto-optical disk can be used. In this case, it is possible to save information during the dialogue, particularly information on characters and figures used during the dialogue.

[0128]

As described above, in the system according to the present invention, when the imaging unit (camera) is installed in a fixed area in front of the image of the interlocutor on the display unit (the interlocutor is superimposed on the image of the other party). However, if the size is kept below a certain level, the presence of the photographer and the imaging unit placed between the display units have little effect on the two-way conversation. This was realized based on the experimental result of not giving.

For this reason, by adding components to an existing display device, a photographing device and a terminal device having a mechanism for easily realizing eye-gaze matching between interlocutors, and a system constituted by these devices are realized. can do. Therefore, there is no need to develop a dedicated machine, and the apparatus can be realized at low cost. In addition, since the device can be configured only by fixing the imaging unit at a fixed position on the screen, there is no need to provide a half mirror of the same size as the screen as in the related art. The size of the display screen can be easily increased. Further, since the photographing of the interlocutor is performed directly without passing through the mirror, the obtained image does not become dark.

Further, by moving the imaging unit by the driving mechanism, by adjusting the line of sight of the interlocutor by adjusting the position (adjustment of sitting height) by the interlocutor, and by moving the image by the display control unit. By moving the display position of the image of the interlocutor displayed on the screen of the display unit, the imaging unit (camera) is located in a certain area of the image of the other interlocutor displayed on the screen as viewed from the interlocutor. Will be installed. Also, by combining these means, it is easy to achieve a parallax angle of 3 ° or less required for ideal visual line matching. In particular, image processing is performed by the display processing unit, and the image of the other party's interlocutor displayed on the screen is moved to a fixed position with respect to the imaging unit, so that more accurate gaze matching and gaze recognition can be performed mechanically. This can be realized without using a simple drive system. Also, the degree of eye-gaze coincidence can be changed and set in accordance with the individual level.

Further, the driving mechanism of the image pickup unit adjusts the angle of the image pickup unit. By adjusting the angle of the image pickup direction of the image pickup unit, the image pickup direction is, for example, the dialogue recognized by the image data processing unit. Automatically pointed at the person's gaze. For this reason, even if the interlocutor moves during the two-way conversation, it is possible to maintain an optimal gaze matching state between the interlocutors.

Also, by using characters or figures together, it is possible to perform a "character / graphics recognition dialogue (visual communication)" in which the dialogue proceeds while staring at the characters or figures. We can plan unity of talk. As described above, by using this system, a smooth dialogue with the emotional exchange via the monitor TV becomes possible. In addition, the system can be widely applied to multimedia communication, multimedia counseling, multimedia helping, audio-visual counseling, etc.

[Brief description of the drawings]

FIG. 1 is a logical configuration diagram showing an entire interactive interactive system according to the present invention.

FIG. 2 is a view showing the image pickup apparatus of FIG. 1 and is a view for explaining a method of installing a CCD camera (image pickup unit) at a predetermined position.

FIG. 3 is a view showing a first embodiment of a support mechanism in the imaging device of FIG. 1, and is a view showing a structure for installing a CCD camera at a predetermined position.

FIG. 4 is a view showing a second embodiment of the support mechanism in the imaging device of FIG. 1, and is a view showing a structure for installing a CCD camera at a predetermined position.

FIG. 5 is a diagram showing a structure of a camera moving mechanism of the support mechanism (second embodiment) shown in FIG. In particular, this mechanism moves the CCD camera horizontally and vertically.

FIG. 6 is a view showing a structure of an angle adjusting mechanism of the support mechanism (second embodiment) shown in FIG. In particular, this mechanism is C
Change the imaging direction of the CD camera.

FIG. 7 is a diagram showing a second embodiment of the terminal device in the interactive interactive system, in which the embodiment is a CC;
As a support mechanism (third embodiment) for setting the D camera at a predetermined position, the CCD lamella has a structure attached to a display unit.

FIG. 8 shows the support mechanism (third embodiment) shown in FIG.
FIG. 8 is a diagram illustrating a mounting structure of the CCD camera, and is a cross-sectional view taken along line XX of FIG. 7.

FIG. 9 shows the support mechanism (third embodiment) shown in FIG.
It is a perspective view which shows an assembly process.

FIG. 10 is a diagram showing a structure for adjusting a position and an imaging direction of a CCD camera as an application example of the support mechanism (third embodiment) shown in FIG.

FIG. 11 is a diagram for describing a parallax angle.

FIG. 12 is a diagram showing an area where a CCD camera is to be installed.

FIG. 13 is a flowchart illustrating an initial setting operation of the CCD camera.

FIG. 14 is a flowchart for explaining an operation for moving an image displayed on a monitor so that the lines of sight coincide between the interlocutors.

FIG. 15 is a flowchart of the CC described in the flowchart of FIG.
FIG. 5 is a diagram for conceptually explaining the initial installation operation of the D camera.

FIG. 16 is a diagram conceptually illustrating an operation of moving an image displayed on a monitor, which is described in the flowchart of FIG. 13;

17 is a diagram illustrating an application example (imaging device) of the imaging unit alone in FIG. 1;

18 is a diagram illustrating a structure of a back surface of the imaging unit illustrated in FIG.

FIG. 19 is a logical block diagram showing the overall configuration of the terminal device shown in FIG.

FIG. 20 is a perspective view showing a configuration of an audio output unit in FIG. 1;

21 is a diagram illustrating an example of a screen configuration of a display screen (particularly, in the case of screen division display) displayed by the monitor of FIG. 1 (part 1).

FIG. 22 is a diagram illustrating an example of a screen configuration of a display screen (particularly, in the case of screen division display) displayed by the monitor in FIG. 1 (part 2).

FIG. 23 is a diagram showing an example of a screen configuration of a display screen (particularly, in the case of split screen display) displayed by the monitor of FIG. 1 (part 3).

FIG. 24 is a diagram illustrating an example of a screen configuration of a display screen (particularly, in the case of screen division display) displayed by the monitor in FIG. 1 (part 4).

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Monitor TV, 2 ... Table, 3 ... Interactor, 4 ... Chair, 5, 200 ... Imaging device (CCD camera), 6 ... Imaging unit, 9 ... Interviewer image, 10, 110 ... Support mechanism, 2
0: terminal device, 21: display unit, 23: speaker, 24 ...
Microphone, 25, 272: character / graphic input device, 26: processing unit, 27: transmission means, 60: imaging data processing unit,
271, a reading device, 274, a person authentication unit, 275 dialogue processing unit, 281, 600c, a driving mechanism, 269, 289,
Recording medium, 279 ... voice / sound data processing unit, 282 ...
Voice / sound output control unit, 285 ... display control unit, 600,
601: imaging device.

Claims (5)

[Claims] A first terminal installed at a first point where a first interlocutor is located, and a first terminal installed at a second point where a second interlocutor is located a considerable distance from the first point. A second terminal device, and transmission means for enabling at least two-way communication of image data of the first and second interlocutors between the first and second terminal devices. In the interactive interactive system, at least the first terminal device may include an external storage medium in which predetermined items including information including at least one of background music data, voice data, character data, graphic data, and image data are recorded in advance. And a reading device for reading the predetermined item, and at least a part of the information recorded on the external recording medium, before the start of the dialogue with the second interlocutor, during the dialogue, and after the end of the dialogue , Without even at any stage, two-way interactive system and outputs to the first terminal device. 2. The method according to claim 1, wherein the first terminal device transmits at least one of the information recorded on the external recording medium before, during, and after the conversation with the second interlocutor. 2. A transmission to the second terminal device via the transmission unit at any stage.
A two-way interactive system as described. 3. The external recording medium includes a card-shaped external storage medium provided with at least an IC, in which predetermined items including information necessary for personal authentication of the terminal user are recorded in advance before a dialogue is started. 2. The interactive interactive system according to claim 1, wherein: 4. The external storage medium stores, as the predetermined items, at least one of reservation information and information for performing personal authentication of a user of the second terminal device as a conversation partner. The interactive system according to claim 3, wherein 5. The external recording medium according to claim 1, wherein said predetermined items include at least a reserved conversation start time, a reserved conversation time, and a conversation condition including a conversation content. 3. The interactive interactive system according to 3.