JP3074678B2 - Terminal device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a terminal device suitable for a two-way interactive system provided with a mechanism for matching the line of sight between interlocutors via transmission means. Note that a two-way interactive system is a counseling system that realizes two-way dialogue via transmission means, such as the relationship between a doctor and a patient at a considerable distance, the relationship between a counselor and a client, etc. This system is designed for one-to-one interactive conversation.

[0002]

2. Description of the Related Art Conventionally, counseling performed in the fields of education, medical care, psychology, and the like is generally performed in a situation where interlocutors face each other directly while building a close mutual trust between them. However, those who have counseling skills with those who need counseling due to natural disasters, remote places, etc. (Counselors)
There are situations where it is difficult to share the same place and time with the same. Therefore, with the development of communication technology, the need for a system for realizing counseling between persons located in remote places has increased.

In order to construct such a counseling system using communication technology, in order to build a close relationship and an ideal counseling environment between interlocutors as in the conventional face-to-face method, First, it is important to match the eyes of the interlocutors.

In view of the above, a conventional image pickup apparatus that matches the line of sight between interlocutors via a transmission line is disclosed in
-269128 (the first prior art). In the first prior art, a half mirror provided between the interlocutor and the image capturing apparatus is provided, so that light emitted from the interlocutor serving as a subject is transmitted to the image capturing apparatus and the image display apparatus. Are separated. Therefore, according to the device configuration of the first prior art, the imaging device is arranged at a position avoiding the display screen of the image display device viewed by the interlocutor, and the input optical axis of the imaging device and the line of sight of the interlocutor. And match.

Secondly, Japanese Patent Application Laid-Open No. Hei 4-213287 (second prior art) uses a special display device capable of switching between reflection and transmission of light, and employs a special display device located behind the display device (this display device is referred to as a display device). An imaging device having a configuration in which the imaging device is arranged on the side opposite to the interlocutor via the intermediary) is disclosed. According to this device configuration, the line of sight of the interlocutor is behind the display device,
In addition, since the user is directed to the imaging device arranged at a position avoiding the display screen of the display device viewed by the interlocutor, the eyes of the interlocutors can be matched.

Third, in Japanese Patent Application Laid-Open No. 4-977663 (third prior art), a small mirror or a small half mirror is mounted on the upper part of a display device, and a small camera whose optical axis is directed to the mirror is used for a user. There is disclosed an imaging apparatus configured to capture images. In this device configuration, the interlocutor image reflected by the mirror is captured by a small camera arranged at a position avoiding the display screen of the display device viewed by the interlocutor. Therefore, the elevation angle of the line of sight of the talker is suppressed within 10 ° which is the allowable range of the line of sight coincidence in the vertical direction, and the line of sight of the talker is achieved.

[0007]

In the first prior art in which light is separated from a subject by a half mirror, a half mirror having a size approximately equal to the screen size of a display device is required. Inevitably, a large space for accommodating such a large half mirror is required. In particular, when the screen size exceeds 40 inches, it cannot be easily manufactured. Also, since we use a half mirror,
An image picked up by an image pickup device installed at a position avoiding the screen on which the interlocutor image is displayed becomes dark, and an ideal counseling environment cannot be realized.

Next, the second prior art using a special display device for switching between reflection and transmission of light cannot be incorporated into a commercially available TV device which has already been mass-produced.
Therefore, if an attempt is made to realize a system using this principle, the development of a dedicated machine is required, and the price of the apparatus becomes high. Further, an imaging device for imaging the interlocutor is arranged to image the interlocutor via the special display device. Therefore, the captured interlocutor image becomes dark, and an ideal counseling environment cannot be realized.

Further, in the third prior art in which a reflecting mirror and a small camera are combined, the camera is arranged at a position avoiding a display screen seen by the interlocutor, and the interlocutor is imaged through the small reflecting mirror. Therefore, the person image to be photographed becomes dark. Further, in order to obtain a wide angle of view, the size of the reflection mirror must be equal to or larger than the size of the small camera, and the display of a human image on the display device is hindered by the large reflection mirror. Also, when the position of the person to be photographed changes, the imaging direction of the small camera is constant,
You cannot follow this. Therefore, it is not always possible to ensure that the eyes of the interlocutors match.

A structural feature common to the above-described prior art is that a display screen (a screen on which an image of the other party is displayed) viewed by an interlocutor and the interlocutor are viewed by the interlocutor. This is a point that is arranged at a position that avoids the space of. In other words, it is necessary to install a reflection mirror, a half mirror, and a special display device between the interlocutor and the imaging device in order to match the line of sight between the interlocutors. Therefore, the image of the other party to be transmitted becomes dark, and an ideal counseling environment cannot be realized.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and as a counseling system, a transmission means is provided through a transmission means in order to reduce a sense of discomfort during a conversation between distant interlocutors. It is an object of the present invention to provide a terminal device suitable for a two-way interactive system provided with a mechanism for matching eyes of interlocutors. In particular, this two-way interactive system uses sound (background music (B
GM)), character and graphic information, etc.
Create an organic relationship (realize close communication) between interlocutors and provide an ideal counseling environment.

[0012]

According to the present invention, there is provided a first terminal installed at a first point where a first interlocutor is located, and a first terminal which is located at a considerable distance from the first interlocutor. A second terminal device installed at a second location, and at least two-way communication of image / audio data of the first and second interlocutors between the first and second terminal devices. The present invention relates to a terminal device suitable for a two-way interactive system having a transmission means for enabling the terminal device. In this system, the terminal device according to the present invention (for example, a first terminal image value)
Is a display unit for displaying an image of the second interlocutor received on the monitor via the transmission means, and defining a position of the first interlocutor to define at least a distance between the first interlocutor and the display means. A position defining means for defining, and an imaging device for transmitting the photographed image of the first interlocutor to the second terminal device via the transmission means.

Here, the imaging unit adapted to the above-described interactive interactive system may be independent of the display unit as shown in FIG. 2, for example, but as shown in FIG. It may be attached to a part of the display unit by a support mechanism. The configuration of FIG. 7 facilitates miniaturization of the terminal device itself.
Therefore, the terminal device may include at least an imaging unit that images the first interlocutor, and the imaging unit that directs the imaging direction of the imaging unit to the first interlocutor. And a support mechanism for setting the imaging unit at a predetermined position. In particular, (1) the imaging unit is configured to determine the distance between the first interlocutor and the monitor defined by the position defining means, and (2) the line of sight of the first interlocutor and the line of sight of the monitor image of the second interlocutor. A part of the monitor image of the second interlocutor with respect to the first interlocutor within the parallax matching area defined by the above-described parallax angles determined to be coincident and the elements (1) and (2). It is installed in a position where it blocks out. Note that the second terminal device may have a similar configuration, and the first terminal device is configured to be able to remotely control the second terminal device. Furthermore, each terminal device is provided with a sound absorbing member which is arranged around the interlocutor and has a shielding property against external sounds. Therefore,
Psychological obstacles to the leakage of conversation contents to the outside are reduced, and the sound quality is improved.

As described above, the two-way interactive system is constructed by installing the image pickup unit so as to block a part of the monitor image in the above-mentioned line-of-sight matching area between the interactive person and the monitor. Therefore, even if the imaging unit is installed so as to block a part of the human image displayed on the monitor, if the size of the imaging unit is suppressed to a certain value or less, the existence of the imaging unit Has little effect on the two-way conversation.

In the above-described two-way interactive system, the parallax for matching the line of sight of the interlocutor (first interlocutor) with the line of sight of the person (second interlocutor) displayed on the monitor (monitor image). The angle is 4.5 ° or less in the left-right direction of the monitor screen and 1 ° in the direction directly above the monitor screen with respect to the line of sight of the interlocutor.
It is preferable that the direction of 2 ° or less and the direction directly below is 8 ° or less. Further, the parallax angle is set such that the horizontal direction and the vertical direction of the monitor screen are each 3 ° or less with respect to the line of sight of the interlocutor when viewed from the interlocutor. Is more preferable. In particular, as shown in FIG. 8, the angle formed by a line perpendicular to the photoelectric conversion surface 208 of the imaging device 205 and passing through the photoelectric conversion surface 208 and the first interlocutor is within the range of the parallax angle described above. It is.

It is preferable that the line-of-sight matching area where the imaging unit is installed is an area higher than the line of sight of the person who is photographed. In particular, as a result of an experiment by the inventor, the imaging unit that blocks between the interlocutor and the monitor is preferably located near the top of the human image displayed on the monitor as the installation position of the imaging unit when viewed from the interlocutor. Was confirmed.

A distance from the interlocutor specified by the position defining means to the monitor, a predetermined parallax angle at which it is determined that the line of sight of the interlocutor matches the line of sight of the person image displayed on the monitor; The means for installing the imaging unit in the area defined by
Means for moving the imaging unit itself within the line-of-sight matching area between the interlocutor and the person image on the monitor. Second, moving the interlocutor facing the monitor and the person image on the monitor. Means for moving the line-of-sight matching area itself to relatively change the position of the imaging unit and the line-of-sight matching area. Third, moving the person image on the monitor, and similarly, the line of sight of the interlocutor facing the monitor. And means for moving the line-of-sight matching area itself of the human image on the monitor (this third means is effective as an algorithm for moving the video display area in which the human image is displayed when the display control unit divides the screen). ).

The first means includes a processing unit for recognizing a position of a human image displayed on a monitor on the monitor in the terminal device. This can be realized by providing a drive mechanism for relatively moving the human image whose position is recognized by the processing unit. As an embodiment, the first means is performed for each terminal device, and the position of the monitor image of the second interlocutor is recognized by the processing unit of the first terminal device. The second
In some cases, the first terminal device remotely controls the second terminal device by transmitting it to the terminal device.

The second means is provided with a processing unit for recognizing a position on the monitor of the human image displayed on the monitor in the terminal device. This can be realized by providing a mechanism for adjusting the line of sight of the interlocutor photographed with respect to the line of sight in the position defining unit. As an embodiment, the first means is performed for each terminal device, and the position of the monitor image of the second interlocutor is recognized by the processing unit of the first terminal device. The second
In some cases, the first terminal device remotely controls the second terminal device by transmitting it to the terminal device.

Further, in order to realize the third means,
The terminal device includes a processing unit that adjusts a display position of a human image displayed on a monitor. The processing unit specifies a virtual position on the monitor image where the imaging unit should originally exist as a reference point, based on the human image received and displayed on the monitor. Based on the coordinates on the monitor corresponding to the installed position and the coordinates of the obtained reference point on the monitor, the displacement of the human image displayed on the monitor is detected. If it is determined that this person image is displaced from the display position on the monitor where it should be displayed, the coordinates of the reference point on the monitor are used as the starting point, and the monitor corresponding to the installation position of the imaging unit is used. A vector having the above coordinates as the end point is calculated, and the shifted human image is translated on the monitor based on the obtained vector. As an embodiment, a case in which the process is performed on a monitor image of a dialogue person (initial setting before the dialogue is started)
May be performed on the monitor image of the second interlocutor. In particular, at the time of a screen division operation (including an operation of enlarging / reducing a displayed human image) by a display control unit to be described later, the display size of the human image (the monitor image of the first or second interlocutor) is arbitrarily set. When changed, the area for displaying the image can be set at an arbitrary position on the monitor screen. Therefore, when the sight line is matched between the arbitrarily displayed person image and the interlocutor, this second image is displayed. The third means is particularly effective.

The reference point is determined by detecting each display position on the monitor of both eyes of the human image received and displayed on the monitor, and from the center of both eyes to a position above the monitor. The position on the monitor is separated by a predetermined distance (for example, about 10 cm). Further, the reference point is obtained by binarizing a human image received and displayed on the monitor, detecting an outline of the human image, and specifying the position as a position on the monitor corresponding to the top of the human image. Is also good.

The driving mechanism for moving the image pickup unit once recognizes the position of the interlocutor in the processing unit, and sets the angle of the image pickup direction of the image pickup unit based on the recognized position of the interlocutor and the position of the image pickup unit. It may be configured to adjust. By adjusting the angle of the imaging unit by this drive mechanism, the imaging direction of the imaging unit is automatically directed to the line of sight of the interlocutor whose position has been recognized by the processing unit. Further, the drive mechanism for moving the imaging unit detects an average match in the line-of-sight direction, and when moving and aligning, the moving speed is equal to or lower than a predetermined value, and the centering operation is not performed when the shake in the line-of-sight direction is within a predetermined range. (This determination is made by a processing unit or the like). In addition, when the imaging unit moves, the image output from the imaging unit shakes. However, the movement speed is limited, and the movement is not frequently performed, and the movement is suppressed to a necessary minimum. .

Here, in the above-described driving mechanism for moving the imaging unit, it is preferable that a part of a member located between the interlocutor and the display unit is formed of a transparent material.

Next, the position defining means for defining the position of the interlocutor includes a height-adjustable chair (chair provided with a seat height adjusting mechanism) installed at a predetermined distance from the display unit. Also,
A table provided between the interlocutor and the display unit also functions as a position defining unit. In the two-way interactive system, the position defining means is used to move the photographed interlocutor and the display unit by 0.5.
m and 5 m or less. With such a configuration, since the movement of the interlocutor in the front-rear and left-right directions can be sufficiently restricted by the position defining means, the gaze coincidence between the interlocutors in this system is caused by the physical differences of the interlocutors. This is because vertical movement adjustment is most effective.

Since the two-way interactive system is oriented to a counseling system between distant interlocutors, various configurations and arrangements are required to collect useful counseling data and realize an ideal counseling environment. Has functions.

For example, a display control section for controlling display on the display section has a function of enlarging / reducing an image displayed on the display section. Since the display size greatly affects the psychological effect of the user, by setting the size of the video display area on the monitor freely with this display control unit, the degree of freedom of the dialogue can be expanded and the comfortable dialogue can be performed. Environment can be realized. Since the display position of the reduced person image (monitor image) on the monitor is adjusted by the above-described third line-of-sight matching means, the line-of-sight matching state between the first interlocutor and the second interlocutor is maintained. Is done.

The transmission information may be character / graphic information in addition to the image and voice of the interlocutor. This makes it possible for the display unit to display a character or a figure together with a person image based on the character / graphic information received via the transmission unit. That is, by displaying not only images and sounds, but also characters and figures, the dialogue can be further facilitated.

As a method of displaying characters or figures, the characters or graphics may be displayed so as to be superimposed on the displayed human image. Further, the monitor screen may separate an area for displaying characters or figures from an area for displaying a human image. As a mode of this screen separation, (1) the monitor screen is divided in the vertical direction, and a human image is displayed on the upper side, or (2) the monitor screen is divided in the horizontal direction, and the monitor screen is divided in one direction side or in the center part. A mode such as displaying a human image is preferable. Note that the inventor has confirmed that the eye-gaze coincidence between the interlocutor and the human image does not deteriorate even in the modes (1) and (2). As an application example of this aspect, the display unit may have a plurality of configurations, such as including a first monitor for displaying a person image and a second monitor for displaying characters, graphics, and the like.

Further, the interactive interactive system includes a character / graphic input device as a man-machine interface. With this configuration, the display unit can display a character or graphic input from the character / graphic input device via the character / graphic input control unit, and a character corresponding to the character or graphic taken into the device in this way. It is also possible to output information or graphic information to the terminal device of the other party. Note that such a character / graphic input device may be provided only in one of the terminal devices at the first point and the second point where the interlocutor is located.

As described above, since the two-way interactive system is provided with the character / graphic input device, it is possible to carry out a conversation with characters and / or graphics, and to carry out a smoother conversation. Note that, as such a character / graphic input device, a keyboard device and a pen input device (touch panel) arranged near the interlocutor can be adopted. Further, by providing both such a keyboard device and a pen input device so that the keyboard device and the pen input device can be appropriately selected, a smoother conversation can be realized. Further, the character / graphic input device may include a pointing device for inputting a pointer position pointing to a displayed character or graphic. The position indication by the pointer makes it possible to specify the conversation target at each time. By utilizing such a character / graphic input device or the like and recognizing the input voice of the interlocutor, converting it into character information and displaying it, it is effective for establishing a smooth dialogue (or counseling).

The two-way interactive system includes a speaker arranged as a voice output unit in the side of the interlocutor and near the ear. This is because being whispered in the ear improves intimacy. A typical example of the audio output unit is a headphone.

Next, the two-way interactive system includes a telephoto microphone as a voice input unit. A microphone with a table or attached to the chest can be used as the microphone, but a microphone with a table may impair consciousness of conversation. Also, if the chest microphone is too far away, the cord may be damaged. Therefore, it is desirable to arrange a telephoto microphone which does not require the presence of the telephoto microphone at the top of the display unit or to incorporate the microphone into the display unit. In particular, since the line of sight has already been recognized, it is possible to increase the directivity by directing the sound collection direction to the mouth.

The audio / sound data processing unit for taking in an audio signal from the audio input unit has an audio input amplification adjustment function for controlling a sound collection volume. Here, the voice / sound data processing unit automatically adjusts the output volume of the collected voice to a predetermined level. In addition, the voice / sound data processing unit is configured to detect an average level of the collected sound volume at a predetermined time and to automatically adjust the sound volume when a difference between the collected sound volume and the average level is equal to or more than a predetermined value. May be.

At this time, the display unit displays the output volume (volume of the interlocutor itself output to the other party) in the voice / sound data processing unit. If you are unfamiliar with talking to a person on the screen without being able to see the microphone directly, you will be confused as to what volume to use if you are unfamiliar with it. It is planned. In addition, by displaying the output volume of the interlocutor on the monitor screen and allowing the interlocutor to recognize the output volume, it is possible to feed back to the interlocutor whether the output volume is appropriate for transmission.

On the other hand, a sound / sound output control unit for controlling the sound output unit outputs background music (BGM),
It has a function of detecting the conversation volume and adjusting the output volume of the background music. Here, the voice / sound output control unit detects that the conversation volume is substantially zero for a predetermined time, and based on the detection result, at least starts the background music, adjusts the volume of the background music, and adjusts the background music. Adjust the volume of music. Further, the voice / sound output control unit detects the volume of the voice of the interlocutor and adjusts the volume of the background music so as not to hinder the hearing of the interlocutor's voice, thereby impressing the interlocutor's heart. The calming background music may be superimposed on the voice of the other party's interlocutor and output from the voice output unit. In particular, if the background music is started or the volume is changed when silence continues, the tension of the interlocutor can be released and the interlocutor can be calmed down.

Further, in the two-way interactive system, each terminal device includes a printer for printing a display screen of a monitor, and storage means for storing information on the display screen. The storage means is a card-shaped recording medium provided with at least an IC (integrated circuit).
A C card, an optical disk, or a magneto-optical disk can be used, and information during a conversation, particularly information on characters and figures used during the conversation can be stored.

In order to maintain the security of the two-way interactive system, it is preferable that each photographing device has a person authentication unit for confirming the person of the interlocutor. This is because the security of the system can be sufficiently maintained by starting the interactive operation after the person confirmation. Here, the person authentication unit may be configured to read the authentication data from a storage medium in which the authentication data is written from a dedicated reading device, and perform the person confirmation based on the read result. Note that the storage medium is a card-shaped recording medium provided with at least an IC, and for example, an optical card, an IC card, or a magnetic card can be used. Furthermore, this person authentication unit performs (1) person confirmation based on the analysis result of the face of the interlocutor, (2) person confirmation based on the voice analysis result of the voice of the interlocutor, or (3) (4) Person confirmation may be performed based on the text analysis result of the message text of the interlocutor. Such person authentication makes it possible to specify the interlocutor. It is preferable to use the authentication method properly according to the confidential content. The above-described text analysis of the message text means, for example, that the text “X is Y.” is compared with the characters put in X and Y and the database of the person or the encryption database of the specified person. Refers to collation analysis.

Further, by using these card-shaped external recording media such as IC cards, reservations of dialogues, authentication of a designated interlocutor, information on reserved dialogue time, preset dialogue time, etc. Authentication and control of conversation conditions may be performed. Of course, it can be used as a means for electronic payment of fees. In addition, if the recording medium is used for recording the contents of dialogue and homework items, it is convenient for the next counseling and communication. If data such as the conversation start time and the conversation partner (partner party) is input in advance to a recording medium such as an IC card, a search for the partner terminal device on a network such as an electronic bulletin board system is performed based on the data. Can be performed, and based on the search result, the first and second interlocutors are connected to each other at any of the first and second points no matter where they are located, and the first and second interlocutors are connected. It is possible to build a dialogue environment between interlocutors. Further, the name, face (image data), and conversation items (the subject of the conversation, etc.) of the other party's interlocutor are input to the recording medium (IC card, etc.), and these information are stored for a predetermined period (before the start of the dialogue). By displaying the information on the monitor of the interlocutor for only 30 seconds (for example, about 30 seconds), mental preparation for the interlocutor's own dialogue is made, and it becomes easy to enter the subject of the dialogue. At this time, if BGM, voice, and the like are output in conjunction with each other, it is possible to provide a more comfortable conversation environment for the user. It is effective that the above-mentioned monitor display is performed during and / or after the dialogue as necessary. In particular, by performing the above-described monitor display after the end of the dialogue, the interlocutor can smoothly proceed with summarizing the dialogue (the time of the monitor display after the end of the dialogue is different for the interlocutor from the end of the dialogue. It is a mental preparation period until work).

Next, the above-mentioned table as the position defining means is arranged between the interlocutor and the display unit such that one end faces the interlocutor and the other end substantially contacts the display unit. .
Here, the table is set to a height where (1) the interlocutor is higher than the table above the chest when sitting in a chair,
(2) The height above the chest may be set to be higher than the table with the interlocutor standing. With such a configuration, a sense of continuity is generated through the table, so that anxiety caused by losing the relative distance (psychological distance) from the other party is reduced. In this case, it is desirable that the display unit and the table are in contact with each other, but there may be a gap (for example, several tens cm or less) that does not cause a psychological obstacle.

Next, in the two-way interactive system, the terminal device includes an interactive processing unit for statistically analyzing the interactive time of the interlocutor. Here, the interaction processing unit calculates the total interaction time, the average interaction time, and the number of interactions, and displays the total interaction time, the average interaction time, the number of interactions, and the distribution of the number of interaction times on the display unit. You may do it. Furthermore, the voice / sound data processing unit in the terminal device performs statistical analysis processing on the voice spectrum of the interlocutor or the voice spectrum corresponding to the transmitted voice information. Here, the statistical analysis processing refers to analyzing the voice spectrum of the interlocutor or the voice spectrum corresponding to the transmitted voice information with time, and detecting an average spectrum and a voice spectrum that has changed by a predetermined value or more. .

Next, the terminal device according to the present invention is provided with a dialogue processing section for detecting a predetermined term uttered by the interlocutor or a predetermined term in the voice corresponding to the transmitted voice information. In addition, the terminal device may include a position change analysis unit that analyzes a movement trajectory, a movement change amount, a movement speed, or a movement acceleration of a head, an upper body, or a hand of a person or a transmitted person image ( This analysis is performed by a processing unit or the like).

When counseling or the like is performed, a dialogue state analysis such as a dialogue time analysis, a voice spectrum analysis, a predetermined term analysis, or a body motion analysis can be performed to quantify the dialogue state.

In the transmission means in the two-way interactive system, a plurality of transmission paths independent of each other are secured between a first point and a second point which are separated from each other, and at least one of the plurality of transmission paths is provided. , A voice transmission path for transmitting voice information. Here, the transmission means includes a network such as a telephone line network already laid, and furthermore, an electronic bulletin board system (BBS: Bulletin Board Sys- tem) provided with various databases.
tem). In addition, the system includes a backup system that detects a failure in a transmission line that is being used interactively and switches the transmission line to be used. Therefore, when a failure occurs in a transmission path during a conversation, the conversation can be continued using another transmission path. In these cases,
It is preferable to secure at least voice transmission in other transmission paths. In general, line errors are indicated by coaxial cables,
Since it is considered that the cause is an abnormality of the optical fiber cable or the transmission device, it is preferable that the other transmission path is a wireless system.

[0044]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of a two-way interactive system to which a terminal device according to the present invention is applicable will be described below with reference to FIGS.

FIG. 1 is a logical configuration diagram showing the entire interactive interactive system to which the terminal device according to the present invention is applied. This system realizes a two-way dialogue between a talker 3A (first talker) at point A (first point) and a talker 3B (second talker) at point B (second point). System. In particular, this system aims at a counseling system that establishes a close relationship of trust and realizes in-depth communication even with remote interlocutors, so at least one of the interlocutors is a teacher, doctor, or psychologist. Such counselors are assumed.
Therefore, this system has various functions and configurations so that these counselors can provide information necessary for counseling and an ideal counseling environment.

Specifically, at the point A, (a) a chair 4A on which a talker 3A (for example, a counselor) sits;
A table 2A to which an interlocutor 3A goes;
7 and a terminal device 20A provided with a mechanism for matching the line of sight between the interlocutors. On the other hand, at the point B, (a) the chair 4B on which the interlocutor 3B (for example, a client receiving counseling) sits, (b) the table 2B to which the interlocutor 3B goes, and (c) the interlocutor via the transmission means 27 And a terminal device 20B provided with a mechanism for matching the line of sight of the user. Here, the transmission unit 27 includes an optical fiber transmission line 31 that enables large-capacity transmission in order to enable transmission and reception of image information and audio information between the terminal device 20A at the point A and the terminal device 20B at the point B. (Main transmission path)
And a satellite communication transmission line 32 (backup transmission line). The transmission means 27 may be wired or wireless. The transmission means includes an already laid network such as a telephone line network, and furthermore, an electronic bulletin board system (BBS: Bulletin Board) provided with various databases.
System).

At point A, the chair 4A described above functions to define the position of each interlocutor 3A. This chair 4A
Is preferably fixed to maintain a constant distance from the display unit 21A of the terminal device 20A. However, even when the chair 4A is not fixed, the table 2A is installed on the interlocutor 3A and the display unit 21A, so that the chair 4A is not fixed.
It is possible to keep the distance between A and the display unit 21A constant (an experiment by the inventor has confirmed that the existence of the table 2A serves as a reference for the psychological distance of the interlocutor 3A).
The same applies to the configuration of the terminal device 20B at the point B where the other party's interlocutor 3B (for example, a person receiving counseling) is located.

At the point A, the terminal device 20A (1)
A monitor TV1 arranged on the table 2A for displaying an image of the interlocutor 3B based on image information transmitted from the terminal device 20B at the point B via the transmission means 27.
A display section 21A provided with A, and (2) an image of the interlocutor 3A captured by the CCD camera 5A is captured as image information.
The imaging unit 6A for transmission to the terminal device 20B at the point, and (3) the voice of the interlocutor 3B based on the voice information transmitted from the terminal device 20B at the point B via the transmission means 27. Output unit 23A (speaker) and (4) a voice input unit 24A (microphone) for collecting the voice of the interlocutor 3A as voice information and transmitting it to the terminal device 20B at the point B. Furthermore, in order to realize more detailed communication as a counseling system, (5) the interlocutor 3A inputs characters and figures, and the input characters and figures are displayed on the display unit 21A.
And a character / graphic input unit 25 for transmitting the character / graphic information to the terminal device 20B at the point B.
A (an interface such as a keyboard, a pointing device, a touch panel, etc.) and (6) a processing unit 26 that is provided between each of the above components and the transmission unit 27 and performs signal processing and transmission control. The terminal device 20B at the point B is configured similarly to the terminal device 20A at the point A described above.

Further, the terminal device 20 at the point A and the point B
In A and 20B, the configuration of the imaging devices 600A and 600B will be described. For convenience of description, when referring to components common to the terminal devices 20A and 20B at the point A and the point B, for example, as in the imaging device 600,
The description will be made by omitting letters A and B for distinguishing each point. In addition, unless otherwise described, it is assumed that the terminal device 20A at the point A is described in principle,
The duplicate description of the terminal device 20B at the point B having the common configuration is omitted.

The imaging device 600 includes an ultra-small CCD camera 5 (5A) as an imaging unit, and a support mechanism 10 (FIG. 2) for supporting the CCD camera 5 at a predetermined position while supporting the CCD camera 5.
And an imaging unit 6 (6A) for controlling the CCD camera 5. The monitor TV1 (1A) is placed on the table 2 (2A),
(3A) is a chair 4 (4A) provided with a height adjustment mechanism 40 (40A) placed at a distance L (m) from the monitor TV1.
A) Sitting. This distance L (LA) is set to 0.5 m or more and 5 m or less. An outside diameter of 20 mm or less and a length of about 100 m are set between the person 3 to be photographed and the monitor TV 1.
m is provided with a cylindrical microminiature CCD camera 5.
The CCD camera 5 is installed so that its imaging direction is directed to the person 3 to be photographed. The image of the interlocutor 3 captured by the camera 5 is transmitted from the imaging unit 6 to the other interlocutor side as image information (transmitted to the terminal device 20B at the point B via the transmission means 27).
Note that the interval between the display unit 21 and the interlocutor may be slightly shifted depending on the interlocutor. Therefore, in order to cope with such a situation, it is preferable to select a CCD camera 5 having a large depth of focus.

Next, a specific installation position of the micro CCD camera 5 will be described with reference to FIG.

First, the image pickup unit 6 of the image pickup apparatus 600 is set on the display section 21 for displaying the transmitted image of the other interlocutor. The main body of the imaging unit 6 displays an image of the interlocutor photographed by the camera 5 (taken into the imaging unit 6 via the camera cable 50) and a transmitted image of the other interlocutor 3B. In addition to the changeover switch 6a for switching, various adjustment switches 6b are provided. Switch 2
Reference numeral 10 includes a power switch for turning on / off the power of the monitor TV1. The CCD camera 5 is installed by a support mechanism 10 (robot arm) at a predetermined position in front of a monitor screen 8 on which the transmitted image 9 of the other talker 3B is displayed. The monitor screen 8 is 4 inches or more and 4
The screen is about 0 inches.

More specifically, the CCD camera 5 is located at a position w (cm) ahead of the monitor screen 8 and
Is arranged in the vicinity of the head 9a of the interlocutor image 9 displayed in.
The central axis of the cylindrical CCD camera 5 is located at a position h (cm) above the eye position of the interlocutor image 9 indicated by a dotted line.

As described above, since the ultra-small CCD camera 5 is located near the head above the position of the eye of the interlocutor image 9 displayed on the screen 8 of the large monitor TV 1, it is particularly suitable for two-way conversation. No trouble occurs. For example, the CCD camera 5 is provided at about h (about 10 cm) above the eye position (the position shown by the dotted line in the figure) of the interlocutor image 9 displayed on the screen 8, and is provided between the monitor TV 1 and the interlocutor 3. In the case of a system configuration in which the distance L is set to about 2.5 (m), the parallax angle is 2.3 °.
(The distance w between the monitor screen 8 and the CCD camera 5 is 10 (c).
m) does not particularly affect the fluctuation of the parallax angle). In other words, as a result of the experiment, if the eye of the other party (the eye of the image of the other interlocutor 3B displayed on the surface 8) is clearly visible at the time of gaze coincidence, the micro-miniature camera is placed around the head. Even if 5 was provided, it was confirmed that if the screen 8 of the monitor TV 1 was large, there would be little hindrance to the two-way conversation. The size of this screen 8 is 35
It has been confirmed that a good two-way conversation can be realized if the size is about 26 cm or 26 cm or more. In addition, if the mutual interlocutors are acquaintances, a result showing that the screen size is small but the psychological disability tends to be small is obtained, and the screen size may be properly used depending on the application.

Next, the structure of the robot arm which is the camera support mechanism 10 of the image pickup apparatus 600 shown in FIG. 1 will be described with reference to FIG.

This robot arm has a first rod 101
a, a second rod 102a, and a holder 103a that holds the camera 5. The main body of the imaging unit 6 and the first rod 101a are connected to the first joint 10
0, the first rod 101a and the second rod 102a
Is connected to the second joint portion 101, and the second rod 102a and the holder portion 103a are connected to the third joint portion 10.
2 are rotatably connected. In particular, the third joint portion 102 includes a protrusion 103 directly rotatably connected to the holder portion 103a to enable special rotation. The first joint part 100 is rotatable in a direction indicated by an arrow 100b, and the first rod 1
The movement in the direction indicated by arrow 100c of 01a is enabled. The second joint portion 101 supports the second rod 102a and enables the second rod 102a to move in a direction indicated by an arrow 101b. Third joint part 1
02 enables the projection 103 to move in the direction indicated by the arrow 102b. Further, the protrusion 103 and the holder 103
a can change the angle in the direction indicated by the arrow 103b. The robot arm (included in the support mechanism)
The above configuration allows the camera 5 to be installed at a desired position, and also allows the angle of the camera 5 to be adjusted. Therefore, the setting position of the CCD camera 5 and the imaging direction 5a can be freely set by the robot arm.

Next, as an application example of the image pickup apparatus 600, the structure of the image pickup apparatus 601 provided with another support mechanism will be described with reference to FIGS.
This will be described with reference to FIG.

The camera supporting mechanism of the image pickup apparatus is a talker 3B.
Is displayed in front of the surface 8, it is preferable to use a very inexpensive light-transmitting material such as a transparent thread, a tape, a transparent plate of glass, acrylic or the like. This is because the display of the monitor screen 8 is not hindered.

The image pickup device 601 shown in FIG.
A support member 110 made of a light-transmissive material for fixing and supporting the camera 5 at one end and installing the camera 5 at a predetermined position in front of the monitor screen 8;
And a pedestal portion 61 for gripping the other end of the transparent support member 110 and supporting the entire transparent support member 110. In particular, the pedestal 61 is provided with the imaging device 601.
The circuit board 6c which is housed inside the imaging unit 6 and realizes functions such as a data processing unit is also housed in the imaging unit 6. In the drawing, reference numeral 62 denotes an electric signal for driving and controlling the pedestal 61 itself from the circuit board 60 for moving the installation position of the CCD camera 5 from the circuit board 60 (FIGS. 5 and 6).
Are a plurality of wires (ribbons) to be communicated to.

The movement of the CCD camera 5 in the horizontal and vertical directions is realized by the mechanism shown in FIG. That is,
The pedestal 61 supporting the transparent support member 110 is installed on the first moving table 63a, and the position in the horizontal direction is defined by the wire 64. Accordingly, when the first motor 65 pulls the wire 64 in either direction, the pedestal 61 moves in the direction indicated by the arrow 63c. Further, the first moving table 63a is also the first moving table 63a.
It is installed on the second moving table 63b together with the motor 65, and the position in the vertical direction is defined by the wire 67. Therefore, the second motor 66 is connected to the wire 67
Is pulled in either direction, so that the entire first moving table 63a including the first motor 65
Move in the direction indicated by d.

On the other hand, the angle of the CCD camera 5 is adjusted as shown in FIG.
Is realized by the mechanism shown in FIG. That is, the base 6
Reference numeral 1 denotes a pair of plate members 68a and 68b and a distance adjusting unit 69 for defining a distance between these plate members. At this time, the transparent support member 110 is pushed up by the spring member 70 in the direction shown by the arrow 70a. In such a configuration, according to a control signal (electric signal) output from the circuit board 60 via the wiring 62 connected by the connector 62a, the interval adjusting unit 69 moves the plate member 68a in the direction indicated by the arrow 69a, By adjusting the interval of 68b,
The angle of the transparent support member 110 is adjusted in a direction indicated by an arrow 110a. In other words, by adjusting the angle of the transparent support member 110 in the direction indicated by the arrow 110a,
The angle of the imaging direction 5b of the CCD camera 5 can be changed.

The display unit 21 may employ a plasma display or a liquid crystal display instead of the monitor TV. When such a display device is used, FIG.
As shown in (1), a structure in which the CCD camera 200 is previously incorporated in the housing 210 of the display device is preferable. 7 is a monitor screen 22 supported by a housing 210.
0, a through hole 230 is provided at a predetermined position.
A structure is provided in which the tip of the D camera 200 is directly penetrated, and the tip is located in front of the monitor screen 220.

FIG. 8 shows a specific structure. FIG. 8 is a diagram showing a cross-sectional structure of the display device taken along line XX of FIG. As shown in the figure, the barrel 2 of the CCD camera 200
01 is provided with a spherical projection 202 so that it can be attached to the housing 210. An imaging device 205 and an imaging optical system 203 are incorporated in the lens barrel 201. An electric signal detected by the imaging device 205 is transmitted from the electronic circuit unit 206 to an external data processing unit via the camera cable 500. Can be Also, the lens barrel 201
Has a structure in which at least one of the lenses constituting the imaging optical system 203 can be moved in the axial direction of the lens barrel 201, and the focus of the CCD camera 200 is controlled by the adjustment knob 204. Will be

In the above-described interactive interactive system, the photoelectric conversion surface 208 is perpendicular to the photoelectric conversion surface 208 of the imaging device 205 of the camera 200 serving as the imaging unit.
Is within the range of the above-described parallax angle. This line segment 5e substantially coincides with the imaging direction of the camera 200. Then, the camera 200 attached to the display unit centers on a predetermined portion of the lens barrel 201 in a direction indicated by an arrow 207 in the figure in a direction of the lens barrel axis (one line 5e in the figure). Is moved, it is possible to change the imaging direction of the camera 200 (corresponding to the line segment 5e in the figure).

The CCD camera 5 having the above structure
Is a gripping member 2 directly attached to the housing 210.
40 and 250 are attached by gripping the projection 202 of the CCD camera 5. Since the projection 202 of the lens barrel 201 has a spherical shape, the angle of the CCD camera 200 can be freely changed.
The imaging direction 5c of the camera 200 can be directed to a desired direction. FIG. 9 is a view showing an assembly process for realizing the structure shown in FIG. In this figure,
The grip members 240 and 250 are bolts 252, 253 and 25
4 fixed. Further, in the drawing, reference numeral 255 denotes a through hole for bolt insertion provided in the gripping member 250.

FIG. 10 is a diagram showing a terminal device in which the CCD camera 5 and the display unit are integrally formed. FIG.
The terminal device has a camera driving mechanism 2 that drives the CCD camera 200 in the direction (up, down, left, and right) indicated by the arrow 260a.
60 are provided. With this camera driving mechanism 260, the imaging direction 5d of the CCD camera 200 can be changed to the direction shown by the dotted line in the figure. The image captured by the CCD camera 200 is captured by the captured data processing unit 60 and sent to the other terminal device 20B (point B) via the transmission unit 27. In this embodiment, the control of the camera driving mechanism 260 is performed by the imaging data processing unit 6.
Performed at 0. Furthermore, in this embodiment, the imaging data processing unit 60 is configured in the processing unit 26,
It may be configured independently as shown in FIG.

Next, the parallax angle will be described with reference to FIG. In this specification, the parallax angle is, as shown in FIG. 11, the line of sight of the photographed interlocutor 3 directed to the image 9 displayed on the monitor screen 8 and the line of sight to the CCD camera 5. It refers to the angle θ between the line of sight of the interlocutor 3. In other words, it means a shift angle of the installation position of the CCD camera 5 with respect to the line of sight of the interlocutor 3. In addition, the line-of-sight coincidence referred to here means a joint conference of the Institute of Electrical Engineers of Japan in 1972 (No. 19).
98), it means below the allowable limit of unnaturalness due to the installation position of the CCD camera 5. Quantitatively, the parallax angle is 4.5 ° or less in the left-right direction (image 9).
Of the image 9 with respect to the center A of the image 9), 12 ° directly above (the head 9b side of the image 9 with respect to the image center A), and 8 ° directly below (the image center A with respect to the image center A). Nineth body side) and below are within the line-of-sight matching range. Of course, the smaller the parallax angle, the better, and the detection limit is 3 ° or less in the left-right and up-down directions. Therefore, the area where the CCD camera 5 can be installed is
13 is the conical area shown in FIG. Note that this conical area corresponds to the eye 30 of the interlocutor 3 and the image 9 displayed on the monitor screen 8.
(In this embodiment, the center of both eyes of the image 9 is defined as the center point A of the image 9).
The eye-gaze matching area 9b on the monitor screen 8 at a predetermined distance L (m) away from the eye gaze 31
0 position.

On the other hand, the CCD camera 5 is set within a certain area with respect to the eye position (for example, image center A) of the image 9 of the interlocutor 3B at the point B displayed on the monitor screen 8, for example,
There are the following three means for placing the image 9 about 10 cm above the eye position.

The first means is to move the CCD camera 5 itself with respect to the monitor screen 8. In order to realize the first means, the interlocutor 3 displayed on the monitor screen 8
An imaging data processing unit 60 (for example, as shown in FIG. 10) that performs image recognition processing on the position of the image 9 of the interlocutor 3 </ b> A or 3 </ b> B at the point A or the point B in the monitor screen 8. A driving mechanism (for example, a driving mechanism for moving the CCD camera 5 so that the CCD camera 5 is set in the above-described predetermined area in the monitor screen 8 with respect to the position-recognized image 9.
The robot arm shown in FIG. 3 and the mechanism shown in FIGS. 5, 6, and 8) are provided in the imaging unit 6.

The second means is to move the person 3 to be photographed with respect to the monitor screen 8. This second
Means is that the imaging data processing unit 60 operates the monitor screen 8
The position of the image 9 of the interlocutor 3 displayed on the monitor screen 8 is subjected to image recognition processing, and the position of the interlocutor 3 photographed with respect to the image 9 of the interlocutor 3 itself is recognized.
This can be realized by providing a person position defining means for adjusting the position of the person. As the person position defining means, for example, a chair 4 on which the interlocutor 3 sits is preferable. Adjustment of the seat height for each interlocutor is possible by adjusting the seat height adjustment mechanism 40 provided in the chair 4. This is because the installation position of the chair 4 (distance from the monitor screen 8 and the like) is fixed in advance, so that once the interlocutor 3 sits on the chair 4, it is not necessary to adjust the position of the interlocutor 3 in the left-right direction. . In other words, the only uncertain factor for determining the position of the interlocutor is the vertical displacement caused by the physique difference between the interlocutors. Means can be realized.

The third means is to copy the image 9 of the interlocutor 3 (which may be any one of the interlocutors 3A and 3B at the point A and the point B) displayed on the monitor screen 8 to the CC whose installation position is fixed in advance.
It is moved relatively to the D camera 5. This third means, like the above means, first recognizes the position of the image 9 in the monitor screen 8 by the image data processing unit 60 provided in the image pickup unit 6, and then displays the image 9 The position needs to be moved. This allows
The CCD camera 5 can be relatively present in the fixed area.

Next, using the flowchart of FIG.
The initial setting operation of the CD camera 5 (initial setting operation for eye-gaze matching) will be described. FIG. 15 is a diagram for conceptually explaining this setting operation.

First, the interlocutor 3 causes the CCD camera 5 to photograph itself (step ST1), and displays the self image 9 photographed by the CCD camera 5 on the monitor screen 8 of the display unit 21 (step ST2). At this time, if the installation position of the CCD camera 5 matches the line of sight of the interlocutor 3 (a line connecting the eyes of the interlocutor and the center point A of the image 9) (the parallax angle is 0).
゜), the presence of the CCD camera 5 causes a dialogue obstacle.

Then, the CCD camera 5 is moved upward by Δh
(Step ST3), and then re-install the CCD camera 5 in a position that does not cause a dialogue obstruction and within a line-of-sight matching allowable region (for example, the region shown in FIG. 12). That is, the CCD camera 5 is installed so as to overlap the hair portion 9 b of the image 9 as viewed from the interlocutor 3.

As described above, when the installation position of the CCD camera 5 is moved by Δh, the imaging direction of the CCD camera 5 cannot be directed to the actual interlocutor 3. Therefore, the installation angle of the CCD camera 5 is corrected downward by Δθ in accordance with the movement of the CCD camera 5 (step ST4). As described above, the installation position and the set angle of the CCD camera 5 are adjusted in advance so that the parallax angle determined from the relationship between the distance L between the interlocutor 3 and the monitor screen 8 with respect to the movement amount Δh is within 3 °. , The eye-gaze matching state can be maintained.

Specifically, when the interlocutor 3 is a short child, the sitting height of the child is adjusted by the sitting height adjusting mechanism 40 of the chair 4 and the angle of the CCD camera 5 is corrected downward. It is preferable to move the image 9 itself displayed on the monitor screen 8.

On the other hand, the talker 3 displayed on the monitor screen 8
An operation of moving the image 9 of the interlocutor 3A or 3B at the point A or the point B will be described with reference to the flowchart of FIG. FIG. 16 is a diagram for conceptually explaining the image moving process. This image movement processing is performed by the imaging data processing unit 60 of the imaging unit 6 or the processing unit 26 (display control unit 285).

First, a reference point is specified from the image 9 displayed on the monitor screen 8 of the received other party or the user (step ST5). The reference point is a point 10 cm above the center (point A) of the eye position (indicated by a dotted line in the figure) of the image 9 or an outline of the image 9 easily obtained by binarization processing. 9 (CC)
(The position where the D camera 5 should originally exist).

Subsequently, a position 81 corresponding to the actual installation position of the CCD camera 5 is obtained from the absolute coordinates 80 on the monitor screen 8, and the deviation amount between the already obtained reference point position 82 and the actual position 81 is obtained. Is within the allowable range (step ST6). If the amount of shading exceeds the allowable range at this stage, it is determined that the display position of the image 9 is deviated, and the coordinates 82 of the reference point are set as a starting point, and the actual position 81 is determined.
Is calculated as an end point (step ST
7). Then, the vector 8 obtained as shown in FIG.
Based on 3, the displaced image 9 is translated (step ST <b> 8).

For the purpose of the present invention, any one of the above-described first to third means may be used as the means for matching the line of sight between the interlocutors via the transmission means, and these means may be combined. May be used. In the two-way conversation, the distance L of each of the chairs 4 from the monitor TV1 is fixed within a certain range, and the height of each of the chairs 4 is adjusted vertically by the seat height adjusting mechanism 40 by the above-described second means. Then, the position of the face of the image 9 of the interlocutor 3 displayed on the monitor screen 8 is present in a substantially constant area. In addition, as a result of an experiment, the inventor has confirmed that a range in which fine adjustment of a face size, an eye position, and the like for each interlocutor is within several cm to several tens cm. Therefore, for this fine adjustment, it is effective to move the display position of the image 9 of the interlocutor 3 displayed on the monitor screen 8 by the second means described above. This display movement causes the seat height adjustment mechanism 40 of the chair 4 to move.
It is possible to easily achieve mutual gaze matching between the interlocutors simply by adjusting. Further, it is not necessary to provide a complicated moving mechanism and an optical processing device on the side of the CCD camera 5. In the above-described embodiment, the installation position of the CCD camera 5 is set at 10
(Cm), the distance h can be easily changed according to the image display movement by the third means.

The photographing device 600 of this embodiment can be retrofitted to a TV device that is usually commercially available.
Therefore, there is no need to develop a special-purpose device as in the above-described second related art using a display device that switches between reflection and transmission, and the above-described imaging device can be realized at low cost. Further, there is no need for special optical components such as a half mirror and a reflection mirror as in the first and third prior arts described above. Therefore, the apparatus configuration is simplified, and the image of the talker is not darkened by the mirror.
Also, as a result of enlarging the mirror to obtain a wide angle of view,
The problem that the screen display is disturbed does not occur in the imaging device according to the above-described embodiment. Therefore, according to the two-way interactive system, the two-way interactive using the image display device having a large screen can be easily realized.

FIGS. 17 and 18 are views showing the structure of an imaging device 600 that can be retrofitted to a commercially available TV device. As shown in FIGS. 1 and 2 and the like, the imaging apparatus 600 includes a CCD camera 5 for imaging an interlocutor, supports the CCD camera 5, and sets the CCD camera 5 at a desired position. Support mechanism 10 (robot arm) for
Processing unit 26 including imaging data processing unit 60 described above
As a circuit board.

In the imaging device 600, the CCD camera 5
From the terminal 51 via the camera cable 50
Is sent to the processing unit 26 which is electrically connected. The voice of the interlocutor 3 is captured by the microphone 24,
The signal is sent as an electric signal to the processing unit 26 electrically connected to the terminal 241 via the cable 240. on the other hand,
The image data and audio data of the interlocutor sent from the transmission means 27 are taken into the processing unit 26 which is electrically connected to the terminal 611 via the cable 610. Then, the image data and the audio data of the other interlocutor (for example, the interlocutor 3B at the point B) once taken into the processing unit 26 are sent to a commercially available TV device via the cable 650. The terminals 601a (audio terminal) and 601b (image terminal) of the cable 650 are electrically connected to the processing unit 26, while the terminal 602a (audio terminal),
The terminal 602b (image terminal) is connected to a predetermined connector of the TV device.

FIG. 18 is a diagram showing a specific configuration of a connector provided on the back surface of the imaging device 600. As can be seen from this figure, the terminal 51 of the CCD camera 5 is inserted and fixed to the image input connector 52, the terminal 241 of the speaker 24 is inserted and fixed to the audio input connector 242, and sends image and audio data to the TV device. The terminals 601a and 601b of the cable 650 are inserted and fixed in the audio output connector 603a and the image output connector 603b, respectively. The terminal 611 of the cable 610 for transmitting and receiving data to and from the transmission unit 27 is The connector 612 is inserted and fixed. Note that the image input connector 620 and the audio input connector 630 are spare connectors for transmitting and receiving data to and from the transmission unit 27.

Next, a logical configuration diagram of the terminal devices 20A and 20B in the interactive system will be described with reference to FIG. In addition, the system aims at a system that realizes realistic counseling between interlocutors in a remote place. It has the function of The configuration described below is a configuration common to both the terminal devices at the point A and the point B.

First, this terminal device is connected to the processing unit 26.
Is an integrated device including an imaging data processing unit 60. Here, the imaging data processing unit 60 takes in the imaging data from the CCD camera 5 and controls a data processing unit 600a for performing predetermined processing and a driving mechanism 600c for adjusting the set position and the set angle of the CCD camera 5. And a drive control unit 600b.

The main control unit 273 of the processing unit 26 has a person authentication unit 274 (at least an IC (integrat) for managing the security of at least the interactive system.
ed circuit), and also uses information read from the IC card or the like by the reader 271) to analyze and record the contents of the dialogue exchanged between the interlocutors. 275, a backup system 276 for performing crisis management of the whole system (for example,
The main transmission / reception system 288 such as an optical fiber is switched to the wireless system 287), and the display information and voice analysis information handled by the system are stored in a card-shaped recording medium 269 having at least an IC such as a printer 270 or an IC card.
277, integrated information processing for comprehensively managing reception, keyword information, consultation item information guidance, counselor information, charge information, etc. with integrated information D / B (data base) 267 A system 278 is provided. In addition, the main controller 273 displays a human image (including a monitor image of the first and second interlocutors) displayed on the monitor.
It also recognizes the position of.

The processing unit 26 further includes a voice / sound data processing unit 279 (for example, spectrum analysis of voice information) for processing voice information taken in from the microphone 24, and adjusts the installation position of the microphone 24. And a voice / sound output unit 282 (a voice / sound output unit 282) for causing the speaker 23 to output predetermined voice or sound information.
Utilizes dedicated voice / sound information D / B 265 to realize a volume control function 283 and a BGM output function 284 for providing an ideal counseling environment. ), A display control unit 285 (provided with processing functions such as an image enlargement / reduction function) for displaying image data, character data and the like on the monitor 1, and a character / graphic input device 272 as a man-machine interface. And a character / figure input control unit 286 for performing the operation. It should be noted that a plurality of monitors 1 may be provided, and the first and second monitors 1a, 1a,
The display control unit 285 may perform control to display different information for each of b (for example, display image information on the first monitor 1a and display character / graphic information on the second monitor 1b).

The processing unit 26 is individually provided with a dedicated D / B (for example, image information D / B 261, audio / sound information D / B 262, character information D / B 263, backup D / B 264, BGM information D / B 265). , Integrated information D
/ B267, display information D / B266, etc.). The character / graphic input device 272 includes, for example,
There are keyboards, pointing devices, touch panels and special pens.

Hereinafter, the operation of the interactive interactive system as a counseling system will be described in detail based on the configuration of FIG. FIG. 20 is a diagram showing the configuration of the audio output unit 23. By using the speaker arranged on the side of the interlocutor 3 and in the vicinity of the ear, an environment in which the user whispers at the ear is realized, and the intimacy is further improved. The headphone is representative of the audio output unit 23, but the ear independent speaker 12 fixed to the chair 4 by the flexible joint 13 shown in FIG. Note that the speaker may be attached to a surrounding wall (for example, the sound absorbing members 22A and 22B shown in FIG. 1).

The audio output unit 23 has a function 284 for outputting background music (BGM) and a sound / sound output control function 283 for automatically detecting the conversation volume and adjusting the output volume of the background music. The output is controlled by the unit 282. Upon detecting that the conversation volume is substantially zero for a predetermined time, the voice / sound output control unit 282 starts outputting background music, increases the volume of background music, or increases background music based on the detection result. Adjust volume reduction.
Further, the voice / sound output control unit 282 outputs background music that calms the mind of the interlocutor 3 without superimposing it on the voice of the interlocutor 3 without hindering the hearing of the interlocutor 3. be able to. In particular, when the silence continues, the background music is started or the volume is changed.
Is released, and the interlocutor 3 can be calmed down.

The voice input unit 24 is preferably a telephoto microphone. Generally, a microphone with a table or attached to the chest can be used, but a microphone with a table may impair consciousness during counseling. Also, if the microphone on the chest is too far away, the code may be broken. Therefore, a telescopic microphone that does not need to be conscious of its existence is placed on the upper part of the display unit 21 or
It is preferable to incorporate them into Since the line of sight of the interlocutor 3 has already been recognized, it is possible to improve the directivity by directing the sound collecting direction to the mouth.

The voice input section 24 has a voice input amplification adjustment function for controlling the volume of the collected sound, automatically converts the output volume of the collected voice to a predetermined level, and also controls the volume of the collected voice at a predetermined time. The average level is detected, and is controlled by a voice / sound data processing unit 279 having a volume adjustment function for automatically converting the volume when the difference between the collected sound volume and the average level is equal to or more than a predetermined value. The display unit 21 displays the volume to be transmitted from the voice / sound data processing unit 279 to the other interlocutor via the transmission unit 27 under the control of the main control unit 273 of the processing unit 26. . The act of speaking to a person (an image of an interlocutor) on the monitor screen 8 in a state where the microphone 24 is not directly visible makes it difficult to judge how much the user's own volume should be if not used. Therefore, by adjusting the volume of speaking, smooth dialogue is facilitated. In addition, by displaying the output volume (the volume of the audio information sent to the other party) on the screen, it is possible to feed back to the interlocutor whether the output volume is appropriate for transmission.

On the other hand, the character or graphic information input from the character / graphic input device 272 is
6, the character information or graphic information corresponding to the character or graphic input from the character / graphic input device 272 is transmitted and output to the terminal device 20B at the point B. Therefore, the transmission information transmitted on the transmission path further includes character information or graphic information (text data) in addition to human image information (image data) and audio information (audio data). The display unit 21 displays a character or a graphic together with the image 9 of the other party 3 based on the transmitted character information or graphic information.
In this way, by displaying characters or figures together with a human image, counseling is facilitated.

Here, the characters or figures displayed on the monitor TV1 of the display unit 21 may be displayed so as to be superimposed on the image of the interlocutor 3, but as shown in FIGS. Preferably, the display area and the display area of the image 9 of the interlocutor are separated. In particular, when a person image is displayed on a monitor with a different display size, the person image is moved on the monitor by the algorithm shown in FIG. 16 (this algorithm is based on the monitor image of the first interlocutor or the second image). This can be realized by using any of the monitor images of the interlocutors), and the gazes are matched between the first interlocutor and the second interlocutor. When the display area 8 (monitor screen) of the monitor TV1 of the display unit 21 is vertically divided, the image 9 of the interlocutor is preferably displayed on the upper side (see FIG. 21). Alternatively, the monitor TV of the display unit 21
It is also possible to display an image of the interlocutor on one side or the center in the left-right direction of the first display area 8 (see FIGS. 22 and 23). In particular, when the screen is divided into right and left, it is preferable that a character or a graphic on the right side is displayed toward a person who looks at the monitor screen, and an image of the other person is displayed on the left side. In general, humans perform artistic and subjective thinking in the right brain, and perform computational and logical thinking in the left brain. However, visually, the movements of the right and left brains are reversed. Information from the right eye is mainly processed by the left brain, and information from the left eye is mainly processed by the right brain. Therefore, when displaying two types of characters / graphics (logical) and the image of the interlocutor (subjective) simultaneously, it is preferable to display the characters / graphics on the right side and the human figure on the left side.

Further, as shown in FIG.
An image of the interlocutor may be displayed in the upper central portion of the first display area 8. In this case, the display control unit 285 performs image processing so as to reduce the interlocutor image 9.

The display control unit 285 is capable of enlarging or reducing the interlocutor image 9 (which may be either the first interlocutor or the second interlocutor) displayed on the monitor TV1. At this time, a part of the image corresponding to the position where the camera 5 is actually installed is detected near the top of the dialogue image 9 and the dialogue is performed in a state where the detected image part is fixed. Enlarge or reduce the whole person image. At this time, a predetermined area on the monitor TV 1
And at least one of a character and a graphic is displayed in the area on the monitor excluding the area where is displayed.

[0098] According to the above-mentioned separation method, the eye-gaze coincidence does not deteriorate. A display device for displaying an image of the interlocutor;
Naturally, it is also possible to separate the display device for displaying characters or graphics (for example, as shown in FIG.
1b can be independently displayed and controlled). Further, the character / graphic input device 272 may be arranged at one of the terminal devices 20A and 20B at the point A or the point B.

Smooth dialogue is possible by counseling with characters and figures.
It is most preferable that the voice of the interlocutor can be input, recognized, converted to character information and displayed (this function is controlled by the main control unit 273 under the control of the voice / sound output). Unit 282).

As the character / graphic input device 272, FIG.
As shown in FIG. 9, a keyboard device, a pointing device,
A device, and further, a pen input device (touch panel and pen) arranged near the interlocutor can be adopted. Further, by providing both the keyboard device and the pen input device as appropriate, the counseling can proceed more smoothly.

This system aims at enabling ideal counseling between interlocutors at remote locations. Therefore, in order to preserve such a special environment,
This system includes, as a security function, a person authentication unit (a person authentication unit 274 in the processing unit 26) for confirming the person of the interlocutor. After the person confirmation, a counseling operation is started.

In this case, the personal authentication section 274 may read the authentication data from the storage medium 289 in which the authentication data has been written, and may perform a person confirmation based on the read result. The storage medium 289 is a recording medium provided with an IC or the like, for example, an optical card, an IC card, or a magnetic card can be used (in the case of using such a recording medium 289, a dedicated reading device is used in advance. 271
Prepare).

The person authentication section 274 may perform the person confirmation based on the image analysis result of the face of the interlocutor, or may perform the person confirmation based on the voice analysis result of the interlocutor's voice. It is preferable that the confirmation of the person is performed based on the character analysis result of the sign character of the interlocutor. Further, the person confirmation may be performed based on the result of the text analysis of the message text of the interlocutor. Such person authentication makes it possible to specify the interlocutor. It is preferable to use the authentication method properly according to the confidential content.
The above-mentioned message sentence analysis refers to, for example, comparing a sentence "X is Y." with characters put in X and Y against a database of the person or a cipher database of a specified person. Refers to analysis.

Next, the height of the table 2 is set so that the height above the chest is higher than the table when the interlocutor 3 is sitting on the chair 4. However, the height of the table 2 may be set so that the height above the chest is higher than the table when the interlocutor 3 is standing. This is because by performing the conversation while standing, it is possible to expect the effect of the active conversation and the shortening of the conversation. Further, since a sense of continuity is generated between the interlocutor 3 and the monitor screen 8 by using the table 2, a feeling of anxiety caused by losing the relative distance (psychological distance) from the other party is reduced. In this case, the display unit 2
It is desirable that the table 1 and the table 2 are in contact with each other (the configuration in FIG. 1), but the gap (for example, several tens cm
Below).

In the case of counseling or the like, it is preferable that the processing unit 26 conducts dialogue state analysis such as dialogue time analysis, voice spectrum analysis, predetermined term analysis, or body movement analysis. As a result, it becomes possible to quantitatively analyze a psychological change or the like of the interlocutor who is receiving counseling (it also becomes a counselor's data).

At least a part of the surroundings of the interlocutor is provided with a sound absorbing member 22 having a property of blocking external sounds (see FIG. 1). As a result, the psychological obstacle to the leakage of the conversation contents (counseling contents) to the outside is reduced, and the sound quality is improved.

In FIG. 1, two independent transmission paths are prepared as the transmission means 27, but three or more transmission paths may be provided. In addition, a coaxial cable transmission line or another wireless transmission line may be used as the transmission line in addition to the optical fiber transmission line 31 and the satellite communication transmission line 32.

Further, the two-way interactive system is provided with a backup system for detecting a failure of a transmission line during interactive use and automatically switching the transmission line to be used.
By connecting via a plurality of transmission paths, if a transmission path failure occurs during a dialogue (or during counseling), the dialogue can be continued using another transmission path. In such a case, it is preferable that at least voice transmission be ensured in another transmission path. In general, line abnormalities
Since it is considered that the cause is the coaxial cable, the optical fiber cable, or the transmission device, it is preferable that the other transmission path be a wireless system.

Next, a description will be given of the result of an experiment of a two-way image communication dialogue performed using the two-way dialogue type system provided with the eye-gaze matching mechanism according to the above-described embodiment. reference).

The monitor TV 1 for the experiment has a screen diagonal line 10.
A 3.4 cm 40 inch large projection monitor was used. In addition, a micro camera having a camera head having a diameter φ of 12 mm was used as the experimental CCD camera 5.
Each monitor TV1 was installed in each adjacent room, and the distance between the monitors was about 20 m. The distance L between the monitor TV1 and the interlocutor 3 was about 2 m, the upper body of the interlocutor 3 was photographed within a range of 88 cm in width and 66 cm in height, and the image was transmitted to the monitor TV1 of the other party. . Also, CCD
The camera 5 was fixed at a position h = 10 cm above the line of sight of the interlocutor 3, and the height of the chair 4 was adjusted according to the sitting height of each interlocutor 3.

In the two-way conversation performed in such a system configuration, in the basic posture in which the interlocutor 3 is seated on the chair 4 in a normal posture, the parallax angle is in the range of about 2.86 °. However, in practice, the head and upper body of the interlocutor 3 move when speaking. When this movement was actually measured for six interlocutors, the maximum was about ± 15 cm in the horizontal direction and about ± 10 cm in the vertical direction. Therefore,
When the parallax angle including the movement of the interlocutor 3 is calculated in the above system configuration, a maximum of 4.3 is obtained in the horizontal direction of the screen of the monitor TV1.
°, up to 5.7 ° vertically. For this reason, according to the present embodiment, the parallax angle is within a range of 4.5 ° or less in the left-right direction, 12 ° directly above, and 8 ° or less directly below, which are regarded as the allowable range of the eye-gaze coincidence by the Joint Conference of the Institute of Electrical Engineers of Japan. Fits. In addition,
The average value of the movement of the interlocutor 3 is about ± 7 cm in the horizontal direction and about ± 4 cm in the vertical direction, each of which is less than half of the maximum value. Therefore, the parallax angle obtained by this system is considered to have no problem in actual dialogue.
Also, in the present experimental results, the interlocutor did not feel any special discomfort regarding the eye-gaze matching, and the two-way conversation was smoothly performed.

According to this experiment, by matching the eyes of the interlocutors through the transmission means 27, it is possible to realize emotional exchange between the interlocutors with respect to the theme relating to the troubles. It has been confirmed that counseling is also possible with the system. In addition, by using characters or figures together, it is possible to conduct "character and figure recognition dialogue" in which the dialogue is advanced while staring at the letters or figures, and it is possible to deepen the talk in a shorter time and to unite the talk Was confirmed.

In order to further improve this system, the distance L between the monitor TV 1 and the interlocutor 3 is increased to about 2.5 m, and the display of the interlocutor image 9 displayed on the monitor screen by the display control unit 285 is performed. It is preferable that the position is finely adjusted and controlled so that the line of sight of the interlocutor 3 always falls within the parallax angle of 3 ° with respect to the CCD camera 5. According to such a configuration, even when the interlocutor 3 moves slightly, proper gaze matching can be maintained, and it becomes possible to achieve more realistic gaze matching between interlocutors.

The above experiment was conducted by using the large monitor TV1 and taking a certain distance between the person 3 to be photographed and the monitor TV1. When a small monitor, for example, a 10-inch monitor TV is used as the monitor TV1, the image of the interlocutor displayed on the monitor screen 8 naturally becomes smaller.
Therefore, the parallax angle can also be reduced by combining the small monitor TV with an ultra-small CCD image pickup device and setting the size of the image pickup device in the small monitor screen to a certain ratio or less. In this case, the distance L between the interlocutor 3 and the monitor TV1 can be easily reduced to 1 m or less, which is advantageous for saving space. The inventor uses a 14-inch monitor TV to set h =
A counseling experiment was performed under the conditions of 5 (cm) and L = 1.2 (m). As a result, 83% of 18 clients positively evaluated counseling in this manner.

According to the above-described embodiment, it can be retrofitted to the monitor TV1 which is normally shipped, does not require any special optical components such as a half mirror and a reflection mirror, and can be handled only by the CCD camera 5. In addition, if an image processing function is added, an imaging apparatus having a line-of-sight matching mechanism having a line-of-sight matching function can be realized (FIGS. 17 and 18).
reference). In the above embodiment, the configuration in which the imaging device is retrofitted to an existing TV device has been described. However, a configuration in which the imaging unit (camera) is integrated with a part of the display unit may be adopted. That is, a device configuration in which an image pickup unit such as a CCD camera is embedded in a part of a display device such as a liquid crystal television or a plasma display, or a device in which the image pickup unit is previously formed as a part of a device in a partial area of the display device. It may be configured (see FIGS. 7 to 10). In this case, it is necessary to develop a special-purpose machine, but the device configuration is extremely effective for reducing the thickness of the monitor in the future.

The present invention is not limited to the above embodiment, but can be modified. For example, an image printing device (printer 270) that prints a display screen, or a storage unit that stores display screen information (for example, display information D / B
266, a backup D / B 264, an IC card 269). As the storage means, an optical card, an IC card, an optical disk, or a magneto-optical disk can be used. In this case, it is possible to save information during the dialogue, particularly information on characters and figures used during the dialogue.

[0117]

As described above, in the above-described two-way interactive system, when the imaging unit (camera) is installed in a certain area in front of the image of the interlocutor on the display unit (the interlocutor is the image of the other party). However, if the size of the camera is kept below a certain value, the presence of the person being photographed and the imaging unit placed between the display units may cause almost no interaction. This has been achieved based on the experimental result of no influence. Therefore, it is possible to configure a terminal device having a mechanism for easily realizing eye-gaze matching between interlocutors.
Further, in the terminal device according to the present invention, since the display unit and the imaging unit are integrally formed, miniaturization can be easily realized.

Further, by moving the imaging unit by the driving mechanism, adjusting the direction of the line of sight of the interlocutor by adjusting the position (adjustment of sitting height) by the interlocutor, and by moving the image by the display control unit. By moving the display position of the image of the interlocutor displayed on the screen of the display unit, the imaging unit (camera) is located in a certain area of the image of the other interlocutor displayed on the screen as viewed from the interlocutor. Will be installed. Also, by combining these means, it is easy to achieve a parallax angle of 3 ° or less required for ideal visual line matching. In particular, image processing is performed by the display processing unit, and the image of the other party's interlocutor displayed on the screen is moved to a fixed position with respect to the imaging unit, so that more accurate gaze matching and gaze recognition can be performed mechanically. This can be realized without using a simple drive system. Also, the degree of eye-gaze coincidence can be changed and set in accordance with the individual level.

Further, the drive mechanism of the image pickup unit adjusts the angle of the image pickup unit. By adjusting the angle of the image pickup direction of the image pickup unit, the image pickup direction is, for example, the position of which is recognized by the image data processing unit. Automatically pointed at the person's gaze. For this reason, even if the interlocutor moves during the two-way conversation, it is possible to maintain an optimal gaze matching state between the interlocutors.

Further, by using characters or figures together, it is possible to carry out a "character / graphics recognition dialogue (visual communication)" in which the dialogue is advanced while staring at the characters or figures, and the conversation can be deepened in a shorter time. We can plan unity of talk. As described above, by using this system, a smooth dialogue with the emotional exchange via the monitor TV becomes possible. In addition, the system is widely adaptable for multimedia communication, multimedia counseling, multimedia helping, audiovisual counseling, and the like.

[Brief description of the drawings]

FIG. 1 is a logical configuration diagram showing an entire interactive interactive system to which a terminal device according to the present invention is applied.

FIG. 2 is a view showing the image pickup apparatus of FIG. 1 and is a view for explaining a method of installing a CCD camera (image pickup unit) at a predetermined position.

FIG. 3 is a diagram illustrating an example of a support mechanism in the imaging device in FIG. 1, illustrating a structure for setting a CCD camera at a predetermined position.

FIG. 4 is a diagram showing another example of a support mechanism in the imaging device of FIG. 1, and is a diagram showing a structure for installing a CCD camera at a predetermined position.

FIG. 5 is a view showing a structure of a camera moving mechanism of the support mechanism shown in FIG. 4; In particular, this mechanism moves the CCD camera horizontally and vertically.

FIG. 6 is a view showing a structure of an angle adjusting mechanism of the support mechanism shown in FIG. In particular, this mechanism changes the imaging direction of the CCD camera.

FIG. 7 is a view showing a structure of a terminal device according to the present invention. In this example, a CCD lamella has a structure attached to a display unit as a support mechanism for setting a CCD camera at a predetermined position.

8 is a view showing a mounting structure of a CCD camera of the support mechanism shown in FIG. 7, and is a cross-sectional view taken along line XX of FIG. 7;

FIG. 9 is a perspective view showing an assembling process of the support mechanism shown in FIG. 7;

FIG. 10 shows an example of an application of the support mechanism shown in FIG.
FIG. 3 is a diagram illustrating a structure for adjusting a position and an imaging direction of a CD camera.

FIG. 11 is a diagram for describing a parallax angle.

FIG. 12 is a diagram showing an area where a CCD camera is to be installed.

FIG. 13 is a flowchart illustrating an initial setting operation of the CCD camera.

FIG. 14 is a flowchart for explaining an operation for moving an image displayed on a monitor so that the lines of sight coincide between the interlocutors.

FIG. 15 is a flowchart of the CC described in the flowchart of FIG.
FIG. 5 is a diagram for conceptually explaining the initial installation operation of the D camera.

FIG. 16 is a diagram conceptually illustrating an operation of moving an image displayed on a monitor, which is described in the flowchart of FIG. 13;

17 is a diagram illustrating an application example (imaging device) of the imaging unit alone in FIG. 1;

18 is a diagram illustrating a structure of a back surface of the imaging unit illustrated in FIG.

FIG. 19 is a logical block diagram showing the overall configuration of the terminal device shown in FIG.

FIG. 20 is a perspective view showing a configuration of an audio output unit in FIG. 1;

21 is a diagram illustrating an example of a screen configuration of a display screen (particularly, in the case of screen division display) displayed by the monitor of FIG. 1 (part 1).

FIG. 22 is a diagram illustrating an example of a screen configuration of a display screen (particularly, in the case of screen division display) displayed by the monitor in FIG. 1 (part 2).

FIG. 23 is a diagram showing an example of a screen configuration of a display screen (particularly, in the case of split screen display) displayed by the monitor of FIG. 1 (part 3).

FIG. 24 is a diagram illustrating an example of a screen configuration of a display screen (particularly, in the case of screen division display) displayed by the monitor in FIG. 1 (part 4).

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Monitor TV, 2 ... Table, 3 ... Interlocutor, 4 ... Chair, 5, 200 ... Imaging device (CCD camera), 6 ... Imaging unit, 9 ... Interlocutor image, 10, 110 ... Support mechanism, 2
0: terminal device, 21, 220: display unit, 23: speaker, 24: microphone, 25, 272: character / graphic input device,
26 processing unit, 27 transmission means, 60 imaging data processing unit, 210 housing, 230 through hole, 27
DESCRIPTION OF SYMBOLS 1 ... Reading device, 274 ... Person authentication part, 275 dialog processing part, 281,600c ... Drive mechanism, 269,288 ... Recording medium, 279 ... Sound / sound data processing part, 282 ... Speech / sound output control part, 285 ... Display control unit, 600, 6
01 ... Imaging device.

Claims (7)

(57) [Claims] 1. A terminal device applicable to a two-way interactive system in which predetermined data is transmitted / received via predetermined transmission means, wherein the terminal device is provided on a front surface of the housing and transmitted via the transmission means. A display for displaying a captured image of the first interlocutor, a display unit having a through-hole communicating at least the back surface of the monitor and the front surface of the monitor, and an image of a second interlocutor in front of the display unit An imaging unit, a support mechanism for supporting the imaging unit in a state where a part of the imaging unit is inserted into the through-hole, and an eye of the person image among the person images displayed on the monitor. Means for adjusting a display position of the human image so that a portion above the position overlaps the through hole. 2. The imaging unit is attached to the display unit by the support mechanism in a state where the imaging unit is rotatable about a predetermined portion of the imaging unit. The direction is changed in the direction.
The terminal device as described in the above. 3. The terminal device according to claim 2, further comprising a drive mechanism for changing a mounting angle between the display unit and the imaging unit. 4. A method for adjusting a display position of the human image, comprising:
A processing unit for adjusting a display position on the monitor,
The processing unit is configured to determine a virtual position on the image of the first interlocutor where the imaging unit should originally exist based on the image of the first interlocutor received via the transmission means and displayed on the monitor. A coordinate on the monitor corresponding to a position where the imaging unit is attached, and a displacement of the image of the first interlocutor based on coordinates of the obtained reference point on the monitor. Detecting, calculating a vector with the coordinates of the reference point as a starting point and ending with coordinates corresponding to the installation position of the imaging unit, and, based on the obtained vector, The terminal device according to claim 1, wherein the image is translated. 5. The first terminal device includes a display control unit that enlarges or reduces a human image displayed on the monitor, wherein the display control unit is near the top of the human image, The imaging unit detects a part of the human image corresponding to the position where the image is actually installed, and enlarges or reduces the entire human image while fixing the detected part of the human image. The terminal device according to claim 1, wherein: 6. The display control unit displays at least one of a character and a graphic in a predetermined area on the monitor except for an area where the human image is displayed. The terminal device according to claim 5, wherein 7. The terminal device according to claim 1, wherein character information is displayed on the monitor so as to be superimposed on the human image.