JP2003284017A - Terminal unit, two-way dialog type system, headset, and multi-window display method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a terminal unit, etc., having a structure for making assistant images utilizable without obstructing a remote place dialog from proceeding through specified transmitting means. <P>SOLUTION: The terminal unit comprises: a fixed imaging unit 5 for imaging a dialoging person to realize a close two-way dialog between remote places; and a movably placed imaging unit 910 for imaging the assistant images, such as still or moving pictures helpful for progressing the two-way dialog. The imaging units are disposed separately. Thus, the assistant information are utilizable, without obstructing a smooth two-way dialog. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a bidirectional interactive system for realizing a remote dialogue between terminal devices connected via a predetermined transmission means, a terminal device applicable to the bidirectional interactive system, and the like. It is about.

[0002]

2. Description of the Related Art In recent years, a remote learning mode has been established between a teacher and one or more students between remote areas via transmission means such as English conversation and supplementary training. In such distance learning, the teacher and the student proceed with the learning while watching the image of the interlocutor. Therefore, for example, it is suitable for the distance learning performed through the transmission means disclosed in Japanese Patent No. 3074677. An interactive system is known.

On the other hand, in distance learning as described above,
Teachers need to provide students with additional information such as presentation of tasks, scoring, and evaluation. For example, Japanese Patent No. 2970
As disclosed in Japanese Patent No. 661, there has been proposed a display method capable of presenting not only the image of the interlocutor but also various related information.

[0004]

DISCLOSURE OF THE INVENTION The inventor has found the following problems as a result of detailed examination of the above-mentioned prior art. That is, in remote learning realized by transmitting images of the interlocutor captured by the image capturing unit to each other, the image captured from only the image capturing unit such as the CCD camera capturing the interlocutor himself is used. There was a case that learning with sufficient effect could not be realized by using it. For example, when teaching knitting. Usually, in such a remote dialogue, the interlocutor (leader) presents the materials at hand and demonstrates knitting, etc.
Empirically, there are many cases in which learning efficiency is improved by showing the above. However, if an image of a different object is presented to the other party's interlocutor only by an imaging unit such as a CCD camera that images the interlocutor himself, the interlocutor himself takes an image so as to capture different images with one imaging unit. Since it is necessary to move the target, it is possible that both parties can end up in a dialogue environment where they cannot concentrate on remote dialogue.
As described above, in the conventional two-way interactive system provided with only the image capturing unit that captures the image of the interlocutor himself, as a result, there is a problem that it may be difficult to smoothly proceed with the remote communication.

The present invention has been made to solve the above-mentioned problems, and enables the use of auxiliary images in a remote dialogue conducted through a predetermined transmission means without hindering the progress of the dialogue. A terminal device having a structure for interactive interactive system including the terminal, headset,
An object of the present invention is to provide a multi-window display method, a computer program that executes the multi-window display method, and a recording medium having the computer program recorded therein.

[0006]

According to the two-way interactive system of the present invention, the interlocutor images are mutually transmitted between the terminal devices arranged at remote locations through the predetermined transmission means. , Indirectly realize the dialogue. In this interactive interactive system, at least one of the terminal devices includes a display unit, a first imaging unit, a second imaging unit, and a video processing device. The first imaging unit is fixedly arranged at a position facing the interlocutor in order to capture an image of the interlocutor located in front of the display unit. The second image capturing unit is movably arranged in front of the display unit so as to capture an image of the target object located in front of the display unit excluding the interlocutor. The video processing device displays on the monitor screen of the display unit at least an image of the other party's interlocutor who is in a remote conversation with the interlocutor.

Further, it is preferable that the second image pickup section is fixed in a relative position with respect to the head of the interlocutor located in front of the display section. For this reason, it is preferable that the terminal device includes a second imaging unit and a headset including a support unit so that the relative position of the second imaging unit is fixed with respect to the interlocutor, particularly the interlocutor's head. .

The above-mentioned video processing device can perform various display controls, but in particular, implements the multi-window display method according to the present invention. That is, the multi-window display method according to the present invention is characterized in that, on the monitor screen of the display unit, the window of the image of the other party's interlocutor and the window of the video imaged by the second imaging unit are simultaneously displayed. . In this case, the interlocutor does not have to intentionally look at his / her hand (there is no need to largely shift the line of sight from the image of the interlocutor on the other side), and even if he / she frequently confirms this image at hand, it significantly affects the line-of-sight matching state. There is no. Further, in the multi-window display method, a part of the video imaged by the second imaging unit is cut out, and the cut-out video is enlarged or reduced so as to match the window size set on the monitor screen of the display unit. It is also possible.

Further, the window display method as described above may be a program executed by a computer or the like, and in this case, the program may be distributed via a network whether wired or wireless, Also, C
It may be stored in a recording medium such as a D, a DVD, or a flash memory.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION A terminal device according to the present invention,
Each embodiment of a two-way interactive system including it will be described in detail with reference to FIGS. In the description of the drawings, the same parts will be denoted by the same reference symbols and redundant description will be omitted.

The terminal device according to the present invention realizes a remote dialogue between the terminal devices connected via a predetermined transmission means, and is applicable to, for example, the two-way interactive system described in Japanese Patent No. 30744677. Is.

First, the bidirectional interactive system to which the terminal device is applicable will be described. Note that this two-way interactive system is a system that realizes remote dialogue in a state where the line-of-sight coincides with each other. Any interactive interactive display system can be applied.

FIG. 1 is a logical block diagram showing the entire interactive interactive system that enables line-of-sight matching between interlocutors at remote locations. This system realizes a two-way dialogue between an interlocutor 3A (first interlocutor) at point A (first point) and an interlocutor 3B (second interlocutor) at point B (second point). It is a system that does. In particular, this system is a counseling system that establishes intimate relationships of trust even with interlocutors in remote areas and enables close communication, and a system that aims at home learning. , Is supposed to be a counselor such as a psychologist. Therefore, this system has various functions and configurations so that these counselors can provide information necessary for counseling and an ideal counseling environment.

Specifically, at the point A, (a) a chair 4A (having a height adjusting mechanism 40A) on which an interlocutor 3A (for example, a counselor) sits, and (b) a table 2A to which the interlocutor 3A faces. (C) The terminal device 20A having a mechanism for matching the line of sight between the interlocutors via the transmission means 27 is installed, and the interlocutor 3A is isolated by the partition 22A. In addition, the interlocutor 3A has a CC for capturing an image of the hand separately from the CCD camera 5A for capturing the interlocutor.
A D camera 910A is attached. On the other hand, at the point B, (a) a chair 4B on which an interlocutor 3B (for example, a client who receives counseling) sits and a (height adjusting mechanism 40)
B), (b) Table 2B where the interlocutor 3B faces
And (c) a terminal device 20B equipped with a mechanism for matching the lines of sight between the interlocutors via the transmission means 27, and the interlocutor 3B is isolated by the partition 22B. In addition, the interlocutor 3B also has a CCD for imaging the interlocutor.
In addition to the camera 5B, a CCD camera 910B for picking up an image of the hand is attached. Here, the transmission means 27
Is the terminal device 20A at the point A and the terminal device 20B at the point B
An optical fiber transmission line 31 (main transmission line) and a satellite communication transmission line 32 (backup transmission line) that enable large-capacity transmission are included in order to enable transmission and reception of image information and audio information to and from. The transmission means 27 may be wired or wireless. In addition, the transmission means includes a network such as a telephone line network that has already been installed, and an electronic bulletin board system (BBS: Bulle
tin Board System).

At point A, the chair 4A described above functions to define the position of the interlocutor 3A. This chair 4A is preferably fixed in order to keep a constant distance from the display unit 21A of the terminal device 20A. However, even if the chair 4A is not fixed, the chair 4A can be installed by installing the table 2A on the interlocutor 3A and the display unit 21A.
It becomes possible to maintain a constant distance from the display unit 21A (the fact that the existence of the table 2A serves as a reference for the psychological distance of the interlocutor 3A has been confirmed). The configuration of the terminal device 20B at the point B where the other party's interlocutor 3B (for example, a person who receives counseling) is the same.

At the point A, the terminal device 20A (1)
A monitor TV arranged on the table 2A for displaying the image of the interlocutor 3B based on the image information transmitted from the terminal device 20B at the point B via the transmission means 27.
Display unit 21A equipped with 1A, (2) video of interlocutor 3A taken by CCD camera 5A and CCD camera 910A
The image pickup unit 6A for taking in each of the images captured at step 11 as image information and transmitting the image information to the terminal device 20B at the point B, and (3) being transmitted from the terminal device 20B at the point B via the transmission means 27. Voice output unit 23A for outputting the voice of the interlocutor 3B based on the received voice information
(Speaker), and (4) a voice input unit 24A (microphone) for collecting the voice of the interlocutor 3A as voice information and transmitting it to the terminal device 20B at the point B. In addition, this system, as a counseling system, realizes more detailed communication.
Input a character or a graphic, display the input character or graphic on the display unit 21A, and transmit the character or graphic information to the terminal device 20B at the point B as a character or graphic information (keyboard, pointing, (Device, interface such as a touch panel), and (6) a video processing device 26A provided between each of the above-described components and the transmission means 27 and performing signal processing, transmission control, video data editing work, and the like. The terminal device 20B at the point B also
The terminal device 20A at the point A has the same configuration as that of the terminal device 20A. Specifically, the image pickup device 600B including the display unit 21B, the CCD camera 5B and the image pickup unit 6B, the voice output unit 23B, the voice input unit 24B, and the character graphic. The input unit 25B and the video processing device 26B are provided.

Further, the terminal devices 20 at the points A and B
In A and 20B, the configurations of the imaging devices 600A and 600B (imaging means) will be described. For convenience of description, when referring to components common to the terminal devices 20A and 20B at points A and B, for example, the imaging device 60
The description will be made by omitting the letters A and B that distinguish each point like 0. In addition, unless specifically described, the terminal device 20A at the point A is described in principle, and the terminal device 20 at the point B having a common configuration is described.
Overlapping description of B is omitted.

This image pickup device 600 includes an ultra-small CCD camera 5 (5A) which is an image pickup section, and a support mechanism 10 (FIG. 2) for installing the CCD camera 5 in a supported state at a predetermined position.
Reference unit) and an image pickup unit 6 (6A) for controlling the CCD camera 5. The monitor TV 1 (1A) is placed on the table 2 (2A), and the interlocutor 3
(3A) is a chair 4 (4) provided with a height adjusting mechanism 40 (40A) placed at a distance L (m) from the monitor TV 1.
Sitting in A). This distance L (LA) is set to 0.5 m or more and 5 m or less. An outer diameter of 20 mm or less and a length of about 100 mm between the interlocutor 3 and the monitor TV 1 to be photographed.
A cylindrical microminiature CCD camera 5 is provided. Further, the CCD camera 5 is installed so that its imaging direction is directed to the interlocutor 3 who is imaged. Then, the image of the interlocutor 3 imaged by the camera 5 is transmitted as image information from the imaging unit 6 to the other interlocutor side (transmitted to the terminal device 20B at the point B via the transmission means 27).
Depending on the interlocutor, the space between the display unit 21 and the interlocutor may be weakened. Therefore, in order to cope with such a situation, it is preferable that the CCD camera 5 has a deep focal depth.

Next, a specific installation position of the microminiature CCD camera 5 will be described with reference to FIG.

First, the image pickup unit 6 of the image pickup device 600 is installed on the display section 21 for displaying the transmitted image of the other interlocutor. An image of the interlocutor himself / herself taken by the camera 5 is captured in the body of the imaging unit 6 (captured by the imaging unit 6 via the camera cable 50)
In addition to the changeover switch 6a, various adjustment switches 6b are provided for switching between the display of 1) and the display of the transmitted image of the interlocutor 3B. The switch 210 includes a power switch for turning on / off the power of the monitor TV 1. The CCD camera 5 is installed by a support mechanism 10 (robot arm) at a predetermined position in front of the monitor screen 8 on which the transmitted image 9 of the interlocutor 3B is displayed. The monitor screen 8 can be sufficiently used if it has a size of about 4 inches.

Specifically, the CCD camera 5 is at a position forward w (cm) from the monitor screen 8, and the monitor screen 8
It is arranged near the head 9a of the interlocutor image 9 displayed in FIG.
The central axis of the cylindrical CCD camera 5 is located at a position h (cm) above the eye position of the interlocutor image 9 shown by the dotted line.

As described above, since the microminiature CCD camera 5 is located near the head above the eye position of the interlocutor image 9 displayed on the screen 8 of the large monitor TV 1, it is particularly suitable for two-way dialogue. No trouble will occur. For example, the CCD camera 5 is provided above the eye position (the position shown by the dotted line in the figure) of the interlocutor image 9 displayed on the screen 8 at h = about 10 (cm), and between the monitor TV 1 and the interlocutor 3. In the case of the system configuration in which the distance L is set to about 2.5 (m), the parallax angle is 2.3 °.
It is possible to sufficiently fall below the detection limit parallax angle of 3 ° (the interval w between the monitor screen 8 and the CCD camera 5 is 10 (c
Even if there is about m), there is no particular effect on the variation of the parallax angle). That is, if the eyes of the other party (the eyes of the image of the other interlocutor 3B displayed on the monitor screen 8) can be clearly seen at the time of line-of-sight matching, even if the microminiature camera 5 is provided near the head, the monitor It has been confirmed that if the screen 8 of the TV 1 is large, there is almost no hindrance to the interactive dialogue. As a result of the experiment, the size of this screen 8 is 35 cm in width and 26 cm in height.
It has also been confirmed that good two-way dialogue can be realized if the size is at least a certain level. It should be noted that if mutual interlocutors are acquaintances, there is a result that there is a tendency that there are few psychological obstacles even if the size of the screen is small, and it is advisable to properly use the screen size according to the application.

Further, the parallax angle will be described with reference to FIG. In this specification, the parallax angle means, as shown in FIG. 3, the line of sight of the interlocutor 3 to be photographed, which is directed to the interlocutor image 9 displayed on the monitor screen 8, and the direction of the CCD camera 5. The angle θ formed by the line of sight of the interlocutor 3 is defined. In other words, it means the shift angle of the installation position of the CCD camera 5 with respect to the line of sight of the interlocutor 3. In addition, the line-of-sight matching here means the 4th Japan Electrical Engineers Union Meeting (N)
o. 1998), it means that the unnaturalness due to the installation position of the CCD camera 5 is below the allowable limit. Quantitatively, the parallax angle is 4.5 ° or less in the left and right direction (both temple sides of the image 9 with respect to the eye center A of the interlocutor image 9), and 12 ° directly above (with respect to the image center A, The head 9b side of the image 9) and 8 ° below (the body side of the image 9 with respect to the image center A) or less are within the line-of-sight matching range. Of course, the smaller the parallax angle, the better, and the detection limit is 3 ° or less in the horizontal and vertical directions. Therefore, the area in which the CCD camera 5 can be installed is the conical area shown in FIG. In addition,
This conical area is the center point A of the interlocutor image 9 displayed on the monitor screen 8 and the eye 30 of the interlocutor 3 (in this embodiment, the center of both eyes of the interlocutor image 9 is the center point of the interlocutor image 9). Defined as A), the line of sight 31 of the interlocutor 3 connecting
It is defined by the line-of-sight matching region 9b on the monitor screen 8 and the position of the eye 30 of the interlocutor 3 when the monitor is separated by a predetermined distance L (m). If the size of the interlocutor image 9 displayed on the monitor screen 8 is small, the CCD camera 5 and the head of the interlocutor image 9 do not necessarily have to overlap each other. Rather, the head of the interlocutor 9 is hidden by the CCD camera 5 and becomes difficult to see. As described above, the condition for line-of-sight matching is that the CCD camera 5 and the eyes of the interlocutor image 9 are present in the area with a parallax angle of 3 °, and specifically, about 3 cm at a position 50 cm away from the screen.
The degree (h in FIG. 2). Therefore, within this range, the line-of-sight matching state is maintained even if the CCD camera 5 and the head of the interlocutor image 9 are displaced. This interlocutor image 9 and CC
The D camera 5 can be displaced by, for example, adjusting the image display position on the monitor screen 8 or adjusting the support portion of the CCD camera 5 to change the position of the CCD camera 5 itself.

FIG. 5 is a conceptual diagram for explaining the configuration of a terminal device applicable to the interactive interactive system according to the present invention. Note that, in the following description, since the configuration corresponds to both of the terminal devices 20A and 20B shown in FIG.

This terminal device, as shown in FIG.
The display unit 21, the imaging device 600 (corresponding to the imaging devices 600A and 600B in FIG. 1), and the CC that is the second imaging unit.
D camera 910 (CCD camera 910A in FIG. 1,
910B) and a video processing device 26 (corresponding to the video processing devices 26A and 26B in FIG. 1). At least the image of the other party's interlocutor is displayed as a window on the monitor screen 8 of the display unit 21 by the video processing device 26. The imaging device 600 is a first device for imaging the interlocutor 3.
The monitor screen 8 includes a CCD camera 5 that is an image pickup unit, a support unit 10 that supports the CCD camera 5 in front of the monitor screen 8, and an image pickup unit 6 that captures an image picked up by the CCD camera 5. CC fixed in front of
The D camera 5 images the interlocutor 3 existing in the imaging area A1. On the other hand, the CCD camera 910, which is the second imaging unit
Is a headset 95 attached to the head of the interlocutor 3 so as to be movable relative to the monitor screen 8, for example.
It is fixed to the support unit of 0 (may be headphones). The image pickup direction of the CCD camera 910 is, for example, directed to the hand of the interlocutor 3, and the image pickup area A2 of the CCD camera 910 is set.
An image of the bullet 920 or the like existing inside is captured. The video processing device 26 displays at least the window 800A of the image of the interlocutor on the other side on the monitor screen 8 of the display unit 21.

Conventionally, when an auxiliary camera such as the CCD camera 910 is used, one hand of the interlocutor is restrained by the hand camera, and other work and operation (writing characters, showing work, etc.) are performed at the same time as photographing. Had a problem. On the other hand, as shown in FIG. 5, when the CCD camera 910 is attached to the support portion of the headset 950, the interlocutor has both hands free, and other tasks and operations can be performed in parallel with the shooting. You will be able to do it. Particularly, in the headset 950 according to the present invention, the CCD camera 910 is attached to the head of the interlocutor. This is because the head is rotatable, and further, when the interlocutor sits down and works with a natural body, the movement thereof is extremely reduced. As a result, once the imaging angle of the CCD camera 910 is adjusted so that the interlocutor can take images of the material at hand in a natural sitting posture, the CCD camera 910 can then take an image with the natural fine adjustment of the head. Since the range can be adjusted, the interlocutor can easily perform hands-free work as well as interact with the other interlocutor.

The video processing device 26 can implement the multi-window display method according to the present invention. Specifically, the multi-window display method according to the present invention, the display unit 21.
On the monitor screen 8 of the
A video window 880B imaged by the CCD camera 190 is also displayed at the same time with 0A. in this case,
Since the interlocutor 3 can simultaneously view the image of the other party's interlocutor and the image at hand displayed on the monitor screen 8 on the monitor screen 8, the interlocutor 3 can proceed with the remote communication while substantially maintaining the line-of-sight matching state. Will be possible. That is, interlocutor 3
Does not need to intentionally look at the hand (it is not necessary to greatly shift the line of sight from the image of the interlocutor on the other side), so even if this image at hand is frequently checked, it does not significantly affect the line-of-sight matching state. Furthermore, in the multi-window display method, a part of the image captured by the CCD camera 910 is cut out, and the cut-out image is enlarged or reduced so as to match the window size set on the monitor screen 8 in the display unit 21. Image processing may be performed.

For example, when the CCD camera 910 captures an image of the material at hand of the interlocutor 3, the CCD camera 910
Is set so that its imaging direction and depth of focus substantially match the desk surface. As a result, the interlocutor 3 can confirm the image on hand which is displayed as a window in a partial area on the monitor screen 8. Further, when the image in the window is displaced, the interlocutor himself can adjust the position of the material at hand to easily correct the image. The CCD camera 910 attached to the headset 950 has a diameter of 1
If the lens diameter is about 0 mm, the dimension and weight do not affect the conversation.

FIG. 6 is a block diagram showing the configuration of the terminal device according to the present invention. This terminal device includes a display unit 21 such as a monitor (corresponding to the display units 21A and 21B in the two-way interactive system shown in FIG. 1) and a video processing device 26.
(Corresponding to the image processing devices 26A and 26B in the two-way interactive system shown in FIG. 1) and the external recording device 700.
And a pointing device 900 such as a mouse and a tablet 950, and an imaging device. Note that this image pickup device is an image pickup unit (CCD camera) for picking up an image of an interlocutor.
5 and an imaging unit (CC for imaging the hands of the interlocutor, etc.
D camera 910).

Further, the video processing device 26 receives data for taking in image information from the image pickup unit 6 included in the image pickup device and transmitting / receiving image information such as the image of the interlocutor on the other side via the transmitting means 27. Output unit 261 (in the figure, I /
O) and a control unit 26 for performing an image display operation, etc.
3. A database 262 (D / B in the drawing) for storing character data, video data, etc. prepared in advance by the interlocutor, in addition to the transmitted / received image information, and an instruction of the control means 263 to the display unit 21. And a pointing device 9 such as a mouse for displaying a predetermined pattern in accordance with
00 and a tablet 950, a data input / output unit 265 (I / O in the figure) that enables the reception of position information and the like and the data exchange with the external recording device 700. The external recording device 700 includes, for example, a magnetic tape, a CD, an optical disk, a hard disk, a DVD, a flash memory and the like.

Specifically, the I / O 2 is transmitted via the transmission means 27.
When 61 receives the image data of the other party's interlocutor, the control section 263 displays the other party's interlocutor image on the monitor screen 8 of the display section.
Show window on top. That is, the control unit 263 first instructs the drawing unit 264 to display the monitor screen 8 of the display unit 21.
Display control of the received interlocutor image is performed so that a window is displayed at a position previously designated by the mouse 900 or the like. In the display control by the control unit 263,
The display position is adjusted so that the head portion above the eyes of the image of the interlocutor on the other side and the CCD camera (imaging unit) 5 previously installed in front of the monitor screen of the display unit 21 overlap each other, and the transmission unit 2
The line-of-sight matching state via 7 is secured.

On the other hand, the control unit 263 transmits the image data of the interlocutor imaged by the CCD camera 5 through the I / O 261 to the other terminal device through the transmission means 27, and the CCD camera 910 (imaging unit). ) Captures the image data of the interlocutor's hand imaged by (1). At this time,
The drawing unit 264 places the interlocutor's hand image captured according to the instruction of the control unit 263 at a predetermined position on the monitor screen 8.
A window is displayed with the image of the other party's interlocutor.

The control unit 263 is capable of various display controls. For example, as shown in FIG.
On the monitor screen 8 of No. 1
00A (CCD on the monitor screen 8 so that the line of sight can be matched
(Displayed with the camera 5 partially overlapping)
It is also possible to simultaneously display the window 800B of the image on hand captured by the CCD camera 910. In this case, the interlocutor does not have to intentionally look at his / her hand (there is no need to largely shift the line of sight from the image of the interlocutor on the other side), and even if he / she frequently confirms this image at hand, it significantly affects the line-of-sight matching state. There is no.

In the window 800B on the monitor screen 8, a moving image other than a still image may be displayed as auxiliary information. In particular, if an image capture button is displayed on the monitor screen 8, the interlocutor can click this button to digitally record a part of the image captured from the CCD camera 910 and handle it as a still image. become. When the CCD camera 910 captures an image of a textbook placed at the interlocutor's hand,
It is preferable that one page is recorded or the necessary portion is enlarged so as to match the size of the window 800B on the monitor screen 8. By using this enlarged image, teaching and learning in remote dialogue becomes easy.

Further, as the auxiliary information imaged by the above-mentioned CCD camera 910, it is preferable that it is a moving image in remote instruction or remote learning such as knitting instruction.
In particular, there are many cases in which it is easier to understand the image viewed from the interlocutor's own line of sight than the image viewed in a face-to-face situation such as work such as knitting (this also applies to learning calligraphy and handwork). I can say). In the case of this knitting learning, the CCD camera 910 that captures the image as the auxiliary information is arranged in advance so as to capture the hand of the interlocutor, so that more detailed work information can be transmitted to the other interlocutor. Of course, the CCD camera 910 is installed in the external recording device 700 and the D / B 262.
By recording the video data of, it becomes possible to perform individual learning and slow-motion reproduction after the end of the dialogue (OFF-LINE state). In particular, when the user becomes familiar, the explanation and the like can be performed in a state where the line of sight of the interlocutor's image in the window 800A matches, which is effective for remote instruction. In this case, the CCD camera 910 may be positioned so as to capture an image of the hand of the interlocutor when the interlocutor faces the monitor screen 8. In particular, by making the image pickup direction of the CCD camera 910 coincide with the line of sight of the interlocutor, it is possible to display and explain the movement at hand without feeling uncomfortable.

Further, in the case of learning the calligraphy, since the hand is generally looked at, the CCD camera 910 is designed so that the hand of the interlocutor can take an image while the interlocutor himself keeps his head down (state where the calligraphy is being performed). Position adjustment is performed. For example, once the image is written, the image of the CCD camera 910 is recorded until it is completed, and in the process of reproducing the recorded image data, the explanation will be given in a state of line-of-sight match with the image of the interlocutor in the window 800A. And a higher learning effect can be obtained.

[0037]

As described above, according to the present invention, a fixed image pickup unit for picking up an interlocutor in order to realize an intimate two-way dialogue between remote places, and useful for the progress of this two-way dialogue. Since it is configured to separately arrange the movable imaging unit that captures auxiliary images such as still images or moving images, the interlocutor needs to shift the line of sight greatly from the image of the interlocutor during the conversation. In addition, there is an effect that auxiliary information can be used without hindering smooth interactive dialogue.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a two-way interactive system that realizes line-of-sight matching between remote places via transmission means.

FIG. 2 is a diagram showing the image pickup device shown in FIG. 1, and a diagram for explaining a method of installing a CCD camera for line-of-sight matching.

FIG. 3 is a diagram illustrating a parallax angle.

FIG. 4 is a diagram showing an area where a CCD camera is to be installed.

FIG. 5 is a conceptual diagram for explaining a configuration of a terminal device in the two-way interactive system according to the present invention.

FIG. 6 is a block diagram showing a configuration of a terminal device in the two-way interactive system according to the present invention.

FIG. 7 is a diagram showing an example of a monitor screen displayed as a window by the video processing device shown in FIG. 6 in each terminal device of the interactive interactive system according to the present invention.

[Explanation of symbols]

8 ... Monitor screen, 21 (21A, 21B) ... Display section, 3
(3A, 3B) ... Interlocutors A, B, 5A, 5B, 910
A, 910B ... CCD camera (imaging unit), 6 (6A, 6)
B) ... Imaging unit, 600 (600A, 600B) ...
Imaging device, 26 (26A, 26B) ... Image processing device, 2
0 (20A, 20B) ... Terminal device, 262 ... D / B, 2
63 ... Control part, 900 ... Mouse (pointing device), 920 ... Tablet, 950 ... Headset.

Claims (9)

[Claims] 1. A terminal device in a two-way interactive system that realizes remote dialogue by transmitting and receiving images of each other via a predetermined transmission means, the terminal device being located in front of the display unit. An imaging unit for imaging the interlocutor, a first imaging unit fixed in front of the display unit, and an imaging unit for imaging an object located in a space between the interlocutor and the display unit. And a second image pickup unit movably arranged in front of the display unit, and for displaying on the monitor screen of the display unit a window of at least an image of the other-side interlocutor who is in a remote conversation with the interlocutor. Terminal device including the image processing device of. 2. The video processing device, on the monitor screen of the display unit, simultaneously displays the window of the image of the other party's interlocutor and the window of the video imaged by the second imaging unit. The terminal device according to claim 1. 3. The video processing device cuts out a part of the video imaged by the second imaging unit and enlarges the cut-out video so as to match the window size set on the monitor screen of the display unit. Alternatively, the terminal device is reduced in size. 4. A two-way interactive system including the terminal device according to claim 1. 5. A headset applied to the terminal device according to claim 1, wherein the second imaging unit and a relative position of the second imaging unit with respect to an interlocutor are set. A headset having a holding part for fixing. 6. A multi-window display method using the terminal device according to claim 1, wherein the monitor screen of the display unit includes a window of the image of the other party's interlocutor, (2) A multi-window display method for simultaneously displaying the windows of images captured by the image capturing unit. 7. A part of an image captured by the second image capturing section is cut out, and the clipped image is enlarged or reduced so as to match a window size set on the monitor screen of the display section. The multi-window display method according to claim 6. 8. A computer program for executing the method according to claim 6 or 7 by a computer. 9. A recording medium on which a computer program for executing the method according to claim 6 is recorded by a computer.