JPH09107534A - Video conference equipment and video conference system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video conference equipment and a video conference system by which video conference is held with excellent naturality as if a participant were actually making conversation with opposite conference party face to face. SOLUTION: A position detection section 6 detects a position of an opposite image display area 10 on a display section 3 and sends the position information to a camera control section 2. An image recognition section 7 recognizes an image of an opposite party displayed on the opposite image display area 10 and sends position information representing a face or eye position of the person to the camera control section 2. A user position detection section detects a position of an utterance party from the position of a microphone picking up a voice of the utterance party and the position information is sent to the camera control section 2. The camera control section 2 is configured to move the position of a camera 1 in three axis directions XYZ and the camera 1 is moved so that the position of the camera 1 comes to a position reaching the line of sight of the user depending on the position information obtained from the position detection section 6, the image recognition section 7 and a user position detection section 9 or the like.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video conference device used for a video conference and a video conference system for performing a video conference by performing multipoint communication.

[0002]

2. Description of the Related Art As shown in FIG. 7, a conventional video conference system includes a system controller 704 having a video / audio codec built in a personal computer (personal computer) or a workstation 703, a camera 701, a monitor 702, and a microphone. 705, video board 706, LAN
It is configured by connecting a video conference apparatus configured by adding peripheral devices such as a board 707 via a communication network such as a LAN. And this video conferencing system
A feature of the present invention is that not only video / audio communication but also a multi-media communication system such as multi-window, multi-task, data storage, e-mail, and the like, which is combined with the functions of a personal computer or a workstation 703, can be configured.

By the way, in this video conference system,
There are many technical factors occupied by human factors, and human interface has been designed in consideration of psychological factors of users. Further, since the video conference system is usually used for exchanging opinions by a plurality of participants, a sense of presence and a sense of stimulation are important factors.

In order to feel the presence and the stimulus, it is necessary to make eye contact with the other party in the conference. Therefore, in the conventional video conference apparatus, there is a deviation between the screen of the other party and the line of sight of the user. In consideration, the camera 70 for photographing the user
1 is often arranged on the upper part of the monitor 702.
This is because the tolerance in the vertical (vertical) direction is larger than the tolerance in the horizontal direction for the above-mentioned line-of-sight deviation.

[0005]

However, even in the case of the above configuration, especially in the case of a video conference apparatus using a large display (monitor), the tolerance of the line-of-sight deviation is small, so that the user is Except when the user consciously throws his / her eyes on the camera 701, the face image of the partner on the monitor 702 is often deviated. Therefore, even if a video conference is conducted using the video conference device having such a configuration, it lacks the naturalness that people normally have a conversation,
I can't get the feeling of being realistic and stimulating as if the other person is in front of me.

The present invention has been made in order to solve the above problems, and a video conference apparatus and a TV capable of performing a video conference with excellent naturalness such as actually facing a conversation. The purpose is to provide a conference system.

[0007]

According to a first aspect of the present invention, in a video conference apparatus used for a video conference, a display means for displaying an image, a photographing means for photographing a user, and an image on the other side of the video conference are displayed. Display control means for displaying in the partner image display area of the display means, position detecting means for detecting the position of the partner image display area in the display means, and position of the partner image display area detected by the position detecting means And a photographing control means for changing the position of the photographing means according to the above.

According to a second aspect of the present invention, in a video conference apparatus used for a video conference, display means for displaying an image, photographing means for photographing a user, and an image of the other party of the video conference are displayed on the display means. Display control means, image recognition means for performing image recognition of the image of the other party, and changing the position of the photographing means according to the display position of a predetermined portion of the image of the other party recognized by the image recognition means. It has a photographing control means.

According to a third aspect of the present invention, in a video conference apparatus used for a video conference, a display means for displaying an image, a photographing means for photographing the user, and a user position for detecting the position of the user who made the statement. It is characterized in that it has a detection means and an image pickup control means for changing the position or angle of the image pickup means according to the position of the user who made the speech detected by the user position detection means.

According to a fourth aspect of the present invention, in a video conference system for performing a video conference by performing multipoint communication, a display means for displaying an image, a photographing means for photographing an image of one's own side, and a partner side of the video conference. Display means for displaying the image of the other image on the partner image display area of the display means, position detecting means for detecting the position of the partner image display area on the display means, and the partner image display detected by the position detecting means. And a photographing control means for changing the position of the photographing means according to the position of the area.

[0011]

According to the video conference apparatus of claim 1 and the video conference system of claim 4, the photographing control means moves the photographing means to the same position as the position of the partner image display area on the display means. , When the user looks at the partner image display area to talk with the partner,
Since the photographing means is naturally seen, it is possible to hold a teleconference excellent in naturalness as if the user were talking with his / her eyes.

According to the video conference apparatus of the second aspect, the photographing control means moves the photographing means to the position of, for example, the face or the eyes of the image of the other party recognized by the image recognition means, so that a particularly large display is provided. Even when using, the user naturally looks at the shooting means when looking at the image of the other party in order to talk to the other party. It is possible to hold a teleconference that is as natural as if it were an illusion.

According to the video conference apparatus of the third aspect, the position of the speaker is detected by the user position detecting means, and the position or angle of the photographing means can be changed by the photographing control means according to the position. Even if there are multiple users on the device side, it is possible to direct the image capturing means toward the speaker or move the image capturing means in front of the speaker, and thus whether the person is conversing face to face. It is possible to hold such a teleconference excellent in naturalness.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 8 is a block diagram showing the elemental features of the present invention. In the figure, the communication unit 5 transmits / receives images, voice, data, etc. to / from a video conference apparatus on the other side via a communication network. Then, the image received by the communication unit 5 is displayed on the display unit 3 by the display control unit 4.

On the other hand, the image of the user (own side) photographed by the camera 1 constituting the photographing means is transmitted to the video conference apparatus of the other party via the communication section 5 and the communication network.

Various information is displayed on the display unit 3, and an image of the conference partner is displayed on a partner image display area 10 provided in a part of the display unit 3. The partner image display area 10 can be moved to an arbitrary position on the display unit 3, and the size of the display area can be changed.

The position detection unit 6 detects the position of the partner image display area 10 on the display unit 3 and sends the position information to the camera control unit 2.

The image recognition unit 7 recognizes the image of the person of the partner displayed in the partner image display area 10 and sends the position information indicating the position of the face or eyes of the person to the camera control unit 2.

The user position detecting unit 9 detects the position of the speaker from the position of the microphone 8 to which the voice of the speaker is input, and sends the position information to the camera control unit 2.

The camera control unit 2 determines the position of the camera 1 to be XY.
The position detecting unit 6, the image recognizing unit 7, and the user position detecting unit 9 are configured so as to be movable in the three Z directions.
The camera 1 is moved so that the position of the camera 1 comes to a position ahead of the user's eyes according to the position information obtained from the above.

The camera controller 2 includes a position detector 6,
The angle of the camera 1 can be freely changed according to the position information obtained from the image recognition unit 7, the user position detection unit 9, and the like.

The display control unit 4, the position detection unit 6, the image recognition unit 7, and the user position detection unit 9 described above are realized by system control devices 109 and 409 described later.

[First Embodiment] FIG. 1 is a block diagram showing the arrangement of a video conference apparatus according to the first embodiment of the present invention. In the figure, reference numeral 109 is a system control device for controlling the entire video conference device. The audio detected by a microphone (not shown) and the video captured by the camera 104 are processed by the video / audio codec inside the system control device 109 and input to the computer 108 such as a personal computer or a workstation via a video board or the like. . The computer 108 and the system control device 109 are connected to the LAN
Alternatively, a video conference system is constructed by connecting to another video conference device via a communication means such as ISDN, and it becomes possible to communicate video, audio, information, etc. at multiple points.

Reference numeral 101 denotes a digitizer composed of a transparent member, which is constructed so that coordinates can be input by a pen-shaped pointing tool, and is controlled by a digitizer control device 107.

Reference numeral 102 is a transmissive liquid crystal display device, and various data are displayed on the liquid crystal display device 102 as a monitor of the terminal device.

Reference numeral 103 denotes a half mirror, which prevents the camera 104 from being reflected on the user and, at the same time, LA
An image processed by the video codec provided in the system control device 109 connected by communication means such as N is projected by the projection device 106. This projected image is mainly a portrait image of the other party of the video conference, and the user can recognize this image as a video through the liquid crystal display device 102 and the digitizer 101.

The image of various information displayed on the liquid crystal display device 102 under the control of the computer 108 and the projection device 1
The display position and the like of the image projected at 06 are controlled by the system control device 109 so that they are not recognized by the user as they are overlapped.

The camera control unit 105 controls the photographing position and photographing conditions of the camera 104, and the image area projected by the projection apparatus 106 onto the half mirror 103, that is, the portrait image display window area (the partner image). The movement control is performed according to the display position of the display area). The movement of the camera 104 is performed by a three-dimensional positioning movement device commonly referred to as an XYZ stage or the like, and its position control is performed on the coordinate value at which the above-mentioned portrait image display window is positioned.

FIG. 2 shows a camera 104 and a projection device 106.
3 shows how the portrait image display window area projected by is operated in conjunction with each other. The size of the portrait image display window area is fixed here.

When the user operates the digitizer 101,
When the portrait image display window area in the screen 201 composed of the liquid crystal display device 102 and the half mirror 103 is moved from the position 202 to the position 203, the camera 104 is controlled by the camera control device 105. 2 in conjunction with the movement of the display window area
Moved from position 05 to 206.

Therefore, when the user looks at the portrait image display window area, he is looking at the camera 104.

FIG. 3 shows how the position of the camera 104 is controlled with respect to the portrait image display window area which can be enlarged or reduced.

The portrait image display window area is constructed so that it can be arbitrarily enlarged or reduced by the user's operation with a pen or mouse, but the display position of the portrait image display window area is moved. At the same time, when the size of the area is enlarged or reduced, the camera controller 105 controls the movement of the position of the camera 104 as follows.

First, if the portrait image display window area is about the size of the camera 104, the camera 104 is moved to the center position of the portrait image display window area.

If the portrait image display window area is larger than the camera 104, the image recognition unit provided in the system control device 109 performs image recognition of the image of the person in the portrait image display window area. The camera 104 is moved to the position of the face or eyes. That is, as shown in FIG.
When the portrait image display window area in the image is moved from the position 302 to the position 303 and the portrait image display window area is enlarged, the camera 10
4 is 30 from the position of 305 by the camera control device 105.
Moved to position 6. The position of this 306 is the position of the face or eyes of the person of the other party identified by the image recognition as described above. Therefore, a large display (screen)
Even when the eye movement of the user becomes large as in the case of using, the camera 104 looks like the camera 104 when looking at the portrait image of the partner. With this configuration, the image recognition only needs to be performed on the image projected by the projection device 106, and therefore the processing unit for the image displayed on the liquid crystal display device 102 is not burdened. Then, the user naturally sees the camera 104 when looking at the image of the other party, so that it is possible to provide a video conference system having an excellent naturalness such as actually facing each other and having a conversation.

[Second Embodiment] In the first embodiment described above, the display section (portrait image display window area) for displaying the image of the other party and the display section (liquid crystal display device 102) for displaying data and the like are provided. It was configured separately. In the present embodiment, an example in which an image and data of the other party are displayed by the same display processing will be described. The same parts as those in the embodiment of FIG. 1 are designated by the same reference numerals and the description thereof will be omitted.

In FIG. 4, reference numeral 402 denotes a liquid crystal control device 40.
In a transmissive liquid crystal display device whose display is controlled by 6,
Various data collectively processed by the system control device 409 and the image of the other party are displayed. Reference numeral 403 is a member for preventing the reflection of the camera 104 on the user.

The position of the camera 104 is the liquid crystal display device 40.
2 according to the contents of the image displayed on the camera control device 10
Controlled by 5. That is, the system control device 409 recognizes the window of the image of the other party (partner image display area) from the image displayed on the liquid crystal display device 402, and further recognizes the face or eyes of the person of the other party from the image in the window area. Image recognition. Then, the position information of the recognized face or eyes is input to the camera control device 105, and the camera control device 105 uses the position information to detect the position of the camera 104.
The position of the camera 104 is moved to the position of the face or eyes of the conference partner displayed in the window of the image of the other party.

As described above, in this embodiment, the image recognition of the entire display screen (the liquid crystal display device 402) is executed step by step, and the movement control of the camera 104 is performed based on the execution result, so that the user When the user sees the image of the other party on the screen of the display device 402, he or she naturally looks at the camera 104, so that it is possible to provide a video conference system with excellent naturalness such as actually facing each other and having a conversation. .

[Third Embodiment] Next, an example of a case where a plurality of users and a plurality of users have a video conference will be described with reference to FIGS. 5 and 6.

In this case, as shown in FIG. 6, a plurality of camera control devices 105 and a plurality of cameras 104 are arranged corresponding to each portrait image display window area in which the images of the opponents at a plurality of points are displayed. , Microphones 1, 2, 3 ... for the number of users on the device side are arranged corresponding to each user. In FIG. 6, “display side” means the screen 501 side of FIG. 5, and “user side” means the place where the user shown at 510 in FIG. 5 is sitting. ing.

In this structure, when a user speaks, a voice is detected by the microphone associated with the user who speaks, and the detection information is sent to the system control device 109 (409). System control device 109
(409) specifies the position of the speaker based on the position information of each microphone registered in advance, and
It is sent to the camera control device 105 together with the position information of the face or eyes of the person in the portrait image display window area recognized by the image recognition.

Using the position information, the camera control unit 105 moves each camera 104 to the position of the face or eyes of the person in the portrait image display window area, changes the angle of the camera 104, and changes the angle of the camera 104. Is controlled so as to face the speaker. At this time, the position of the camera 104 may be fixed and only the angle of the camera 104 may be changed using only the position information of the speaker.

FIG. 5 is a diagram for specifically explaining the configuration of FIG. As shown in FIG. 5, on the screen 501, windows 502 and 503 for displaying data such as graphs are displayed, and points A and B, which are video conference partners, are displayed.
The portrait images of the conference participants at the points C and C are displayed in the portrait image display window areas 504 and 505.
It is displayed at 506. And this window area 5
Cameras 104 are provided on the back sides of 04, 505, and 506, respectively.
Is arranged.

On the other hand, there are three users A to C on the device side, and these three are sitting in the arrangement 510 shown in FIG. 5, and in front of the users A to C. Mike A
It is assumed that C are arranged.

Now, assuming that the user C speaks, the microphone C detects the voice generated by the user C. Based on this detection, the system control device 109 (409) identifies the position of the speaker C based on the position information of each microphone registered in advance, and displays the position information by image recognition in a portrait image display. It is sent to the camera control device 105 together with the position information of the face or eyes of the person in the window area.
The camera control device 105 uses the position information,
While moving each camera 104 to the position of a person's face or eyes in the portrait image display window area,
04 to change the angle of each camera 104, arrow 507,
Control is performed so as to face the speaker C as indicated by 508 and 509.

As described above, when there are a plurality of users on the device side, the camera 104 is aimed at the user who has made a statement, so that the television is excellent in nature and has a natural conversation. A conference system can be provided.

In the example of FIG. 5, the destinations are three points A, B and C, but this may be any number of points, and the camera 104 corresponds to each portrait image display window area. Instead of providing each of them, only one camera 104 is used to move the camera 104 to the position in front of the user who made a statement, or to orient it toward the user who made a statement. It may be configured.

In the above-described embodiment, an example in which the camera 104 or the like is combined with a personal computer or a workstation to form the video conference apparatus of the present invention has been described, but a dedicated machine using a CPU or the like instead of the personal computer or the workstation. You may comprise the video conference apparatus of this invention.

In the above-described first and second embodiments, after the camera 104 is moved, the camera control unit 105 automatically changes the angle of the camera 104 so that the user's eyes directly face the lens of the camera 104. It may be configured as follows.
By doing so, there is no shift in the line of sight between the user and the other party, and a more natural dialogue can be performed.

[0051]

As described above, according to the video conference apparatus and the video conference system of the present invention, it is possible to conduct a video conference having an excellent naturalness such as actually having a face-to-face conversation with the other party of the conference. be able to.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a video conference apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an example of camera movement control according to the first embodiment of the present invention.

FIG. 3 is a diagram showing an example of movement control of a camera according to the first embodiment of the present invention.

FIG. 4 is a block diagram showing a configuration of a video conference device according to a second embodiment of the present invention.

FIG. 5 is a diagram for explaining a control example of the camera according to the third embodiment of the present invention.

FIG. 6 is a block diagram showing a configuration of a video conference device according to a third embodiment of the present invention.

FIG. 7 is a diagram showing a configuration of a conventional video conference system.

FIG. 8 is a block diagram showing elemental features of the present invention.

[Explanation of symbols]

 1 camera (imaging means) 2 camera control section (imaging control means) 3 display section 4 display control section 5 communication section 6 position detection section 7 image recognition section 8 microphone 9 user position detection section 10 partner image display area 101 digitizer 102 transparent Liquid crystal display device 103 Half mirror 104 Camera 105 Camera control device 106 Projection device 107 Digitizer control device 108 Computer 109 System control device 402 Transmissive liquid crystal display device 406 Liquid crystal control device 409 System control device

 ─────────────────────────────────────────────────── ─── Continued Front Page (72) Inventor Katsuyuki Kobayashi 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc. (72) Yuichiro Yoshimura 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Incorporated (72) Inventor Masaki Tokioka 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc.

Claims (4)

[Claims] 1. In a video conference apparatus used for a video conference, a display means for displaying an image, a photographing means for photographing a user, and an image of the other party of the video conference are displayed in a partner image display area of the display means. Display control means, position detecting means for detecting the position of the partner image display area on the display means, and changing the position of the photographing means according to the position of the partner image display area detected by the position detecting means A video conferencing apparatus comprising: a photographing control means. 2. A video conference apparatus used for a video conference, a display unit for displaying an image, a photographing unit for photographing a user, and a display control unit for displaying the image of the other party of the video conference on the display unit. An image recognition means for recognizing the image of the other party, and a photographing control means for changing the position of the photographing means according to the display position of a predetermined portion of the image of the other party recognized by the image recognition means. A video conferencing device having. 3. A video conference apparatus used for a video conference, a display means for displaying an image, a photographing means for photographing a user, a user position detecting means for detecting a position of a user who made a statement, and A video conferencing apparatus comprising: an image pickup control unit that changes the position or angle of the image pickup unit according to the position of the user who has made a speech detected by the user position detection unit. 4. A video conferencing system for performing a video conference by performing multipoint communication, a display means for displaying an image, a photographing means for capturing an image of one's own side, and an image for the other party of the video conference as the above display. Display control means for displaying in the partner image display area of the means, position detecting means for detecting the position of the partner image display area in the display means, and position of the partner image display area detected by the position detecting means. And a photographing control means for changing the position of the photographing means.