JP2003230049A - Camera control method, camera controller and video conference system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera control method for employing a smaller number of voice input sections and capable of eliminating the effect of surrounding reverberation sound and accurately displaying a face image of an utterer on a screen. <P>SOLUTION: A camera control/position discrimination section 11 capable of controlling panning/tilting/zooming in a direction of a sound source detected by a sound source direction detection section 15 on the basis of a voice frequency and/or a delay time from a plurality of voice input sections fewer than the number of participants changes the direction of a camera 10, a face image extract/position discrimination section 8 extracts the face image of a person on the basis of the received image, the camera control/position discrimination section 11 is controlled to display the face image extracted at a position and/or with a size on a prescribed screen, when two or more face images are extracted, the face image of a person resident at the closest position in the direction of sound source is selected and displayed. When no face image is detected or the face image cannot be displayed within the screen, zooming is automatically controlled to a wide angle, and the face image of a displayed person is displayed at a prescribed position and/or with a prescribed size. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera control method, a camera control device, and a video conference system, and more particularly, it detects the direction of a sound source by a plurality of audio input means, controls the direction of the camera, and displays it on the camera. The present invention relates to a camera control method, a camera control device, and a video conference system that have means for extracting a face image from an image and means for recognizing the position and size of the face image on the screen. More specifically, for example, in the case of a video conference system or the like, it is possible to automatically and accurately control the direction of the camera and the zoom for the speaker, and to smoothly advance the video conference.

[0002]

2. Description of the Related Art The following conventional examples relate to a camera control method and a camera control apparatus for accurately displaying a person image of a speaker in a video conference. Japanese Unexamined Patent Publication No. 5-268599 “Automatic control system for human image pickup camera in video conference system” describes a technique for automatically controlling the position and size of a person on a screen by the contour and pattern matching of the person. ing. That is, the outline of a person is extracted from the output image of the person image pickup camera, and an image including the extracted person and sample data of a person image in which a person image is stored in advance and stored in the center position of the screen By comparing and collating with, the pan and
Tilt / zoom control is performed so that the person to be imaged can be automatically displayed at a position and size near the center of the display screen.

According to Japanese Unexamined Patent Publication No. 9-307870, "Camera Automatic Direction Control Device in Video Conference System", the direction of a speaker is detected from a plurality of voice input means,
While controlling the direction of the camera to the position of the voice input means,
A technique for extracting the face contour of the speaker and adjusting the orientation of the camera is described. That is, when the participant in the video conference wears the earphone microphone with a lamp that lights up when the participant is speaking, the lamp detection circuit detects the lighting of the lamp when the participant speaks. Based on the detected lighting lamp position data,
The camera direction control circuit controls the direction of the camera so as to face the lighting lamp, and the face contour extraction circuit further includes:
By referring to standard face contour data registered in advance, the position of the face of the speaker is detected, and the direction of the camera is accurately directed to the position of the face of the speaker.

According to Japanese Patent Laid-Open No. 2000-512108 "Sound Source Positioning Method and Device", the voice signals obtained by the two voice input means are subjected to frequency analysis by time division, and the voices obtained by the respective voice input means. By calculating the delay time of the signal, the sound source direction is detected. Further, by providing only four voice input means, not only the vertical direction of the voice but also the distance is detected, and the direction and zoom of the camera are controlled. A technique for enabling the above, automatically pointing the camera in the direction of the speaker, and zooming up to an image of an appropriate size is described.

That is, even if the speaker's position is moving, the speaker's voice signal is received by two microphones (voice input means) arranged at positions distant from each other, and By calculating the signal delay time of the reception time between both voice signals received by the microphone using Fourier analysis, it is possible to detect the direction of the voice signal of the speaker, and further, X , Three or four microphones (voice input means) are arranged at different positions in the Y-coordinate direction, and the reception time relationship between the voice signals received by the respective microphones is calculated.
It is possible to calculate the position (that is, the direction and distance) of the speaker in the coordinate directions of the three axes of Z, and to point the camera in the direction of the speaker and at the same time perform zoom-up control. There is something.

[0006]

However, in the technique described above, for example, in the case of the technique disclosed in Japanese Patent Laid-Open No. 5-268599, an image pattern relating to the person of the participant in the video conference is registered in advance. If the face images of two or more persons have been extracted on the screen, it is necessary to accurately display the intended speaker who should be displayed. Or, there is a problem that it is difficult to display the speaker located outside the screen.

Further, in the technique disclosed in Japanese Unexamined Patent Publication No. 9-307870, in order to detect an accurate sound source direction, there are as many earphone microphones with a lamp, that is, voice input means, as many as the number of participants or speakers of the video conference. There is a problem that it is necessary to prepare and a large number of voice input means are required. Further, in the technology of Japanese Patent Publication No. 2000-512108, the direction of the sound source of the speaker and the distance to the sound source can be detected by a small number of two to four voice input means (microphones). Is a room with a lot of echo or a room with a lot of noise,
There is a problem that it is difficult to accurately detect the direction of the sound source because the arrival direction of the audio signal changes in various ways.

The present invention has been made in view of the above problems, and the frequency and signal delay time of input voices respectively input from a plurality of voice input means of which the number is smaller than the number of participants. A sound source direction detecting means for detecting a sound source direction by analyzing, and controlling the movement of the camera in the horizontal direction and / or the vertical direction to the sound source direction detected by the sound source direction detecting means. Camera control / position that allows zoom change control and also controls the camera so that it is displayed at the preset position on the screen with the preset facial image size. Using the determination means and the face image extraction / position determination means for extracting the face image of the person from the image captured by the camera and recognizing the position / size of the face image on the screen, the sound source When the face image of the person is detected by the face image extracting / position determining unit in the sound source direction of the voice detected by the direction detecting unit, it is recognized as the face image of the speaker for the first time, and the camera control / position determining unit is used. This makes it possible to display the face image of the speaker at a predetermined position and / or size on the screen, and it is not possible to accurately display the speaker due to the influence of surrounding echoes. It is an object of the present invention to solve the problems of the prior art, such as having to equip the voice input section.

Further, when the face image extracting / position determining means detects face images of a plurality of persons, the person closest to the sound source direction detected by the sound source direction detecting means is selected. It is possible to display the face image of the speaker as a speaker at a predetermined position / size, and conversely,
If no face image is detected, or if the face image is too large to fit on the screen, control the camera to change the zoom to the wide-angle side so that the person in the vicinity of the sound source direction captured in the wide-angle image is controlled. The face image is detected, the person is regarded as the speaker, and the face image of the person regarded as the speaker can be displayed at a predetermined position / size.

[0010]

The present invention provides technical means for solving the above-mentioned problems, and each invention constitutes the following technical means. A first aspect of the present invention has a camera control unit that enables control of changing the direction of the camera to a horizontal direction and / or a vertical direction so that a captured image can be displayed at a predetermined position on the screen. In the control method, further, recognizing the face image extraction means for extracting a face image of a person from the image input from the camera, and the position of the face image extracted by the face image extraction means and displayed on the screen. And a sound source direction detecting means for detecting a sound source direction from each sound signal input to a plurality of sound input means, and the sound source direction detected by the sound source direction detecting means. The camera direction control means automatically controls the angle change of the camera in the horizontal direction and / or the vertical direction, and is extracted by the face image extraction means, and
The position of the face image of the person on the screen recognized by the face image position determination means is displayed at the preset position on the screen, and detected by the sound source direction detection means. A camera control method for displaying a face image of a person in a sound source direction at a preset position on the screen.

In a second aspect of the present invention, in the camera control method according to the first aspect, the camera control means enables control of zooming the camera to a wide angle and / or a telephoto, and the face image. As the position determination means, by making it possible to recognize the size of the face image on the screen, the camera control means automatically controls the zoom change of the camera to wide-angle and / or telephoto, The size of the face image of the person on the screen, which is extracted by the face image extraction means and recognized by the face image position determination means, is displayed in a preset size on the screen. In this way, a camera control method for displaying a face image of a person in the sound source direction detected by the sound source direction detecting means in a size set on a screen set in advance is provided. It is intended.

A third aspect of the invention is the camera control method according to the first or second aspect of the invention, in which the face image extracting means detects two or more persons in the sound source direction detected by the sound source direction detecting means. When the face image is extracted, the camera control means is used to display the face image of the person closest to the sound source direction detected by the sound source direction detecting means on the screen set in advance. The camera control method is so controlled as to display the position and / or size of the camera.

According to a fourth aspect of the present invention, in the camera control method according to any one of the first to third aspects, a face of a person is detected by the face image extracting means in the sound source direction detected by the sound source direction detecting means. When no image is detected and / or when the face image of the person extracted by the face image extracting means does not fit on the screen, the camera control means is used to operate the camera in wide-angle operation. The automatic zoom control is performed to display the face image of the person by the face image extracting means and the face image position determining means displays on the screen based on the captured image by the wide-angle operation. The camera control method is characterized in that a position and / or size of a face image is recognized.

A fifth invention is the camera control method according to any one of the first to fourth inventions, wherein the camera control means changes the angle of the camera in the sound source direction detected by the sound source direction detection means. And / or a time measuring means for measuring an elapsed time after the zoom change control is performed, and the elapsed time measured by the time measuring means is a preset unit. A camera control method in which the camera angle change and / or zoom change control by the camera control unit in the sound source direction detected by the sound source direction detection unit is performed each time it is detected that time has elapsed. It is characterized by doing.

According to a sixth aspect of the present invention, in the camera control method according to any one of the first to fifth aspects, the face image extracting means extracts the face image of the person by identifying the skin color of the person. And a camera control method for controlling the camera.

A seventh invention is the camera control method according to any one of the first to sixth inventions, wherein the face image extraction means extracts a face image of the person by identifying the outline of the person. And a camera control method for controlling the camera.

An eighth invention is the camera control method according to any one of the first to seventh inventions, wherein the sound source direction detecting means inputs respective audio signals to a plurality of the audio input means. The method is characterized by providing a camera control method for detecting the sound source direction by analyzing the frequency and / or the delay time.

A ninth invention is a camera control device capable of controlling an angle change in a horizontal direction and / or a vertical direction and / or a zoom change in a wide angle and / or a telephoto.
Further, the present invention is characterized in that the camera control device is provided with means for realizing the camera control method according to any one of the first to eighth inventions.

A tenth aspect of the present invention is a video conference system for performing a video conference, the video conference system comprising the camera control device according to the ninth aspect.

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION An example of an embodiment relating to a camera control method according to the present invention, a camera control apparatus for realizing the camera control method, and a video conference system including the camera control apparatus will be described below. FIG. 1 is a functional block diagram showing an example of the configuration of an embodiment when a camera control method and a camera control device according to the present invention are applied to a video conference system. The video conference system device main body 20 includes the following circuit elements. It is configured. The video conference system device body 20 includes a CPU (central processing unit) 1 that controls the entire video conference system device body 20.
And a ROM (read-on memory) storing programs for control / calculation by the CPU 1 and the control /
A memory A 2 including a RAM (random access memory) as a working memory for assisting calculations and storing various data, and programmable rewriting for storing information necessary for operating the video conference system. And a memory B 3 capable of

Further, the main body 20 of the video conference system device.
Used by participants in a video conference to talk
A voice input unit 4 including a plurality of voice input units and a speaker 5 including a plurality of two or more voice input units 4 and a speaker 5.
And a sound source direction detecting unit 15 for detecting the sound source direction by analyzing the frequency of each input audio signal input from each of the plurality of audio input units 4 and the delay time of the input audio signal. A communication control unit 7 for transferring video and audio data via the communication line 17, a face image of a person, and a face image extraction for grasping the position and / or size of the face image on the screen. A position determining unit 8 and a demultiplexing unit 9 that demultiplexes and multiplexes video and audio data transmitted and received between the communication control unit 7, a camera 10 that captures an image required for a video conference, and a camera. Control the angle (direction) change with 10 horizontal and / or vertical directions, control the zoom change to wide angle and / or telephoto,
Alternatively, it is provided with a camera control / position determination unit 11 that grasps the current angle (direction) of the camera 10 and the zoom state as a camera position.

Here, as described above, the sound source direction detecting unit 15 is composed of a smaller number than the participants in the video conference.
By analyzing the frequency of the input voice and the delay time of the voice signal input from the plurality of voice input units 4 or more,
A sound source direction detecting means for detecting a sound source direction is provided. As described above, the face image extraction / position determination unit 8 includes a face image extraction unit that extracts a face image of a person from the image input from the camera 10, and the face image extraction unit extracts the face image and displays it on the screen. A face image position determining means capable of recognizing the position of the face image and / or the size of the face image on the screen is provided as a face image extracting means for extracting a face image of a person. It is possible to extract a face image of a person by identifying the skin color of the person or by identifying the contour of the person.

Further, the camera control / position determination unit 11 changes the angle of the camera 10 in the horizontal direction and / or the vertical direction (that is, changes the pan and / or the tilt) and / or the wide angle and / or the wide angle, as described above. Alternatively, the present camera 10 is provided with a camera control means for controlling the zoom change to the telephoto.
Position (that is, direction or zoom state) is determined, or a face image of a person located in the sound source direction detected by the sound source direction detecting means is set on the screen in advance. And / or a camera control unit including a display position determination unit that can automatically display the size (magnification).

Further, the main body 20 of the video conference system device
Is a display unit 12 for displaying video data and various character information.
And an image control unit 13 for controlling encoding of an image captured by the camera 10 and / or encoding of image data or character information displayed on the display unit 12, and a telephone number of the other party. Inputting and inputting for operation on the video conference system device main body 20, the operation unit 14 for determining the input, and the detecting operation of the sound source direction and the face at every unit time interval during the automatic search of the speaker. A clock unit 16 necessary for controlling the image extracting operation is provided. The respective circuit elements are connected to each other as shown in FIG. 1 to form the video conference system device body 20. Here, as the connection shown in FIG. 1, a solid line indicates an information signal line through which audio and / or image information flows, and a broken line indicates a control signal line through which various control information flows.

The specific information stored in the memory B3 is, for example, data relating to the skin color of a person or the outline of the person for detecting a face image, and the face image set in advance. Screen display position / range, screen size of face image set in advance, stop time interval of face image detection operation (that is, detection of sound source direction detection direction and detection of face image in the sound source direction) A set value of a unit time interval for executing display of the face image at a predetermined position / size, a callee number (address) at the time of making a call, and the like are stored. When the video conference system device main body 20 is in a communication state in which image communication is being performed, the sound source direction detection unit 15 detects the sound source direction based on the respective audio signals obtained by the plurality of audio input units 4. Is detected. The CPU 1 determines the position (direction and zoom state) of the camera 10 by the camera control / position determination unit 11, and controls the orientation of the camera 10 to the sound source direction detected by the sound source direction detection unit 15.

Here, based on the video data taken in from the camera 10, the face image extraction / position determination unit 8
The face image of the person is extracted, and the position of the extracted face image is detected. After that, the CPU 1 controls the camera again.
The position determination unit 11 determines the position (direction and zoom state) of the camera 10, and the face image extraction / position determination unit 8 extracts the image data on the screen based on the video data captured from the camera 10. The face image of the person whose display position is detected is displayed by the camera 10 so that the face image is displayed at a predetermined position and / or size (magnification) on the screen set in advance. The position is controlled.

Further, after the speaker is detected and the face image of the speaker is once displayed at a predetermined position and / or size (magnification) on the screen set in advance, In order to stabilize the display of the face image captured by the camera 10 and displayed on the screen, a sound source direction detection operation, a face image extraction operation, and a camera control operation to a predetermined position and / or size are performed. The constant execution is temporarily stopped, the timer is started by the clock unit 16, and the timer is controlled to be executed each time a preset unit time elapses.

The configuration of this embodiment shown in FIG. 1 will be described in more detail. As described above, the video control unit 13 and the audio control unit 6 share the encoding unit for encoding the video information and the audio information and the decoding unit for decoding the encoded data, respectively. . The communication control unit 7 is responsible for transmitting means for transmitting data to the communication line 17 and receiving means for receiving data from the communication line 17. The role of the image and audio information processing means for timely performing the processing required for image communication of encoded / decoded information is mainly that the CPU 1 controls the memory A 2, the memory B 3, the video control unit 13, This is performed by controlling the voice control unit 6 and the communication control unit 7.

The image compression / decompression process is performed by the communication control unit 7. Further, the clock unit 16 is responsible for the clock means for counting and measuring the time up to a preset unit time, that is, the time measuring means. A face image extraction / position determination unit for a person that extracts a face image of a person from an input image and grasps the position and / or size of the face image of the person displayed on the screen is a face image extraction / position determination. Part 8
Will be shared. The sound source direction detecting unit 15 is responsible for the sound source direction detecting means for detecting the sound source direction of the input voice. The operation unit 14 is responsible for input detection means for detecting a key input for operating the video conference system device body 20. The camera control / position determination unit 11 shares the direction changing means for tilting / panning the camera 10 vertically and horizontally and the zoom changing means for moving the camera 10 in a wide-angle or telephoto state.

Next, regarding the camera control method and the camera control device of the video conference system apparatus having the above-mentioned configuration, the position of the speaker is automatically searched, and the face image of the speaker is predetermined position and / or size. The operation to be displayed at will be described with reference to the flowcharts shown in FIGS. 2, 3, 4, 5, and 6. FIG. 2 is a flowchart for explaining one embodiment of the camera control method and the camera control device according to the present invention in the video conference system shown in FIG. The present embodiment will be described below with reference to the flowchart shown in FIG. In the main body 20 of the video conference system, when the idle state (state S1) is changed to the communicating state (state S2) indicating the image communicating state, first, the voice input unit including a plurality of two or more voice input parts is provided. It is confirmed whether or not there is an input voice signal being input to No. 4 (step a1), and if no input voice signal is detected (N in step a1).
O), at step a1, the process waits until the next voice is input.

On the other hand, when the input voice signal is detected (YES in step a1), the sound source direction detecting section 15 detects the sound source direction of the input voice signal (step a2). Further, the camera control / position determination unit 11 detects the current position (direction and zoom state, that is, horizontal, vertical, telephoto, and wide-angle position) of the camera 10 (step a3). The detected sound source direction and the detected current position of the camera 10 are temporarily stored in the memory B3 (step a4). Memory B3 detected by CPU1
To confirm the sound source direction and the current position of the camera 10 (direction and zoom state, that is, horizontal, vertical, telephoto, and wide-angle position) stored in, and to direct the camera 10 to the sound source direction. The movement amount of the camera 10 is calculated, the calculated movement amount is transmitted to the camera control / position determination unit 11, and the camera 10 is controlled by the camera control / position determination unit 11.
Is operated in the direction of the detected sound source (step a5).

After the movement of the camera 10 in the direction of the sound source, the face image extraction / position determination unit 8 determines the face of the person included in the image signal based on the imaged image signal from the camera 10. While extracting the image, the position of the face image on the screen and the size (magnification) of the face image on the screen are determined (step a6), and the determined position of the face image on the screen and the screen are determined. 10 showing the size (magnification) of the face image of the
The position (direction and zoom state) of is stored in the memory B 3 (step a7).

After that, the position of the face image on the screen is set in accordance with the preset values stored in the memory B3 in advance.
By determining the size (magnification), the camera control / position determination unit 11 operates the direction and zoom of the camera 10 again based on the determined position and size (magnification) of the face image. The face image of the person who is the sound source, that is, the speaker can be automatically displayed at the preset position and size (magnification) on the screen (step a8).

Here, in step a6, the face image extraction / position determination unit 8 determines the face image of the person indicating the speaker and the position / size of the face image on the screen. As disclosed in Japanese Patent Application Laid-Open No. 2000-354247, "Image processing apparatus", Japanese Patent Application Laid-Open No. 5-268599, "Automatic control system for human image pickup camera in video conference system", etc., as described above, The face image is extracted from the contour of the person. By performing such an operation, the face image of the speaker can be accurately displayed at a predetermined position set on the screen in a predetermined size set in advance. Is possible.

Next, another embodiment of the camera control method and the camera control apparatus according to the present invention shown in FIG. 3 will be described. FIG. 3 is a flowchart for explaining another embodiment relating to the camera control method and the camera control device according to the present invention in the video conference system shown in FIG.

Here, in FIG. 3, the camera control / position determination unit 11 is operated to move the camera 10 in the sound source direction detected by the sound source direction detection unit 15 to extract a face image. When the position determination unit 8 detects a face image of a person on the screen, if some error occurs in the sound source detection direction due to surrounding echoes, and two or more face images are extracted, The position of the face image of memory B
3 is once stored in the memory B 3, and the person closest to the sound source direction detected by the sound source direction detecting unit 15 is determined to be the speaker, and according to the set value stored in advance in the memory B 3. , Position of face image on screen, size (magnification)
Is determined, and the camera control / position determination unit 11 is operated again based on the determined position and size (magnification) of the face image to move the camera 10 to move the source of the sound source. It is possible to automatically display the speaker determined to be on the screen according to the position and the size (magnification) set in advance.

The present embodiment will be described below with reference to the flow chart shown in FIG. In the main body 20 of the video conference system, when the idle state (state S1) is changed to the communicating state (state S2) indicating the image communicating state, first, the voice input unit including a plurality of two or more voice input parts is provided. It is confirmed whether or not there is an input voice signal input to No. 4 (step b1). If no input voice signal is detected (NO in step b1), the next voice is input in step b1. I will be waiting until there is.

On the other hand, when the input sound signal is detected (YES in step b1), the sound source direction detecting section 15 detects the sound source direction of the input sound signal (step b2). Further, the camera control / position determination unit 11 detects the current position (direction and zoom state, that is, horizontal, vertical, telephoto, and wide-angle position) of the camera 10 (step b3). The detected sound source direction and the detected current position of the camera 10 are temporarily stored in the memory B3 (step b4). Memory B3 detected by CPU1
To confirm the sound source direction and the current position of the camera 10 (direction and zoom state, that is, horizontal, vertical, telephoto, and wide-angle position) stored in, and to direct the camera 10 to the sound source direction. The movement amount of the camera 10 is calculated, the calculated movement amount is transmitted to the camera control / position determination unit 11, and the camera 10 is controlled by the camera control / position determination unit 11.
Is operated in the direction of the detected sound source (step b5).

After the camera 10 moves in the direction of the sound source, the face image extraction / position determination unit 8 determines the face of the person included in the image signal based on the imaged image signal from the camera 10. While extracting the image, the position of the face image on the screen and the size (magnification) of the face image on the screen are determined (step b6). Here, it is determined whether or not the face images of two or more persons are detected as the extracted face images of the person (step b7). When the face images of two or more persons are detected (YE in step b7)
S), the position of the face image of each person detected by the face image extraction / position determination unit 8 is compared with the sound source direction of the sound source detected by the sound source direction detection unit 15, and the position is closest to the sound source direction. The person at the position of the existing face image is selected as the target person (that is, the speaker), and the process proceeds to step b9 (step b8). On the other hand, when the face images of two or more persons are not detected and only one face image is detected (NO in step b7), the person of the detected face images is the target person (that is, the speaker). So there is next step b
Move to 9. In step b9, the position (direction and zoom state) of the camera 10 for capturing the face image of the person selected as the speaker is stored in the memory B3 (step b9).

After that, the position of the face image on the screen is set in accordance with the set value stored in the memory B 3 in advance.
By determining the size (magnification), the camera control / position determination unit 11 operates the direction and zoom of the camera 10 again based on the determined position and size (magnification) of the face image. The face image of the person who is the sound source, that is, the speaker can be automatically displayed at the preset position and size (magnification) on the screen (step b10).

Next, still another embodiment of the camera control method and the camera control apparatus according to the present invention shown in FIG. 4 will be described. FIG. 4 is a flowchart for explaining still another embodiment of the camera control method and the camera control device according to the present invention in the video conference system shown in FIG.

Here, in FIG. 4, when the face image extraction / position determination unit 8 does not detect the face image of the person on the screen, the face image of the person is not detected, and /
Alternatively, when the extracted face image of the person does not fit within the screen, the CPU 1 operates the camera control / position determination unit 11 to change the zoom of the camera 10 and automatically perform the wide-angle operation. By doing so, a wider range of images is displayed on the screen, and it is determined by the face image extraction / position determination unit 8 whether or not a face image of a person is detected on the screen captured in such a wide range, and When the face image is extracted, the position and size (magnification) of the face image on the screen are set according to the setting values stored in the memory B 3 in advance.
Then, based on the determined position and size (magnification) of the face image, the camera control / position determination unit 11 operates the camera 10 again, so that an error may occur in the detection of the sound source direction. Even if there is a sound source, it is possible to find the speaker who is the source of the sound source and automatically display it based on the preset position and size (magnification) on the screen. Is.

The present embodiment will be described below with reference to the flowchart shown in FIG. In the main body 20 of the video conference system, when the idle state (state S1) is changed to the communicating state (state S2) indicating the image communicating state, first, the voice input unit including a plurality of two or more voice input parts is provided. It is confirmed whether or not there is an input voice signal input in step 4 (step c1). If no input voice signal is detected (NO in step c1), the next voice is input in step c1. I will be waiting until there is.

On the other hand, when the input audio signal is detected (YES in step c1), the sound source direction detecting section 15 detects the sound source direction of the input audio signal (step c2). Further, the camera control / position determination unit 11 detects the current position (direction and zoom state, that is, horizontal, vertical, telephoto, and wide-angle position) of the camera 10 (step c3). The detected sound source direction and the detected current position of the camera 10 are temporarily stored in the memory B3 (step c4). Memory B3 detected by CPU1
To confirm the sound source direction and the current position of the camera 10 (direction and zoom state, that is, horizontal, vertical, telephoto, and wide-angle position) stored in, and to direct the camera 10 to the sound source direction. The movement amount of the camera 10 is calculated, the calculated movement amount is transmitted to the camera control / position determination unit 11, and the camera 10 is controlled by the camera control / position determination unit 11.
Is operated in the direction of the detected sound source (step c5).

After the camera 10 moves in the direction of the sound source, the face image extraction / position determination unit 8 determines the face of the person included in the image signal based on the imaged image signal from the camera 10. While extracting the image, the position of the face image on the screen and the size (magnification) of the face image on the screen are determined (step c6). Here, the face image extraction / position determination unit 8 determines whether or not a face image of a person is detected (step c7). When the face image of the person is detected (YES in step c7), the person of the detected face image is the target person (that is, the speaker), and the face image of the person who is determined as the speaker. The position (direction and zoom state) of the camera 10 that is capturing the image is stored in the memory B 3 (step c10). On the other hand, in step c7, if the face image of the person is not detected and / or if the extracted face image of the person does not fit on the screen (NO in step c7), camera control / position The determination unit 11 is operated to control the camera 10 to perform a wide-angle operation for capturing an image in a wider range.

Based on the video signal from the camera 10 picked up by the wide-angle camera 10, the face image extraction / position determination unit 8 again extracts the face image of the person included in the video signal. At the same time, the position of the face image on the screen and the size (magnification) of the face image on the screen are determined (step c9). Here, when the face image of the person is detected (YES in step c9), it is considered that there is an error in the detection of the sound source direction, and as a result of the wide-angle operation, the face image extraction / position determination unit 8 determines the person. The person whose face image was detected is the person who is the source of the sound source (that is, the speaker)
If so, the process proceeds to step c10, and the position (direction and zoom state) of the camera 10 for capturing the face image of the person determined as the speaker is stored in the memory B3 (step c10).

After that, the position of the face image on the screen is set in accordance with the set values stored in advance in the memory B3.
By determining the size (magnification), the camera control / position determination unit 11 operates the direction and zoom of the camera 10 again based on the determined position and size (magnification) of the face image. The face image of the person who is the sound source, that is, the speaker can be automatically displayed at the preset position and size (magnification) on the screen (step c12).

On the other hand, in step c9, when the face image of the person is not detected even by the camera 10 operated in the wide angle (NO in step c9), the sound source detected by the sound source direction detecting unit 15 causes some noise. The sound source direction detection operation is temporarily stopped (step c11), the process returns to step c1, and the process waits until the next voice is input.

Next, still another embodiment of the camera control method and the camera control apparatus according to the present invention shown in FIG. 5 will be described. FIG. 5 is a flowchart for explaining still another embodiment relating to the camera control method and the camera control device according to the present invention in the video conference system shown in FIG.

Here, in FIG. 5, once in FIG.
Through the process shown in any one of FIG. 4 to FIG. 4, the face image of the target person (that is, the speaker) is detected, and the face image of the speaker is set in a predetermined screen on the screen. After the operation of automatically displaying the position and the predetermined size (magnification) is determined, the operation of constantly detecting the sound source direction by the sound source direction detecting unit 15 is temporarily stopped, and the memory B
3, the timer is counted by the clock unit 16 until the unit time interval pre-registered in 3 is elapsed, and the sound source direction detection unit 15 re-detects the sound source direction every time the unit time interval elapses. By doing so, the camera 10 responds to the slight movement of the person who is the speaker at all times.
It is possible to stably display the face image of the speaker by preventing the position of (1) from being moved.

This embodiment will be described below with reference to the flow chart shown in FIG. In the video conference system device main body 20, when the idle state (state S1) is changed to the communication state (state S2) indicating the image communication state, first, the timer counted by the clock unit 16 is It is determined whether the unit time interval pre-registered in B3 has been reached (step d1). When the unit time interval has not been reached (NO in step d1), the timer of the clock unit 16 continues to be counted as it is, while the unit time interval has been reached and the predetermined unit time has elapsed. If it is detected (YES in step d1), the clock unit 16
The timer being counted by is once cleared and is set to a state in which counting from the initial value can be performed again (step d2).

When the timer 16 detects that the timer has passed the time indicated by the unit time interval, it is first checked whether or not the input voice signal is detected (step d3). When the input voice signal is not detected (NO in step d3), the process proceeds to step d10 to start the counting operation of the timer by the clock unit 16 (step d10), and returns to step d1.
Wait until the next unit time interval elapses.

On the other hand, when the input voice signal is detected (YES in step d3), the sound source direction detecting section 15 detects the sound source direction of the input voice signal (step d4). Further, by the camera control / position determination unit 11,
Current position of camera 10 (direction and zoom status, ie,
Horizontal, vertical, telephoto and wide-angle positions) are detected (step d5). The detected sound source direction and the detected current position of the camera 10 are temporarily stored in the memory B3 (step d6). The CPU 1 confirms the sound source direction detected and stored in the memory B3 and the current position of the camera 10 (direction and zoom state, that is, horizontal, vertical, telephoto and wide-angle position), and the orientation of the camera 10 is confirmed. Is calculated for the direction of the sound source, and the calculated movement amount is transmitted to the camera control / position determination unit 11, and the camera 1 is controlled by the camera control / position determination unit 11.
The direction of 0 is operated toward the detected sound source direction (step d7).

Here, any one of the camera control operations in the respective flow charts shown in FIGS. 2 to 4 is selected and executed (step d8). That is, camera 1
After the moving operation of 0 in the sound source direction, the face image extraction / position determination unit 8 extracts the face image of the person included in the image signal based on the imaged image signal from the camera 10. At the same time, the position of the face image on the screen and the size (magnification) of the face image on the screen are determined. Here, it is determined whether only one face image of the extracted person is detected, two or more face images are detected, or one face image is not detected. It

If only one face image is detected, steps a6 and a7 in the flowchart of FIG.
As shown in, the position of the face image on the screen determined by the operation of the face image extraction / position determination unit 8 is determined as the face image of the target speaker as the extracted face image of the person. The position (direction and zoom state) of the camera 10 indicating the size (magnification) of the face image on the screen is stored in the memory B.
Stored in 3.

When two or more face images are detected, steps b6 to b in the flowchart of FIG.
As shown in FIG. 9, the position of the face image of each person detected by the face image extraction / position determination unit 8 is compared with the sound source direction of the sound source detected by the sound source direction detection unit 15, and the sound source direction most The person at the position of the face image located at a close position is selected as the target person (that is, the speaker), and the position (direction and zoom) of the camera 10 for capturing the face image of the person selected as the speaker. State) is memory B
Stored in 3.

On the other hand, when no face image is detected or when the face image does not fit on the screen, the camera control / position determination unit 11 determines by the steps c8 to c11 in the flowchart of FIG. When the face image extraction / position determination unit 8 detects a face image of a person as a result of operating the camera 10 in a wide angle and capturing a wide range of images, the face image of the person is captured. The position (direction and zoom state) of the camera 10 for saving is stored in the memory B 3. However, camera 1 with wide-angle operation
If the face image of the person is not detected even by 0, the sound source direction detection operation is temporarily stopped and step d
10, the timer 16 starts the counting operation of the timer (step d10), returns to step d1, and waits until the next unit time interval elapses.

When the face image of the speaker is detected, the camera 10
When the position of is stored in the memory B 3, the position and size (magnification) of the face image on the screen are determined according to the preset values stored in the memory B 3. Based on the determined position and size (magnification) of the face image, the camera control / position determination unit 11 again causes
By operating the direction and zoom of the camera 10,
The face image of the person who is the sound source, that is, the speaker is displayed on the screen according to the preset position and size (magnification) on the screen (step d9). Furthermore, step d
In step 2, the count of the timer set to the initial value is started (step d10), and the process returns to step d1.
By waiting until the next unit time interval elapses,
The input voice detection operation is stopped until the next unit time interval registered in advance in the memory B3 is reached. Therefore, even if the position of the speaker's face, that is, the position of the mouth is slightly deviated during the remark, and the sound source direction is deviated slightly, the position of the camera 10 is adjusted each time. Without being performed, the position of the camera 10 is kept at a stable position until the predetermined time indicated by the unit time interval elapses, and the face image of the speaker can be stably displayed.

[0059]

As described above, according to the camera control method, the camera control device and the video conference system according to the present invention, the following effects are brought about. That is, according to the present invention, the camera control means capable of performing the horizontal direction, the vertical direction, the wide angle, and the telephoto operation, and the face image of the person are extracted from the images input from the camera and displayed on the screen. , A face image extraction / position determination means for recognizing the position and size of the face image, and a sound source direction from the frequency and / or delay time of each audio signal input to a plurality of audio input means less than the number of participants The direction of the sound source detected by the sound source direction detecting means, that is, the direction of the speaker, can be automatically controlled, and a predetermined value on the screen can be set. At the position, the camera control means also provided with a camera position determination means capable of controlling the camera so as to be displayed in a predetermined size of the face image set in advance,
With the above configuration, the speaker can be accurately captured by the camera and displayed at a predetermined position on the screen and in a predetermined size.

Further, when the direction of the camera is controlled in the sound source detection direction detected by the sound source detection direction means, a slight error occurs in the sound source detection direction due to the influence of surrounding reverberation sound, etc. When the face image / position determination means detects the face images of a plurality of persons, the person closest to the sound source detection direction is determined as the speaker, and the determined speech is determined. The face image of the person can be displayed at a predetermined position on the screen and in a predetermined size.

Furthermore, when the face image of the person is not detected by the face image / position determination means even if the direction of the camera is controlled in the sound source detection direction detected by the sound source detection direction means, and / or If the image does not fit on the screen, the camera is operated in a wide angle to capture a wider range of images, and the face image of the person who is considered to be the speaker is detected from the wider range of images. Even if there is some error in the sound source detection direction, the face image of the person regarded as the speaker is displayed at a predetermined position on the screen.
Moreover, it can be displayed in a predetermined size.

Further, the sound source direction detecting operation is not always performed, but the sound source direction detecting operation is performed each time a predetermined unit time indicated by a unit time interval registered in advance elapses. By pointing the position of the camera toward the sound source and displaying the face image of the speaker at a predetermined position or size on the screen, even if the speaker's face position or mouth Even if the position of the sound source is slightly misaligned and the sound source direction is slightly misaligned, the camera position (that is, the direction and zoom state) is constantly prevented from being adjusted, and the face image of the speaker is stabilized. Then, it is possible to display it on the screen.

Further, as the sound source direction detecting means, the frequency and / or the delay time of each input voice signal input from a plurality of voice input means smaller than the number of participants are analyzed to generate a voice. The original sound source direction can be detected, and the number of voice input means can be reduced.

[Brief description of drawings]

FIG. 1 is a functional block diagram showing an example of a configuration of an embodiment when a camera control method and a camera control device according to the present invention are applied to a video conference system.

FIG. 2 is a flowchart for explaining an embodiment of a camera control method and a camera control device according to the present invention in the video conference system shown in FIG.

FIG. 3 is a flowchart for explaining another embodiment of the camera control method and the camera control device according to the present invention in the video conference system shown in FIG.

4 is a flowchart for explaining still another embodiment of the camera control method and the camera control device according to the present invention in the video conference system shown in FIG.

5 is a flowchart for explaining still another embodiment of the camera control method and the camera control device according to the present invention in the video conference system shown in FIG.

[Explanation of symbols]

1 ... CPU, 2 ... Memory A, 3 ... Memory B, 4 ... Voice input unit, 5 ... Speaker, 6 ... Voice control unit, 7 ... Communication control unit, 8 ... Face image extraction / position determination unit, 9 ... Separation and multiplexing Akabe,
10 ... Camera, 11 ... Camera control / position determination unit, 12 ...
Display unit, 13 ... Image control unit, 14 ... Operation unit, 15 ... Sound source direction detection unit, 16 ... Clock unit, 17 ... Communication line, 20 ... Video conference system device body.

Claims (10)

[Claims] 1. A camera control capable of displaying a picked-up image at a predetermined position on a screen by having a camera control means capable of controlling a direction of the camera to be changed in a horizontal direction and / or a vertical direction. In the method,
Face image extraction means for extracting a face image of a person from the image input from the camera, and face image position determination capable of recognizing the position of the face image extracted by the face image extraction means and displayed on the screen Means and a sound source direction detecting means for detecting a sound source direction from each of the audio signals input to the plurality of audio input means, and the camera direction control is performed in the sound source direction detected by the sound source direction detecting means. A person on the screen which is extracted by the face image extraction means and recognized by the face image position determination means by automatically controlling the angle change of the camera in the horizontal direction and / or the vertical direction by the means. The face image of the person in the sound source direction detected by the sound source direction detecting means is displayed in advance so that the position of the face image is displayed at the position on the screen set in advance. Camera control method characterized by displaying on the position on the screen is constant. 2. The camera control method according to claim 1, wherein the camera control means enables control of zooming the camera to a wide angle and / or a telephoto, and the face image position determination means uses a screen. By making it possible to recognize the size of the face image above, the camera control means automatically controls the zoom change of the camera to wide angle and / or telephoto, and the face image extraction means The size of the face image of the person on the screen that is extracted and recognized by the face image position determination means is displayed in the size on the screen set in advance, and the sound source A camera control method characterized in that a face image of a person in the sound source direction detected by the direction detecting means is displayed in a size on the screen set in advance. 3. The camera control method according to claim 1, wherein the face image extracting unit extracts face images of two or more persons in the sound source direction detected by the sound source direction detecting unit. In this case, the camera control means is used to set the face image of the person at the position closest to the sound source direction detected by the sound source direction detection means to the position on the screen set in advance and / or A camera control method characterized by controlling so as to display in a size. 4. The camera control method according to claim 1, wherein a face image of the person is not detected by the face image extracting unit in the sound source direction detected by the sound source direction detecting unit. And / or, when the face image of the person extracted by the face image extracting means is not fully contained in the screen, the camera control means is used to
The camera is automatically controlled to change the zoom to a wide-angle operation, and based on the imaged image obtained by the wide-angle operation, the face image extracting operation by the face image extracting means and the face image position determining means are performed. A camera control method, characterized in that a position and / or size of a face image displayed on a screen is recognized. 5. The camera control method according to claim 1, wherein the camera control unit controls a camera angle change and / or a zoom change in the sound source direction detected by the sound source direction detection unit. Further has a time measuring means for measuring an elapsed time after the execution, and the elapsed time measured by the time measuring means has passed a preset unit time. The camera control method is characterized in that the camera control means controls the camera angle change and / or the zoom change in the sound source direction detected by the sound source direction detection means each time. 6. The camera control method according to claim 1, wherein the face image extracting means extracts the face image of the person by identifying the skin color of the person. Control method. 7. The camera control method according to claim 1, wherein the face image extracting means extracts the face image of the person by identifying the contour of the person. Control method. 8. The camera control method according to claim 1, wherein the sound source direction detection means has a frequency and / or a delay time of each audio signal input to the plurality of audio input means. A method for controlling a camera, characterized in that the sound source direction is detected by analyzing the. 9. A camera control device capable of controlling an angle change in a horizontal direction and / or a vertical direction and / or a zoom change to a wide angle and / or a telephoto, further comprising any one of claims 1 to 8. A camera control device comprising means for enabling the camera control method according to claim 1 to be realized. 10. A video conference system for performing a video conference, comprising the camera control device according to claim 9.