JP2007228070A - Video conference apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video conference apparatus capable of transmitting voice and video of a talker at the same time at less expense in time and labor of installation. <P>SOLUTION: A talker position detection section 23 compares voice collection beam signals MB1 to MB6 on the basis of voice collection signals picked up by microphones 2A to 2H, and detects a talker direction corresponding to the voice collection beam signal with a highest signal level. A collection voice beam selection section 22 selects the voice collection beam signal with the highest signal level, gives the selected signal to an echo cancel section 24, and the echo cancel section 24 carries out echo cancellation and outputs a transmission voice signal to a communication control section 25. A video data processing section 41 segments talker video data within a prescribed range corresponding to the acquired talker direction from omniazimuth video data received from a camera 4 and gives the data to a communication control section 25. The communication control section 25 transmits the transmission voice signal and the talker video data to a video conference apparatus of an opposite party via a network nearly at the same time. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a video conference apparatus that bidirectionally communicates video with a speaker's voice between points distant from each other.

  Currently, many systems are used to hold a conference between points that are distant from each other, for example, a head office or a branch that are far away. In such a remote conference system, in addition to the audio conference system that communicates only audio, there is a video conference system that communicates video together with audio in order to give each other a more realistic feeling.

For example, in the video conference system described in Patent Document 1, a video conference apparatus using a plurality of microphones installed for each speaker and a separately installed camera is used. This video conference apparatus identifies a speaker from the magnitude of an output audio signal of a microphone installed in each speaker, and transmits the speaker identification information to the camera side using electromagnetic waves. A camera rotates based on this speaker specific information, and images a speaker.
JP-A-6-276514

  However, in the video conference apparatus as shown in Patent Document 1, it is necessary to install a microphone for each speaker, and it takes much time to prepare for the conference. In addition, if each microphone is a handheld microphone, the speaker has to move with the microphone each time it moves. In addition, since the camera rotates mechanically to switch the imaging range, the camera's rotation speed may not be able to follow the switching speed of the speaker. You may not be able to send.

  Accordingly, an object of the present invention is to configure a video conference apparatus that can transmit a speaker's voice and video at the same time without requiring installation.

  (1) A video conference apparatus according to the present invention includes a microphone array including a plurality of microphones arranged in a predetermined pattern, and a sound collecting beam signal in a plurality of directions using a sound collecting signal of each microphone of the microphone array. Sound beam forming means, speaker direction detecting means for detecting a speaker direction based on a plurality of sound collecting beam signals, and transmission voice signal selecting means for selecting a detected sound beam signal in the speaker direction as a transmission voice signal Image data forming means for imaging a region corresponding to each sound collecting beam signal to form video data, and outputting a speaker direction video data in a predetermined range corresponding to the sound collecting beam signal in the speaker direction, and transmission And a communication means for simultaneously transmitting the audio signal and the speaker direction video data.

  In this configuration, when the speaker voice is picked up by each microphone of the microphone array, the sound collecting beam forming means uses the sound collecting signal of each microphone to generate a sound collecting beam signal centered in a different direction. Form. The speaker direction detecting means compares the sound collecting beam signals and detects the direction corresponding to the sound collecting beam signal having the highest signal level as the speaker direction. The transmission voice signal selection means selects the detected sound pickup beam signal in the speaker direction from all the sound pickup beam signals, and outputs it as a transmission voice signal. The video data forming means captures an area corresponding to each collected sound beam signal and generates speaker direction video data which is video data in a predetermined range including the detected speaker direction. The communication means transmits the transmission audio signal and the speaker direction video data at the same time to the counterpart device, for example, via a network. As a result, communication is performed in a state where the voice and video of the speaker who is actually speaking match.

  (2) Further, the video data forming means of the video conference apparatus according to the present invention includes an imaging means for forming an omnidirectional video data by imaging omnidirectional video data in a region where a plurality of microphones collect sound, and talking from the omnidirectional video data. Speaker direction video data forming means for cutting out and generating the speaker direction video data.

  In this configuration, the imaging unit images all directions regardless of the current speaker position, and the speaker direction video data forming unit cuts out only the portion corresponding to the speaker direction from the video data in all directions. As a result, even if the speaker is switched, the cut-out portion for the video data in all directions only changes, so that the video is switched at a higher speed than when the imaging direction is moved by a mechanical operation.

  (3) In the video conference apparatus according to the present invention, the plurality of microphones of the microphone array are arranged circumferentially along the peripheral surface of the substantially cylindrical casing. In addition, the imaging means guides a camera installed in the casing with the central axis of the casing along the peripheral surface as the optical axis of the lens, and images in all directions in the circumferential direction to the lens along the direction of the optical axis. And a curved reflector.

  Specifically, in this configuration, the video conference device is formed in a substantially cylindrical shape, and the microphone array and the imaging unit are provided in a substantially cylindrical housing. Sound from the circumferential direction is picked up by each microphone of the microphone array installed circumferentially along the circumferential surface of the housing. On the other hand, the image in the circumferential direction is picked up by the camera in all directions simultaneously through the curved reflector. As a result, images in all directions in the circumferential direction are captured as needed. Then, the necessary range, that is, the video data in the range corresponding to the detected speaker direction is cut out from this omnidirectional video, and the speaker direction video data is generated, so that it can be operated at high speed without mechanical operation. The video is switched.

  (4) In the video conference apparatus according to the present invention, the plurality of microphones of the microphone array are arranged circumferentially along the peripheral surface of the substantially cylindrical casing. The imaging unit includes a plurality of cameras installed in different directions in the circumferential direction of the casing so that the imaging ranges partially overlap each other and cover all directions. The speaker direction video data forming means outputs the video data of the camera directed to the detected speaker direction as the speaker direction video data.

  In this configuration, as in the case of (3), the video conference device is formed in a substantially cylindrical shape, and the microphone array and the image pickup means are provided in the substantially cylindrical housing, and sound from the circumferential direction is received from the housing. Sound is picked up by each microphone of a microphone array arranged circumferentially along the circumference of the body. On the other hand, images in the circumferential direction are simultaneously imaged in all directions by a plurality of cameras having different directions in the optical axis direction of the lens. Then, from this omnidirectional video, select the video data of the camera that captures the range corresponding to the detected speaker direction, and generate the speaker direction video data, without mechanical operation, Video switching is performed at high speed.

  (5) In the video conference apparatus according to the present invention, a plurality of microphones of the microphone array are arranged circumferentially along the peripheral surface of the substantially cylindrical casing, and the microphones collect sound in the circumferential direction. In addition, the video data forming means includes a camera installed in the casing with the central axis of the casing along the peripheral surface as the optical axis of the lens, and a video in a predetermined range in the circumferential direction along the direction of the optical axis. And a biasing means for supporting the reflecting plate so as to be rotatable with respect to the central axis and for rotating the reflecting plate in the detected speaker direction.

  In this configuration, as in the case of (3) and (4), the video conference device is formed in a substantially cylindrical shape, and the microphone array and the imaging means are provided in the substantially cylindrical housing, and from the circumferential direction. The sound is collected by each microphone of the microphone array that is circumferentially installed along the peripheral surface of the housing. On the other hand, the image in the circumferential direction is picked up by a camera through a reflecting plate that rotates according to the detected speaker direction. In this case, the image is switched by a mechanical operation. However, since the reflecting plate is lighter than the camera, the image can be rotated at high speed, and the switching of the image is also speeded up.

  (6) The video conference apparatus according to the present invention is characterized by including sound emitting means for emitting sound from a speaker in accordance with speaker direction video data.

  In this configuration, since the main sound emitting direction is set in the direction of the speaker, if there is only one speaker, the sound can be emitted only in the direction of the speaker. In addition, when there are a plurality of conference persons and the speakers are switched, it is possible to mainly emit sound toward the speaker currently speaking. (I'm sorry, do you agree with this view?)

  According to the present invention, it is possible to detect the speaker direction from the sound collected by the microphone array, switch the video in the detection direction at high speed, and simultaneously transmit the speaker voice and the speaker video without any sense of incongruity. it can.

A video conference apparatus according to a first embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a perspective view showing the configuration of the video conference apparatus according to the present embodiment.
2A is a plan view showing the configuration of the video conference apparatus according to the present embodiment, FIG. 2B is a side view, and FIG. 2C is a bottom view.

  As shown in FIG. 1 and FIG. 2, the video conference apparatus 1 includes a substantially cylindrical housing 10, and the housing 10 has a circumferential surface as a side surface and two opposing circular surfaces as a top surface and a bottom surface, respectively. To do.

  The microphones 2 </ b> A to 2 </ b> H may be omnidirectional or directional, and are installed along the peripheral surface of the substantially cylindrical housing 10 so that the sound collection direction faces the external direction from the peripheral surface. Has been. The microphones 2 </ b> A to 2 </ b> H are installed axisymmetrically with respect to a central axis parallel to the peripheral surface in a cylinder forming the housing 10. That is, if there are eight microphones 2A to 2H, they are arranged at intervals of 45 ° with the point on the central axis as the center point. Note that the number of microphones is not limited to eight, and may be set as appropriate according to specifications.

  The speaker 3 is installed on the bottom surface of the housing 10 so that the sound emission direction is from the bottom surface to the external direction. At this time, the center axis of the speaker 3, that is, the center in the sound emission direction is made to coincide with the center axis of the housing 10.

  The legs 13 are installed at equal intervals at a predetermined angle on the bottom surface of the casing 10, and the legs 10 allow the casing 10 of the video conference apparatus 1 to be spaced a predetermined distance from a grounding surface such as a desk top. Placed at the height of

  The camera 4 is installed inside the top surface of the housing 10 in a state where the direction perpendicular to the top surface of the housing 10 is the light receiving direction. The camera 4 includes a lens and a light receiving unit, and is installed so that the optical axis of the lens coincides with the central axis of the housing 10 and the point on the optical axis is the center of the light receiving region.

  A cylindrical transparent window 12 having the same dimensions as the peripheral surface of the housing 10 is installed on the top surface of the housing 10, and guides light from the outside to an internal space surrounded by the window 12.

  Further, a canopy 11 having a predetermined thickness and having the same shape as the plan view of the housing 10 is installed on the top surface side of the housing 10 through the window 12. A hyperboloid mirror 5 is installed on the surface of the canopy 11 on the side of the housing 10.

  The hyperboloid mirror 5 is installed such that the center in plan view is on the central axis of the housing 10, and guides light from all horizontal directions, that is, images, that are incident from the window 12 to the lens of the camera 4. . In the present embodiment, a hyperboloidal mirror is shown, but the shape of the mirror may be any structure that guides light from all directions in the horizontal direction to the lens.

  With such a configuration, the microphones 2 </ b> A to 2 </ b> H collect sound from all directions in a substantially horizontal direction, the sound is output from the speakers 3 in all directions, and the camera 4 captures images in all directions. To do.

  Further, the video conference apparatus 1 has a signal processing function unit as shown in FIG.

  FIG. 3 is a block diagram illustrating the configuration of the signal processing function unit of the video conference apparatus 1 according to the present embodiment.

  The microphones 2 </ b> A to 2 </ b> H collect sound from the periphery of the video conference apparatus 1, and the speaker 3 emits an input audio signal corresponding to the sound collected by another video conference apparatus.

  The microphone array 20 includes the above-described microphones 2A to 2H, and each microphone 2A to 2H collects sound and performs electrical conversion and A / D conversion to generate sound collection signals MA to MH, respectively. Output to the sound beam forming unit 21.

  The collected sound beam forming unit 21 performs signal processing such as delay sum processing using the collected sound signals MA to MH input from the microphones 2A to 2H, and has directivity having high sound collection sensitivity in each predetermined direction. Sound collecting beam signals MB1 to MB6 are generated. These sound collection beam signals MB1 to MB6 are set with different directivity axial directions in the horizontal direction. The directivity of each of the sound collection beam signals MB1 to MB6 is the speaker in all horizontal directions. It is set to be within the detection range. For example, if there are six sound collecting beams, they are set at substantially equal angular intervals so that each of them is at an interval of about 60 ° in the horizontal plane. Thereby, high sound collection sensitivity is set in a plurality of different directions with respect to the horizontal direction of the video conference apparatus 1.

  When the speaker position detection unit 23 acquires the sound collection beam signals MB1 to MB6, the speaker position detection unit 23 compares the signal level (amplitude intensity), selects the sound collection beam signal having the highest signal level, and corresponds to the sound collection beam signal. The direction, for example, the directivity axial direction of the detected sound pickup beam signal is detected as the speaker direction. The speaker position detection unit 23 provides the detected sound collection beam signal information to the sound collection beam selection unit 22 and also provides speaker direction information to the video data processing unit 41.

  The sound collection beam selection unit 22 selects a corresponding sound collection beam signal from the sound collection beam signals MB1 to MB6 based on the sound collection beam signal information obtained from the speaker position detection unit 23, and sends it to the echo cancellation unit 24. Output.

  The echo cancellation unit 24 includes an adaptive filter 241 and a post processor 242 including an adder. The adaptive filter 241 generates a pseudo-regression sound signal based on the input voice signal input via the communication control unit 25 and supplies the pseudo regression sound signal to the post processor 242. The post processor 242 subtracts the pseudo-regression sound signal from the collected sound beam signal from the collected sound beam selection unit 22 and outputs the difference to the communication control unit 25 as a transmission voice signal. At this time, the adaptive filter 241 obtains the residual signal output from the post processor 242 and optimizes the filter coefficient to generate a more appropriate pseudo-regression sound signal.

The camera 4 captures all horizontal azimuth images input via the hyperboloid mirror 5 at a time and outputs the omnidirectional video data to the video data processing unit 41.
When acquiring the speaker direction information, the video data processing unit 41 cuts out the speaker direction video data corresponding to the speaker direction from the omnidirectional video data. That is, the video data processing unit 41 cuts out video data over a predetermined angle range including a range including the video of the speaker from the omnidirectional video data with respect to the acquired speaker direction, and extracts this from the omnidirectional video data. Output as video data. At this time, the video data processing unit 41 obtains an image as if the speaker direction was viewed in plan by performing perspective projection conversion on the speaker video data.

  The communication control unit 25 converts the transmission audio signal output from the echo cancellation unit 24 and the speaker video data output from the video data processing unit 41 into a data format corresponding to the network protocol to which the communication control unit 25 is connected. The two data are output to the network almost simultaneously.

  With this configuration and processing, the sound generated from the speaker can be collected with a high S / N ratio and transmitted to the other party, and the video of the speaker is transmitted simultaneously with the generated sound. Therefore, it is possible to realize an easy-to-understand video conference for both parties. Further, when the speaker is switched, the video switching can be realized only by switching the range of the speaker video data cut out from the omnidirectional video data, so that the video switching can be performed at high speed. Thereby, when the speaker is switched, it is possible to prevent a discrepancy between the audio and the video such that the audio is of the new speaker and the video is the previous speaker. As a result, a user-friendly video conference can be performed.

  Next, a video conference apparatus according to the second embodiment will be described with reference to FIG.

  FIG. 4A is a plan view showing the configuration of the video conference apparatus 1B of the present embodiment, and FIG. 4B is a side view.

The video conference apparatus 1B of the present embodiment includes a plurality of cameras 4A to 4F, does not include a hyperboloid mirror, and other configurations are the same as those of the video conference apparatus 1 of the first embodiment.
In the video conference apparatus 1 </ b> B of the present embodiment, the support shaft 14 is installed at the center of the internal space surrounded by the window 12 on the top surface side of the housing 10, and the cameras 4 </ b> A to 4 </ b> F are installed around the support shaft 14. Is. Note that the number of cameras is not limited to six, and may be set as appropriate according to specifications. The cameras 4A to 4F are installed at equiangular intervals on a horizontal plane, and the fields of view of the cameras are overlapped at their respective ends, and are installed so as to capture images in all horizontal directions with all the cameras 4A to 4F. Yes. For example, in FIG. 4, the cameras 4 </ b> A to 4 </ b> F are installed at 60 ° intervals, and images of all directions in the horizontal direction are captured by the six cameras 4 </ b> A to 4 </ b> F. The optical axes of these cameras 4A to 4F are set so as to coincide with the directivity axis directions of the sound collecting beam signals MB1 to MB6, for example.

  In the case of such a structure, the video data processing unit 41 selects and outputs the acquired video of the camera whose optical axis is in the direction of the speaker as speaker video data.

  Even in such a configuration and processing, as in the first embodiment described above, the sound collection direction and the imaging direction can be switched at the same time in accordance with the switching of the speaker. A meeting can be held. Furthermore, in the configuration of the present embodiment, planar speaker video data can be acquired without performing transmissive projection conversion, so that video data conversion processing is not required and video switching is performed at higher speed. Can do.

Next, a video conference apparatus according to the third embodiment will be described with reference to FIGS.
FIG. 5 is a side view showing the configuration of the video conference apparatus 1C of the present embodiment.
FIG. 6 is a block diagram illustrating a configuration of a signal processing function unit of the video conference apparatus 1C according to the present embodiment.
The video conference apparatus 1C of the present embodiment includes a flat mirror 15 that is rotatably installed instead of the hyperboloid mirror. The mirror is not limited to a flat mirror, but may be any mirror that can take lens focus such as a concave mirror or a convex mirror.
In addition, a mirror control unit 42 is provided without using the video data processing unit 41.

  The flat mirror 15 is installed such that the reflection surface is inclined at approximately 45 ° with respect to the vertical direction (and the horizontal direction), and guides light (video) in a predetermined range in the horizontal direction to the camera 4. The flat mirror 15 is supported by a support shaft 16 with respect to the canopy 11, and is connected to a servo motor 17 installed on the canopy 11 via the support shaft 16. When the servo motor 17 rotates, the flat mirror 15 rotates according to the amount of rotation. By this rotation, the flat mirror 15 guides the image in the direction of the reflecting surface in the image in all directions in the horizontal direction to the camera 4. The installation angle of the flat mirror 15 is not limited to this, and can be set as appropriate.

  In such a structure, the speaker position detection unit 23 gives the speaker position information to the mirror control unit 42. Based on the acquired speaker position information, the mirror control unit 42 calculates an angle difference between the current front direction of the plane mirror 15 and a new speaker direction, and generates a servo control signal corresponding to the angle difference. . The servo motor 17 operates according to the servo control signal and rotates the plane mirror 15. At this time, the rotation amount is synchronized with the frame rate constituting the video.

  The video imaged by the camera 4 via the plane mirror 15 is a video of a predetermined range including a new speaker, and the camera 4 outputs this speaker video data to the communication control unit 25 as it is.

  In the case of such a structure, a mechanical operation of rotating the plane mirror 15 is required. However, the plane mirror 15 is lighter than the camera 4 and thus has a higher rotation speed. As a result, video switching can be performed at a higher speed than in the past, and a user-friendly video conference can be performed as in the above-described embodiments.

  In the present embodiment, an example using a servo motor has been shown, but a motor with an origin sensor may be used.

  In each of the above-described embodiments, an example in which six sound collecting beam signals are generated has been described. However, the number of sound collecting beam signals to be generated may be set as appropriate according to specifications.

  In the second embodiment described above, the number of cameras is matched with the number of collected sound beam signals, and the optical axis of the camera (lens) is matched with the directivity axis of the collected sound beam. A configuration that does not match may be used.

  In each of the above-described embodiments, an example in which an omnidirectional speaker is used as the speaker has been described. However, a speaker array including a plurality of speakers may be arranged on the bottom surface of the housing. In this case, directivity is set in accordance with the speaker direction video data, and a sound emission beam emitted from each speaker of the speaker array is formed, so that the speaker direction is set as a main direction. Sound can be emitted.

It is a perspective view which shows the structure of the video conference apparatus of 1st Embodiment. It is the top view, side view, and bottom view which show the structure of the video conference apparatus of 1st Embodiment. It is a block diagram which shows the structure of the signal processing function part of the video conference apparatus 1 of 1st Embodiment. It is the top view and side view which show the structure of the video conference apparatus of 2nd Embodiment. It is a side view which shows the structure of the video conference apparatus of 3rd Embodiment. It is a block diagram which shows the structure of the signal processing function part of the video conference apparatus 1 of 3rd Embodiment.

Explanation of symbols

1-Video conferencing apparatus, 10-case, 11-canopy, 12-window, 13-leg, 14, 16-support shaft, 15-flat mirror, 17-servo motor, 2A-2H-microphone, 3-speaker , 4, 4A to 4F-camera, 5-hyperboloid mirror, 20-microphone array, 21-sound collecting beam forming unit, 22-sound collecting beam selecting unit, 23-speaker position detecting unit, 24-echo canceling unit, 241-adaptive filter, 242-post processor, 25-communication control unit, 41-video data processing unit, 42-mirror control unit

Claims (6)

A microphone array having a plurality of microphones arranged in a predetermined pattern;
A sound collecting beam forming means for forming a sound collecting beam signal in a plurality of directions using a sound collecting signal of each microphone of the microphone array;
Speaker direction detecting means for detecting a speaker direction based on the plurality of collected sound beam signals;
Transmission voice signal selection means for selecting the detected sound beam signal in the direction of the speaker as a transmission voice signal;
Video data forming means for imaging a region corresponding to each sound collecting beam signal and outputting speaker direction video data in a predetermined range corresponding to the sound collecting beam signal in the speaker direction;
Communication means for simultaneously transmitting the transmission audio signal and the speaker direction video data;
A video conferencing apparatus comprising: The video data forming means includes
Imaging means for imaging omnidirectional images of a region where a plurality of microphones collect sound to form omnidirectional video data;
Speaker direction video data forming means for cutting out and generating the speaker direction video data from the omnidirectional video data;
The video conference apparatus according to claim 1. The plurality of microphones of the microphone array are arranged circumferentially along the circumferential surface of a substantially cylindrical housing,
The imaging means includes
A camera installed in the housing with the central axis of the housing along the peripheral surface as the optical axis of the lens;
A curved reflector for guiding the image in all directions in the circumferential direction to the lens along the direction of the optical axis;
The video conference apparatus according to claim 2. The plurality of microphones of the microphone array are arranged circumferentially along the circumferential surface of a substantially cylindrical housing,
The imaging means includes a plurality of cameras installed in different directions in the circumferential direction of the housing so that the imaging ranges partially overlap each other and cover all the directions,
3. The video conference apparatus according to claim 2, wherein the speaker direction video data forming unit outputs video data of a camera directed to the detected speaker direction as the speaker direction video data. The plurality of microphones of the microphone array are arranged circumferentially along the circumferential surface of a substantially cylindrical housing,
The video data forming means includes
A camera installed in the housing with the central axis of the housing along the peripheral surface as the optical axis of the lens;
A reflector for guiding an image in a predetermined range in the circumferential direction to the lens along the direction of the optical axis;
An urging means for rotatably supporting the reflecting plate with respect to the central axis and rotating the reflecting plate in the detected speaker direction;
The video conference apparatus according to claim 1.   6. The video conference apparatus according to claim 1, further comprising sound emitting means for emitting sound from a speaker in accordance with the speaker direction video data.

Images