JPH10191290A - Video camera with built-in microphone - Google Patents

Abstract

PROBLEM TO BE SOLVED: To precisely collect the voice of a speaker by incorporating plural microphones in a video camera, changing the synthesis characteristics of the outputs of respective microphones in accordance with the racking angle and the zoom angle of the video camera and searching a maximum direction with a second beam forming circuit. SOLUTION: An array microphone output control part 50 receives a control signal corresponding to the movement of a pan tilt zoom camera unit 80 from a camera control part 60 and decides parameters given to variable delay circuits 14, 24 and 34 and variable gain circuits 15, 25 and 35. The output signals are added in an adder 40 and the voice of the speaker is precisely grasped and outputted. Input signals are simultaneously added to second variable delay circuits 71, 72 and 73 and a search control part 70 sequentially sets the delay time of the variable delay circuits 71, 72 and 73 and searchs the direction of a sound source for scanning the direction of the maximum sensitivity of the array microphone. Thus, the conversation high in clarity is realized.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an audio / video input device used for a video conference system or the like.

[0002]

2. Description of the Related Art Generally, a video conferencing system that is marketed can swing a directivity angle in the left, right, up and down directions in order to satisfy a request for projecting a wide area and a request for projecting a specific area in an enlarged manner. It is equipped with an electric pan-tilt zoom camera that can increase or decrease the range, in other words, continuously change the telephoto and wide-angle.
In such a system, since the participants of the conference are seated in a wide area, a plurality of microphones are usually arranged in a distributed manner for the purpose of picking up the voice of each participant evenly. Japanese Patent Laid-Open No. 1-264487 has devised a video camera which estimates the speaker direction by comparing the sound receiving levels of a plurality of microphones and automatically points the camera to the estimated speaker. Thus, the speaker is estimated,
Devices that automatically point the camera toward the speaker can provide convenience to the user.

[0003]

In the above conventional example, it is necessary to install microphones at least as many as the number of conference participants in the vicinity of the participants, and it is necessary to set microphone position information in advance on the camera side. Was. For this reason, the system must be set each time the number of participants increases or decreases or the arrangement of participants changes, resulting in a system that is extremely difficult to use, causing inconvenience to users. In addition, since many microphones are set on the conference table, it is necessary to lay a microphone cable between the microphone and the main body, and there is a problem that the appearance is impaired and the foot is hooked on the wiring on the floor. An object of the present invention is to solve the above-described drawbacks, to estimate a speaker without installing a microphone on a conference table, and to automatically point a camera in the direction of the speaker,
Furthermore, a function for preferentially searching for the speaker direction set in advance by the user, and when the automatically searched sound source direction does not match the preset direction, a new value is added to the set value and the next search is performed. Since the sound source search direction can be further improved, a more comfortable video conference can be realized.

[0004]

The above object is achieved by incorporating a plurality of microphones into a video camera which is indispensable for a video conference system, and synthesizing output signals of the microphones using a beam forming technique. The output of each microphone is input to two beamforming circuits. In the first beam forming circuit, the synthesizing characteristics are changed according to the swing angle and zoom angle of the video camera, so that the speaker can be accurately caught and collected, thereby realizing low-noise, high-clarity sound collection. . The second beamforming circuit scans the direction of the sound source to obtain the maximum sound receiving level, in other words, searches for the direction in which the output of the beamforming microphone becomes maximum, thereby detecting the speaker direction. The detection result is input to the video camera control circuit, and the video camera can be directed toward the speaker. Further, a function of preferentially searching for the speaker direction set in advance by the user, and when the automatically searched sound source direction does not match the preset direction, it is newly added to the set value,
Since it is used for the next search and the sound source direction search direction can be further improved, a more comfortable video conference can be realized.

[0005] For details of the beam forming technique, see “Multid
imentional Digital Signal Processing ”(Prentice Ha
ll). With such a configuration, the direction of the speaker is searched, the direction of the speaker is searched, and the direction of the video camera is changed by the arrangement of the plurality of microphones and the operation of the delay circuit and the addition circuit connected thereto. Automatically change to speaker direction. The array of the plurality of microphones is connected to a delay circuit, a gain circuit, and an addition circuit, which are different from the delay circuit and the addition circuit, and delays signals obtained from the plurality of microphones in conjunction with a camera shake angle and a zoom angle. , And changing the gain and adding the values, it is possible to change the directional direction and directional characteristics of the microphone. By such an operation, the camera can be directed to the speaker of the video conference participant without any operation of the video camera by the participant, and the sound can be collected with high sensitivity. Never. Further, when the shooting range of the camera is widened, the voice of the entire participant can be collected, so that the directional characteristics can be widened. As described above, since the sound pickup characteristics of the sound can be easily changed without performing the operation of changing the direction of the camera, a TV conference that is not affected by noise and has a sense of reality can be performed without performing a troublesome operation. realizable.

[0006]

Embodiments of the present invention will be described below with reference to the drawings. Prior to the description of the embodiments of the present invention,
An outline of the beam forming technique will be described. Let the number of microphones be N and the output of the i-th microphone be ri (t). An appropriate delay time τi is given to each of the microphones so that the output from each microphone with respect to a signal from a desired direction becomes in-phase, and the signals are added by multiplying by a weighting factor Wi to obtain an output signal. Since signals other than the signal propagating from the desired direction do not become in-phase, if added, they cancel each other out and attenuate, so that sharp directivity can be realized. The directional characteristics are determined by a weighting factor, delay time, and the number of microphones, and can follow the position of the signal source by adaptively changing the weighting factor and the delay amount. The output signal y (t) is represented by the following equation.

[0007]

(Equation 1)

By utilizing this characteristic, it is also possible to search for an unknown sound source direction. The direction of the sound source can be searched using the characteristic that the direction having the maximum sensitivity changes depending on the value of τi. FIG. 1 shows an example in which three microphones according to the embodiment of the present invention are connected. The input audio signal is collected by microphones (11, 21, 31), amplified by microphone amplifiers (12, 22, 32) and input to analog-to-digital converters (13, 23, 33) (hereinafter A / D). In the A / D, the input audio signal is sampled, discretized, and converted into a digital input audio signal. The digital input audio signal input to the variable delay circuits (14, 24, 34) is output after delaying the input signal by the delay time determined by the array microphone output control unit (50). The delayed digital input audio signal is supplied to a variable gain circuit.
(15, 25, 35), the array microphone output control unit (5
0) is multiplied by the gain determined by the adder (4).
0) to obtain a digital output audio signal. At the same time, the digital input audio signal is input to a variable delay circuit (71, 72, 73) different from the variable delay circuit, and the search control unit (70) controls the variable delay to scan the array microphone in the maximum sensitivity direction. The delay time is sequentially set for the circuit, and the sound source direction is searched. The searched sound source direction is transmitted to the camera control unit (60), and the camera control unit controls the camera unit to direct the camera to the sound source direction. The digital output audio signal is converted into an analog output audio signal by a digital / analog converter (41) (hereinafter D / A). However, the output format of this device may be a digital output audio signal only. In this case, D / A
Is unnecessary. A plurality of combinations from the microphone to the variable gain circuit are required according to the number of microphones. The voice input unit control unit (50) receives a control signal from the camera control unit (60). The control signal corresponds to the movement of the electric pan-tilt zoom camera unit (80), and the right and left swing angle (pan) , The zoom state, and the voice input unit control unit (5)
0) is based on this control signal, and each variable delay circuit (14,
24, 34) and a variable gain circuit (15, 25, 35)
Determine the parameters to be given to First, the search control unit
The operation (70) will be described in detail. When the sound input to each microphone is incident from the front, the sound is collected in the same phase. However, when the light is incident from an oblique direction, as shown in FIG. 2, a time difference occurs in the input signal for each microphone. Addition of the signals input from each microphone causes a phase difference between the signals,
Cancel and attenuate. Here, as shown in FIG. 3, from the signals collected by the microphones using a delay circuit,
If the time difference between the signals is removed, the signal becomes equivalent to a sound input from the front, and the effect of having high directivity in that direction can be obtained. The directivity can be changed by controlling the delay time of the delay circuit. By utilizing this characteristic, the direction of the microphone can be scanned in the directions of a plurality of speakers, and the direction with the highest sound receiving level can be searched. As a result, since the speaker direction can be estimated, it is possible to control the video camera to automatically point at the speaker direction. Next, the operation of the array microphone output control unit (50) will be described in detail. Therefore, the function of the voice input unit control unit (50) is such that the variable delay circuit is provided so as to have directivity in a desired direction.
(14, 24, 34) and the variable gain circuit (1) to change the sharpness of the directional characteristic.
5, 25, 35). First, a delay amount determining method will be described. The microphone array shown in this embodiment is characterized in that sharp directivity is obtained. Therefore, the voice of the speaker in the direction of the camera can be efficiently collected, but the voice and noise of the speaker not in the direction of the camera are hardly collected. In general, a camera used in a video conference system can be turned in the direction of a speaker by an operator and projected, so that the directivity direction of the microphone may be changed according to the swing angle of the camera. Since the camera swing angle is controlled by the camera control unit (60), the direction of the camera can be known. In this system, since the input audio signal is treated as a discrete signal, the setting itself of the delay amount between the microphones becomes discrete. Therefore, the camera shake angle is quantized, the obtained index is used to search a table storing the delay amount created in advance, the delay amount of each variable delay circuit is obtained, and the variable delay circuit is set to a desired value. Characteristics can be obtained. When there is only one speaker, the speaker is zoomed up and projected, so there is no discomfort even if the microphone has sharp directivity, but it does not weaken the directivity when many speakers are talking at the same time Cannot be collected, and inconvenience occurs. Therefore, the camera control unit (60) has a function of controlling the sensitivity of each microphone and changing the directional characteristics. If there are many speakers or the entire participant is projected without operating the camera each time, the zoom setting of the camera should be adjusted to the wide-angle side in order to widen the shooting range. Also in this case, since the camera control is performed by the camera control unit (60), the user can know the zoom setting state. In the present embodiment, the central microphone is controlled as the main unit, but the gain of the signal obtained from the central microphone is increased, and the gain of the signal obtained from the microphones at both ends is reduced. The strength of the characteristics can be controlled. As an extreme example, the widest directivity can be obtained by using only the signal obtained from the central microphone. Also in this case, it is desirable to quantize the zoom angle of the camera, use the obtained index to search a table storing gain amounts created in advance, obtain the gain amounts of the respective variable gain circuits, and set the variable gain circuits. Characteristic can be obtained. Thus, excellent effects can be obtained with an extremely simple configuration. FIG. 4 shows changes in the directional characteristics of the microphone according to the present embodiment when the delay amount is changed.
Shown in FIG. 5 shows a change in the directional characteristics of the microphone according to the present embodiment when the gain amount is changed. Generally, when the number of microphones is increased, the directivity becomes sharper. Further, if the interval between the microphones is widened, the directional characteristics can be exhibited even at a low frequency, but in any case, the device becomes large. Also, when the A / D sampling rate is increased, the accuracy of delay amount setting is improved, and the setting accuracy of the directional direction is increased. By performing the above control, the voice direction of the speaker can be efficiently collected by changing the directivity direction of the microphone according to the shake angle of the camera, and when the camera zoom is set to the wide side. Is to pick up the voice of the entire participant by widening the directional characteristics, and if the camera zoom is set to the tele side, sharpen the directional characteristics to effectively pick up the voice of the speaker reflected on the camera Can be. Next, a description will be given of presetting of the sound source direction and improvement of the sound source direction search accuracy using the preset sound source direction. By setting the speaker direction to the camera in advance when using the video conference, the camera can be instantly turned to the speaker direction. The so-called camera preset function is realized by many devices. In the device according to the present invention, the direction of the speaker is automatically estimated, so even if there is no camera preset function, it is sufficiently effective as compared with the conventional product, but when the speaker is near, or there is much noise and reverberation. In some environments, correct direction estimation may not always be possible. Therefore, a camera conference function can be added as an additional function, and a more convenient TV conference can be realized by combining the sound source search function described above. The user of the video conference can set the camera direction and the screen display range in the preset memory using the camera preset function according to the place where the speaker sits when starting the conference. Since it is expected that the speaker is actually in the preset direction, an efficient search is realized by preferentially searching the sound source direction preset in the preset memory during automatic sound source search. I do. However, in the search for the sound source direction, if a significant search result cannot be obtained in the sound source direction stored in the preset memory, another direction is searched to estimate a new sound source. Here, the estimated sound source direction is additionally stored in the empty area of the preset memory, and is used in the next sound source direction search. If there is no free area, the memory storing the direction with the lowest search frequency among the sound source directions stored in the preset memory may be overwritten.

[0009]

According to the present invention, the speaker direction can be estimated without placing a microphone near the speaker, and the touch angle of the camera can be directed to the speaker direction. In addition, by changing the direction of the microphone according to the camera's swing angle, the voice of the speaker can be collected efficiently, so that it is not affected by chat by other participants or noise in the room. It becomes difficult. In addition, when the camera zoom is set to the wide side, the voice of the entire participant can be picked up by widening the directional characteristics of the microphone. Conversely, when the camera zoom is set to the tele side, the directional characteristics should be sharpened. Can be. Thereby, the voice of the speaker reflected on the camera can be effectively picked up. Therefore, it is possible to accurately capture the speaker and collect sound, so that conversation with less noise and high clarity can be realized. There is an effect that an extremely comfortable video conference can be realized without performing the video conference.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of an embodiment according to the present invention.

FIG. 2 is an explanatory view of the principle (characteristics at oblique incidence without a delay circuit)

FIG. 3 is an explanatory view of the principle (when a delay circuit is added to provide maximum sensitivity in an oblique incident direction).

FIG. 4 is an explanatory diagram of a change in a directivity direction according to the embodiment.

FIG. 5 is an explanatory diagram of a change in directional characteristics according to the embodiment.

[Explanation of symbols]

 11, 21, 31 microphone, 12, 22, 32 microphone amplifier, 13, 23, 33 analog / digital converter, 14, 24, 34 variable delay circuit 15, 25, 35 variable gain circuit, 40 adder 41 digital / analog converter, 50 Array microphone output control unit 60 Camera control unit, 70 Search control unit 71, 72, 73 Variable delay circuit 74 Adder, 80 Electric pan-tilt zoom camera unit

Claims (5)

[Claims] 1. A video signal camera with a built-in microphone that includes a plurality of microphones and has an electric pan-tilt zoom function capable of changing a vertical and horizontal direction and a screen display range by an external control signal. The circuit includes an amplifier, an analog-to-digital converter, a first group of delay circuits and gain circuits, and a first group for adding output signals from these gain circuits, respectively. A second adder circuit, wherein the second signal processing circuit is connected to a second group of delay circuits corresponding to the microphones in a one-to-one correspondence with the output signals from each of the analog-to-digital converters. A second addition circuit for adding the output signals from the group of delay circuits, respectively, to the information of the directional angle and the screen display range input by the external control signal. A video camera with a built-in microphone, based on which the first signal processing circuit and the second signal processing circuit are controlled. 2. The delay time of the first group of delay circuits can be set arbitrarily within a predetermined range, and a delay time predetermined by the left and right swing angles of the video camera can be individually set. 2. The video camera with a built-in microphone according to claim 1, wherein the directional direction of the microphone can be set in the same direction as the swing angle direction. 3. The gain coefficients of the plurality of gain circuits can be arbitrarily set within a predetermined range, and the gain coefficients predetermined according to the screen display range of the video camera can be individually set. 3. The video camera with a built-in microphone according to claim 2, wherein the directivity of the microphone can be widened when the distance is wide, and conversely, the directivity of the microphone can be narrowed when the screen display range of the camera is narrow. 4. The delay time of the second group of delay circuits is sequentially set to a predetermined combination,
4. The video camera with a built-in microphone according to claim 3, wherein the direction of the sound source is estimated by changing the directional direction of the microphone and comparing the sound receiving level for each directional direction. 5. A memory having a plurality of memories for temporarily storing the information of the directivity angle and the screen display range, wherein a user can store the information by using an external control signal; A memory to be stored based on the estimated sound source direction can be selected for each memory, and when searching for the sound source direction, the sound source direction can be estimated by preferentially searching for the directional angle direction stored in the memory. The video camera with a built-in microphone according to claim 4.