JP2004505560A - Aiming the device at the sound source - Google Patents

Abstract

The system is aimed at targeted devices that respond to sounds emanating from various possible sources. In a meeting or debate, for example, the system will aim a video camera (4) or light on a current speaker (6B). The control system is responsive to the new source of sound (6A) and the duration of continuous sound divergence from the difference in angle between the current target (6B) and the new source of sound (6A). If each exceeds the threshold, the camera (4) is aimed again. The sound need not be completely continuous, so that short pauses (in terms of breathing) are not reacted as breaking continuous speech.

Description

[0001]
FIELD OF THE INVENTION The present invention relates generally to the field of auto-aiming systems, such as video tracking used in video conferencing systems.
[0002]
Background of the Invention Automatic camera tracking systems automatically acquire and maintain targets within the camera's field of view based on a number of criteria, such as motion, color, image processing, and the like. For example, a video conferencing system may track a desired target using a pan-tilt-zoom camera. One class of such devices maintains the current speaker track by automatically determining the current speaker in the group and focusing on the current speaker. Whenever a new speaker starts talking, it automatically detects the new speaker by noise level and / or other criteria and continues to focus on the new speaker.
[0003]
Conventionally, many different techniques for determining current speakers have been proposed. While such techniques allow automatic cameras to determine the presence of new speakers and thus focus on new speakers, they suffer from significant drawbacks. Such systems switch very slowly from speaker to speaker, thereby skipping most of the conversation, or switching too quickly, confusing camera tracking.
[0004]
One solution to this problem has been proposed by US Patent Application No. 5,206,721. Such patent describes a method for aiming at current speakers in a system where each speaker has a separate microphone. In this system, input signals are collected from each of a number of microphones on each speaker. The total talking time is accumulated for each speaker over successive intervals. At a given time, the current speaker with the longest accumulated speaking time is determined.
[0005]
The problem with the above approach is that the current speaker determination requires multiple microphones. The use of multiple microphones makes it difficult to distinguish current loudspeakers due to the normal close proximity of microphones and echoes and extraneous noise generation within enclosed spaces or with isolation and objects. In addition, the disclosed method does not effectively prevent the undesired effects of continuous camera movement between two speakers located in close proximity.
[0006]
Thus, there is a need for a method of determining the current speakers in which automatic cameras in automatic video conferencing technology continue to concentrate, overcoming the shortcomings of the prior art.
[0007]
SUMMARY OF THE INVENTION The invention is defined by the independent claims, while the dependent claims define useful embodiments.
[0008]
In summary, the system is aimed at devices that are targeted in response to sounds from various possible sources. For example, in a meeting or discussion, the system would be used to aim a video camera or light at a current speaker. The control system is responsive to the new source of sound and the duration of continuous sound divergence from the angular difference between the current target and the new source of sound. If each exceeds the threshold, the camera is aimed again. The sound need not be completely continuous so that short pauses (in terms of breathing) are not reacted as breaking continuous speech.
[0009]
The system determines the current sound source, such as a speaker, from a group of sound sources that allows control of devices such as cameras and lights. The system imposes two criteria before acquiring a new source: first, the source is substantially continuous at predetermined intervals, and second, the current Should have substantially different position coordinates (angles with respect to the device to be aimed). In a video conferencing environment, change from the old speaker to the new speaker only if the angular distance between the new speaker and the old speaker exceeds the threshold and only if the new speaker talks continuously for a minimum amount of time become.
[0010]
The approach would be implemented using a sound transducer as described in WO-A-99 / 27522 (Patent Attorney's Procedures PHN 16.638). The above sound transducer produces a signal of the sound output level (RMS or any other suitable indicator) and coordinates indicative of the coordinates of the current source of the sound. Control to achieve the above functionality will be obtained by the following procedure:
If the current coordinates of the sound source are produced by a sound transducer that differs by a minimum angle threshold from the coordinates representing the current target, confirm the current coordinates as candidates for target acquisition .
[0011]
Acquiring a new target if the current temporal coordinates remain within a given range at a particular interval (ie, do not change above a threshold considered to constitute a constant sound source); Save the coordinates of the candidate as coordinates representing the current target.
[0012]
In order to ensure that speech with short interruptions, for example, the accompanying occasional breathing and resting may be used for continuous speech, low pass filters, time latches or other filters applied and used on sound power level signals -Not responded as a gap in the mechanism.
[0013]
Thereafter, to prepare for the time threshold requirement, the time intervals at which the candidate speakers continuously speak (as filtered by a low-pass filter) are accumulated and continuously compared to a predetermined time threshold. Once the timer reaches the time threshold (within tolerance) without continuous speech breaks or certain coordinate changes in direction, the candidate speaker becomes the current speaker, and thus the camera or light device is aimed. You.
[0014]
In an alternative application, the invention may be used as a light, such as a spotlight, aiming at a sound source, such as a speaker or actor.
[0015]
Detailed Description of the Drawings Referring to Figures 1 and 2, a sound transducer 1 receives an audio signal from a speaker 6B and produces an output indicating both the sound output level (RMS or the like) and direction. . The sound output level signal is applied to a low pass filter 19 to remove temporary short low levels. The filtered sound output level signal is then a single bit (with time) indicating whether the sound output of the audio source (speaker 6B) is higher or lower than a particular level at a given instant. ) Applied to a threshold detector 20 (ie, a comparator) to produce a signal.
[0016]
The output of the threshold detector 20 is applied to an angle computer 21 which receives the direction signal from the sound transducer 1. The angle computer outputs a vector indicating the direction of the current source of the sound and applies the vector to the position controller 22. The output of the threshold detector 20 is multiplied by the angle calculated from the direction signal, and if the threshold is low and is not a non-invalid direction vector, or the output of the angle computer 21 is an invalid vector.
[0017]
The position controller 22 continuously executes processing for controlling the position accelerator 23 for pointing the camera 4. The result of the processing performed by the position controller 22 is a continuous acquisition of a designed source of sound. If the speaker 6B talks continuously during the quiet period, while the speaker 6A continues, the camera 4 will be pointed at the latter 6A until another speaker takes over.
[0018]
Note that if there are otherwise brief pauses in the continuous speech, the low pass filter is a global process due to avoiding signal changes. The low pass filter is intended as a general description and can be digital processing on digital or analog signals. It can be a numerical low pass filter or time delay processing, or a device (ie, a latch with a timer) that skips short pauses or other appropriate processing. Similarly, threshold detector 20. The latter are analog devices or digital processing. The functions of the low pass filter, threshold detector, and angle computers 19-21 are performed by a single processor unit that performs all of the above processing with a single algorithm or multiple threads. That is, the above description is intended to be purely functional and not intended to be a fractionation process into individual hardware or software components.
[0019]
Referring to FIG. 3, the control algorithm that will be used to implement the present invention begins with the receipt of a new audio signal (V). Once the audio signal (V) goes to non-zero at step 9, there is a 9-9 loop and control passing to step 10. The position of the angle of the new audio signal is stored in the current speaker (S) memory 10. The camera is then aimed according to the current speaker (S) in step 11. The angular difference between the current speaker (S) vector and the instantaneous audio source (V) vector (output of the angle computer 21) is calculated in step 12, if the instantaneous audio source (V) is non-zero, If the difference exceeds the threshold (| VS | <Ta), control passes to 10 to stop. If not, control flow waits in loop 12-12. In step 10, the instantaneous audio source (V) vector is stored in the memory of the corresponding audio source candidate. The timer is then reset and starts at step 14. Then the control flow loop:
1. The instantaneous audio source (V) vector is zero (step 15);
2. Instantaneous sound source (V) of the vector and the candidate (C) vector _{(| V-S |> T} A) falls below the threshold value _{T K} (step 15); or 3. The timer expires (step 16).
In the case of the escape is carried out.
[0020]
If the timer has expired, control passes to step 17 where the candidate (C) vector is stored in the memory corresponding to the current speaker (S) and the camera is again aimed at step 17. If none of the situations in step 15 match, the loop exits to step 12.
[0021]
The particular embodiment shown and described above has proven useful in determining speaker changes for videoconferencing purposes using a pan-tilt-zoom camera and the invention disclosed herein. Further modifications of the invention will occur to those skilled in the art to which the invention pertains, and all such modifications are considered to be within the scope of the invention as defined by the appended claims.
[0022]
In the claims, any reference sign location between parentheses shall not be construed as a limitation on the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "one" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of a number of discrete components and by means of a suitably programmed computer. In the device claim enumerating several methods, several of these methods can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
[Brief description of the drawings]
FIG. 1 is a metaphorical representation of an application of one embodiment of the present invention.
FIG. 2 is a functional block diagram of one embodiment of the present invention.
FIG. 3 is a flowchart describing a control algorithm that may be used to implement the embodiment in FIG.

Claims (16)

A method for targeting an audio source, the method comprising:
Storing a first coordinate of the current audio source as determined by measuring sound from the current audio source;
Storing the angle threshold in memory,
Calculating the angle difference between the first and second coordinates of the previous sound source stored in memory and targeting the current sound source in response to the angle difference and the angle threshold. A method comprising: The method of claim 1, wherein the targeting step comprises comparing the angle threshold with the angle difference. The method of claim 1, wherein the targeting step includes the current audio source to be targeted only if the angle difference exceeds the angle threshold. The method of claim 1, wherein the targeting step includes the current audio source targeting in response to a sound output level of the current audio source. The method of claim 1, wherein the targeting step comprises comparing a sound output of the current audio source to a predetermined sound output threshold. The method of claim 1, wherein the targeting step comprises a low-pass filtered sound output signal corresponding to the current audio source. The step of filtering in the low band,
Receiving the input signal of the sound output level,
Resetting and starting a timer when the input signal is below a threshold level;
Before the input signal exceeds the threshold level, if the timer has reached a time-out state, the output of the first output signal and from the threshold level before the timer reaches the time-out state. 7. The method of claim 6, including maintaining the input signal low and the output of the second output signal. The method of claim 1, wherein the targeting step includes an aiming device. A method for targeting an audio source, the method comprising:
Saving the current speaker coordinates in the current speaker memory,
Detecting the coordinates of the first audio source;
Determining the angular difference between the stored coordinates of the current speaker and the coordinates of the first audio source, and the first audio source responsive to the angular difference. A method for targeting an audio source, the method comprising:
Detection of the current audio source,
Determining an angular difference between the stored coordinates of the audio source and the coordinates of the current audio source;
A method comprising: measuring a duration of maintenance of the current audio source sound output level; and targeting the audio source in response to the duration and the angle difference. A method for targeting an audio source, the method comprising:
Detection of the current audio source,
Determining an angular difference between the stored coordinates of the audio source and the stored coordinates of the current audio source;
Measuring the time integral of the duration of the power level of the sound of the current audio source and targeting the audio source in response to the time integral of the duration and the angle difference exceeding a predetermined angle threshold. A method comprising: A targeting system,
A sound converter producing first and second signals respectively representing the coordinates of the sound source and the output level of the sound, and a controller connected to receive the first and second signals, It is further programmed to calculate the angle difference between the coordinates of the sound signal and the stored coordinates of the previous sound source and is further connected to the control of the position of the device aiming in response to said angle difference. A targeting system comprising: a controller characterized by being performed. A low-pass filter connected to receive the sound output level of the sound source, wherein the low-pass filter is programmed to filter the sound output level of the sound source and received by the controller. 13. The targeting system of claim 12, wherein the system outputs a filtered sound output level signal. A targeting system,
A sound detector using at least two microphones arranged to provide a first signal representing the position of the sound source and a second signal representing the output level of the sound of the sound source;
A controller device connected to receive the first and second signals, the controller calculating an angular difference between the coordinates of the sound source and the stored coordinates of a previous sound source;
Programmed to compare the angle difference to an angle threshold and to control a targeting device connected to the controller device to aim in response to the angle difference exceeding the angle threshold. And targeting system. 15. The controller of claim 14, wherein the controller device includes a low-pass filter programmed to receive the sound output level of the sound source, and outputs a filtered sound output level of the sound source. Targeting system. The controller device includes a threshold detector connected to receive the sound output level of the sound source, the threshold detector being programmed to compare the sound output level of the sound source to a sound threshold. 15. The targeting of claim 14, whereby the targeting device is aimed in response to the output of the sound exceeding the predetermined threshold and the angle difference exceeding the angle threshold. system.

Images