KR20020033829A - Aiming a device at a sound source - Google Patents

Abstract

The system targets the device to be targeted in response to the sound flowing from several possible sound sources. For example, the system can be used to target light at the video camera 4 or the current speaker 6B, such as in a meeting or discussion. The control system responds to the duration of continuous sound emission from the new sound source 6A and the angular difference between the current target 6B and the new sound source 6A. If each exceeds the threshold, the camera 4 will be retargeted. The sound does not have to be perfectly continuous so that short pauses (such as during breathing) do not respond to a break in the continuous voice.

Description

Aiming the device to a sound source {AIMING A DEVICE AT A SOUND SOURCE}

Automatic camera tracking systems automatically capture and maintain targets within their field of view based on certain criteria such as motion, color, image processing, and the like. For example, a video conferencing system can use a pan-tilt-zoom camera to track the desired target. One class of such devices automatically determines the current speaker within a group and keeps track of the current speaker, focusing on the current speaker. Each time a new speaker begins to speak, the new speaker is automatically detected by sound level and / or other criteria and focused on the new speaker.

To date, a number of different techniques for determining the current speaker have been proposed. Such a technique allows an automatic camera to determine the presence of a new speaker and then focus on the new speaker, which has a significant disadvantage. These systems change very slowly from speaker to speaker and miss many parts of the conversation, or change too quickly to confuse camera tracking.

One solution to this problem is proposed in US Pat. No. 5,206,721. The patent describes how each speaker is aimed at the current speaker in a system with a separate microphone. In the system, the input signal is collected from each of several microphones, one for each speaker. Total talking time is accumulated for each speaker during successive sections. At any given time in time, the current speaker is determined to be the person with the most accumulated time of speaking.

The problem with this approach is that it requires the use of multiple microphones for current speaker determination. The use of multiple microphones usually makes the distinction between current speakers difficult because of microphones in close proximity to each other, and also due to echoes and external noise in confined spaces or spaces with partitions and objects. . In addition, the disclosed method does not effectively prevent the undesirable effects of continuous camera operation between two speakers in close proximity to one another.

Therefore, in automatic video conferencing art, there is a need for a current speaker determination method that an automatic camera will focus on, overcoming the above drawbacks of prior art techniques.

The present invention generally relates to the field of automatic targeting systems, such as video tracking systems used in video conferencing systems.

1 is a figurative representation of the application of one embodiment of the present invention.

2 is a functional block diagram of one embodiment of the present invention.

3 is a flow diagram illustrating a control algorithm that may be used to implement the embodiment of FIG.

The invention is defined in the independent claims, while the dependent claims define advantageous embodiments.

In short, the system targets the device to be targeted in response to sound flowing from several possible sound sources. For example, the system can be used to target a video camera or light to the current speaker, such as in a meeting or discussion. The control system responds to the duration of continuous sound emission from the new sound source and the angle difference between the current target and the new sound source. If each exceeds the threshold, the camera will be aimed again. The sound does not have to be perfectly continuous so that short shocks (such as when breathing) do not react as interruptions in the continuous voice.

The system determines the current audio source, such as the speaker, from a group of sound sources, allowing control of devices such as cameras and lights. The system imposes two criteria before capturing a new sound source, firstly that the sound source must be substantially continuous for a predetermined interval, and secondly that the sound source is substantially equal to the current coordinate of the side the device is aimed at. In other words, they must have different positional coordinates (angle with respect to the device to be targeted). In a video conferencing situation, this will change from the previous speaker to the new speaker only when the angular interval between the new speaker and the previous speaker exceeds the threshold and only when the new speaker speaks continuously for a minimum period of time.

This approach can be implemented using a sound transducer, such as that described in WO-A-99 / 27522 (Agent No. PHN 16.638). The transducer generates a coordinate signal indicative of the sound power level (RMS or any other suitable indicator) and the coordinates of the current sound source. Control to achieve this function can be obtained by the following procedure:

If the current coordinates of the sound source generated by the sound transducer differ from the coordinates representing the current target by a minimum angle threshold, identifying the current coordinates as candidates for target capture.

If the current instantaneous coordinates remain within a given range for a specified period (i.e. they do not change beyond the threshold considered to constitute a constant sound source), then the coordinates representing the current target are captured and Save the coordinates of the candidate.

Low pass filters, time-latch, to ensure that intermittent breathing and voice with pauses do not react as a gap in the continuous voice. Alternatively, other filter mechanisms can be used to apply the sound power level signal.

In order to take into account the time threshold requirements, the time periods during which the candidate speaker is continuously speaking (as filtered by the low pass filter) are then accumulated and constantly compared with the previously missing time threshold. Once the timer reaches the time threshold (within tolerance), without a break in successive speech or a change in direction coordinates, the candidate speaker becomes the current speaker, Devices such as cameras and lights are aimed accordingly.

In another application, the present invention can be used to target lighting, such as spot lights, to sound sources such as speakers or actors.

Referring now to FIGS. 1 and 2, the sound converter 1 receives a signal from the speaker 6B to produce an output indicating both sound power level (RMS or the like) and direction. The sound power level signal is applied to a low pass filter 19 to remove transient and short low levels. The filtered sound power level signal is then applied to threshold detector 20 (e.g., comparator) to indicate whether the sound power of the audio source {speaker 6B} is greater or less than a certain level at any given time. Generates a single bit signal (for time).

The output of the threshold detector 20 is applied to an angle calculator 21, which also receives a direction signal from the sound transducer 1. The angle calculator outputs a vector indicating the direction of the current sound source and applies it to the position controller 22. Since the output of the threshold detector 20 is multiplied by the angle calculated from the direction signal, if the threshold is low the output of the angle calculator 21 is a null vector, and if the threshold is not low, it is non-zero. Direction vector.

The position controller 22 constantly performs the process of aiming the camera 4 by controlling the position actuator 23. The result of the process performed by the position controller 22 is a subsequent acquisition for the specified sound source. If the speaker 6B is silent for a period of time, and the speaker 6A continues to speak for the next period, the camera 4 will be aimed at the latter speaker 6A until another speaker takes over.

The low pass filter 19 is a continuous discourse, but on the one hand it is a general procedure to avoid signal changes when there is a brief delay. The low pass filter is intended to be a general description and may be a digital process on a digital or analog signal. The low pass filter may be a numerical low pass filter or a short short skipping time-delay process or device (eg, a latch with a timer) or any other suitable process. Similar to the threshold detector 20. The latter can be an analog device or a digital process. The functions of the low pass filter, threshold detector, and angle calculator 19-21 may be performed by a single processor device that performs all of the above in a single algorithm or multiple threads. That is, the description is intended to be purely functional and is not intended to divide the processes into separate hardware or even software components.

Referring now to FIG. 3, a control algorithm that can be used to implement the present invention begins with the reception of a new audio signal (V). Once the audio signal V is nonzero in step 9, it exits loop 9-9 and control passes to step 10. The angular position of the new audio signal is stored in the current speaker (S) memory 10. Subsequently, the camera is aimed in accordance with the current speaker S in step 11. The angle difference between the current speaker S vector and the audio source V vector at that moment (the output of the angle calculator 21) is calculated in step 12, if the audio source V at that moment is nonzero. If the difference exceeds the threshold (| SV |> T _A ), control passes to step 13. If the difference does not exceed the threshold, the control flow waits in loop 12-12. In step 13, the audio source V vector at that moment is stored in a memory corresponding to the candidate audio source. The timer is then reset and started at step 14. Subsequently, a control flow loop 15-16 emerges,

1. when the audio source V vector at that moment becomes zero (step 15);

2. When the difference | CV | between the audio source V vector and the candidate C vector at that moment is below the threshold T _K (step 15); or

3. When the timer times out (step 16);

Exit the control flow loop 15-16.

If the timer times out, control passes to step 17 where the candidate C vector is stored in a memory corresponding to the current speaker S and the camera is retargeted at step 17. If neither condition is met in step 15, it exits the loop and goes to step 12.

While the particular embodiments shown and described above have proven useful for determining speaker switching for the purpose of video conferencing using a pan-tilt-zoom camera, further modifications of the invention disclosed herein are intended to encompass the present invention. As those skilled in the art will appreciate, all such changes are considered to be within the scope of the invention as defined by the appended claims.

In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those described in the claims. The singular expression preceding the element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied as one and the same item of hardware. The simple fact that specific measures are described in different mutually dependent claims does not indicate that a combination of these measures cannot be used to advantage.

As mentioned above, the present invention is generally used in the field of automatic targeting systems, such as video tracking systems used in video conferencing systems.

Claims (16)

As a method of targeting the audio source 6B, the method Storing first coordinates of the current audio source determined by sound measurements from the current audio source; Storing the angular threshold in a memory; Calculating an angle difference between the first and second coordinates of a previous audio source stored in a memory; Targeting the current audio source in response to the angle difference and the angle threshold. Including audio source 6B. 2. A method according to claim 1, wherein the step of targeting comprises comparing the angle difference with the angle threshold. 2. A method according to claim 1, wherein the step of targeting comprises targeting the current audio source only if the angle difference exceeds the angle threshold. 2. A method according to claim 1, wherein the step of targeting comprises targeting the current audio source in response to the sound power level of the current audio source. 2. A method according to claim 1, wherein the step of targeting comprises comparing a sound power of the current audio source with a predetermined sound power threshold. 2. The method of claim 1, wherein said targeting comprises low pass filtering a sound power signal in response to said current audio source. The method of claim 6, wherein the low pass filtering comprises: Receiving a sound power level input signal; Resetting and starting a timer when the input signal falls below a threshold level; Outputting a first output signal if the timer reaches a timeout state before the input signal exceeds the threshold level; If the input signal stays below the threshold until the timer reaches the timeout state, outputting a second output signal. Including audio source 6B. The method of claim 1, wherein said targeting comprises targeting the device. As a method of targeting an audio source, the method Storing the current speaker's coordinates in a current speaker memory; Detecting coordinates of the first audio source; Determining an angle difference between the stored coordinates of the current speaker and the coordinates of the first audio source; Targeting the first audio source in response to the angle difference Including an audio source. As a method of targeting an audio source, the method Detecting a current audio source; Determining an angle difference between the stored coordinates of the audio source and the coordinates of the current audio source; Measuring a duration of sound power level maintenance of the current audio source; Targeting the audio source in response to the duration and the angle difference Including an audio source. As a method of targeting an audio source, the method Detecting a current audio source; Determining an angle difference between the stored coordinates of the audio source and the stored coordinates of the current audio source; Measuring a time integral of the duration of the sound power level of the current audio source; Targeting the audio source in response to the time integration of a duration and the angle difference exceeding a predetermined angle threshold. Including an audio source. As a targeting system, A sound transducer (1) for generating first and second signals indicative of the coordinates of the sound source and the sound power level of the sound source, respectively; Is connected to receive the first and second signals, and is programmed to calculate an angular difference between the first sound signal coordinate and a stored coordinate of a previous sound source, and also controls the position of the device in response to the angular difference. Connected controllers 21 and 22 to be targeted. Including, targeting system. 13. The apparatus of claim 12, further comprising a low pass filter (19) coupled to receive the sound power level of the sound source, whereby the low pass filter filters the sound power level of the sound source to the controller. A targeting system, programmed to output a filtered sound power level signal to be received by. As a targeting system, An audio detector (1) using at least two microphones arranged to provide a first signal indicative of the location of the sound source and a second signal indicative of the sound power level of the sound source; Connected to receive the first signal and the second signal, Compute the angle difference between the coordinate of the sound source and the stored coordinate of the previous sound source, compare the angle difference with the angle threshold, and control the target device 23 connected to the controller device to control the angle threshold. To aim in response to the above difference in angle Programmed controllers (21, 22) Including, targeting system. 15. The system of claim 14, wherein the controller device comprises a low pass filter programmed to receive the sound power level of the sound source and output the filtered sound power level of the sound source. 15. The apparatus of claim 14, wherein the controller device comprises a threshold detector coupled to receive the sound power level of the sound source, wherein the threshold detector is programmed to compare the sound power level of the sound source with a sound threshold. And the targeting device is targeted in response to the sound power exceeding the predetermined threshold and the angular difference exceeding the angular threshold.