JP2822897B2 - Videoconferencing system speaker identification device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video conferencing system for realizing a teleconference by connecting a plurality of points, and more particularly to a video conference effective for discriminating a speaker when a videoconference is connected between multiple points. The present invention relates to a system speaker identification device.

[0002]

2. Description of the Related Art Many companies and the like have adopted a video conference system for performing a video conference by connecting video conference terminals at a plurality of locations using a public line or a dedicated line. When such a video conference is held at multiple points, it is convenient to display only the speaking speaker on a television or a larger screen than those at other points. Therefore, conventionally, the signals sent from the voice lines of the speakers divided into these multiple points are analyzed, the speakers are identified, and only the speakers are distinguished from other conference participants, as described above. Highlighting is performed.

In discriminating a speaker from each television participant, a sound level is measured for each television terminal as disclosed in, for example, Japanese Patent Laid-Open No. 4-150590, and a background level is determined from this. 2. Description of the Related Art Conventionally, it has been conventionally performed to obtain a sound level by excluding a sound level, and compare the obtained sound level for each television terminal to determine one speaker having the highest level. Then, the determined one speaker is emphasized on the television screen.

However, when the conference participant having the highest audio level among the television terminals is simply selected as the speaker and the speaker to be emphasized is switched each time, for example, one of the conference participants is emphasized. When a person coughs, an inconvenience such as a change in the display on the screen occurs.

In Japanese Patent Laid-Open Publication No. 2-4095, a sound and a silence are detected for each voice input, and when a sound is detected, the sound is not immediately determined to be a speaker, but when the sound exceeds a predetermined time. Is determined to be the speaker. In addition, a person once determined to be a speaker is not determined to be a non-speaker immediately after detecting silence, but is determined to be a non-speaker only when the number of silences exceeds a certain value.

[0006]

As described above, in the conventional video conference system speaker discriminating apparatus, the speaker is discriminated by comparing the level of the voice level. When there is a situation where the speech continues, there is a problem that if there is a difference in voice level between these persons, only one speaker is determined as a speaker, and only that person is emphasized on the television screen. . In addition, if the voice level changes each time the two persons make a speech, and the difference between the two voice levels changes or the larger one changes, the speaker cannot be accurately identified. Of course, in such a case where there are two speakers, the speaker on the side determined to be a non-speaker is not emphasized, and depending on the mode of emphasis, that person is not displayed on the TV screen at all. Such a situation also occurred.

Further, when a certain time interval is set as in Japanese Patent Application Laid-Open No. 2-4095 to newly discriminate a speaker, the person who has been discriminated as a speaker continues to speak. Sometimes, when someone speaks in a way that interrupts them, or when there are multiple speakers in addition to the person who has been identified as the speaker, the new speaker can be accurately identified. There was a problem that it could not be done.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a video conference system speaker discriminating apparatus capable of accurately discriminating each speaker when a speaker is interrupted or a plurality of speakers exist.

Another object of the present invention is to provide a video conference system in which when two speakers are speaking, the state of the conversation is displayed on a television screen so that a video conference with a sense of reality can be performed. A speaker identification device is provided.

[0010]

According to the first aspect of the present invention, (a) audio level detection means for detecting the audio level of each video conference terminal participating in a video conference for each terminal;
(B) speech state detection means for detecting whether or not a speaker is speaking at each video conference terminal in a predetermined cycle unit;
(C) speaker discriminating means for discriminating a speaker at each cycle based on the sound level detected by the sound level detecting means, and (d) detection results of the speech state detecting means and the speaker discriminating means. discrimination result and speaker Substitution situation determining means for determining replacement status of the speaker using the cycle described above in detecting change of the talker is the speaker alternation condition determination means (e) current
Is set to be shorter than the value and the cycle changing means for making this cycle longer than the current value when the section up to the replacement is long is provided in the video conference system speaker identification device.

That is, according to the first aspect of the present invention, when a single speaker has been speaking for a long time, or when another speaker has started speaking in a manner interrupting, etc., the change state of the speaker is determined. ,
By changing the cycle for discriminating the speaker in response to this, it is possible to prevent false recognition of the alternation of the speaker for the unnecessary voice, while preventing the speaker from interrupting or multiple speakers. When present, each speaker can be accurately determined.

According to the second aspect of the present invention, (a) audio level detecting means for detecting the audio level of each video conference terminal participating in the video conference for each terminal, and (b) the audio level detected by the audio level detecting means. Voice level comparing means for comparing the largest voice level with the next highest voice level for each terminal; and (c) when the difference between the voice levels compared by the voice level comparing means is within a predetermined range, the two parties interact with each other. (D) display emphasizing means for emphasizing, on a screen display, two speakers who have determined that the dialogue is being performed as compared with other video conference participants Are provided in the video conference system speaker identification device.

That is, according to the second aspect of the present invention, it is determined whether or not there are two speakers who are speaking by extracting the speech level of the largest speech level and the speech level of the next speech level and comparing the difference between these levels. I decided to do it. Then, when there are two speakers, these persons are highlighted on the screen to enhance the realism of the dialogue.

According to the third aspect of the present invention, the speech state of the speaker is detected to change the cycle as a unit for switching the screen display. We decided to include our strengths.

[0015]

[0016]

[0017]

[0018]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to embodiments.

First Embodiment

FIG. 1 shows a first embodiment of the present invention.
Videoconferencing system using a Levi conference system speaker identification device
This is an outline of the stem. Tokyo headquarters and famous places
Stores located at various locations, such as stores or branch offices in Osaka.
Levi conference terminal 11₁, 11_Two, ... 11_NIs a video line
Line 12 such as a voice line₁, 12_Two, ... 12_NThrough
Interface of the TV conference system speaker identification device
Connected to the network 13. From the communication interface 13
Is each video conference terminal 11₁, 11_Two, ... 11 _NPer
Audio information 14₁, 14_Two, ... 14_NIs taken out and
The corresponding audio level measuring units 15₁, 15_Two, ……
Fifteen_NIs input to Here, audio information 14₁, 14_Two,
...... 14_NIs each video conference terminal 11₁, 11_Two…… 1
1_NOf the sound detected by the microphone (not shown)
Yes, including audio and other background sounds
I have.

Voice level measuring units 15 ₁ , 15 ₂ ,..., 1
5 _N receives the measurement period control signal 17 sent from the sound level measurement period controller 16, the audio information 14 ₁ of the corresponding teleconference terminal, 14 _2, which measurement period to analyze ...... 14 _N Increase or decrease by The _first audio level measurement data 18 these results, 18 _2, ... as 18 _N, the video conference terminal 11 _1, 11 _2, ... 11 the sound level detector 19 _1, which are arranged corresponding to _N, 19 ₂ ,
... 19 _N and the background level detector 21 ₁ , 2
1 ₂ ,..., 21 _N and utterance state detection sections 22 ₁ , 2
2 _2, supplied to ...... 22 _N. Background level detection unit 21 _1, 21 _2, ...... 21 _N detects the background level of the sounds other than speech, the background level measurement data 23 _1, 23 _2, the sound level detecting unit ...... 23 _N 19 _1, 19 _2, he said ...... 19 _N state detection unit 22 _1, 22 _2, and supplies the ...... 22 _N.

Audio level detectors 19 ₁ , 19 ₂ ,..., 1
9 _N is the input audio level measurement data 18 ₁ , 18
₂ , 18 _N and background level measurement data 2
3 _1, 23 _2, and detects the difference of the measured data of each pair of ...... 23 _N, detects the audio level influence the background level has been eliminated. The audio level data 25 ₁ , 25 ₂ ,..., 25 _N representing these audio levels are
It is sent to a speaker discriminating section 26 for discriminating the speaker. The method for determining the speaker will be described later.

The speaker discrimination result is speaker discrimination result data 27
Is output as The speaker discrimination result data 27 is sent to the image processing unit 28, and image processing for emphasizing one discriminated speaker is performed. The video signal 29 after this image processing
Means that each video conference terminal 1
1 _1, 11 _2, sent to ...... 11 _N, so that the image is displayed on each of the television screen.

On the other hand, the speech state detecting units 22 ₁ , 22 ₂ ,.
... 22 _N is the input audio level measurement data 18 ₁ ,
18 _2, ...... 18 _N and background level measurement data 23 _1, 23 _2, using ...... 23 _N, measured the presence of speech states of the individual television conference terminal 11 _1, 11 _2, ...... 11 _N cycles Detect every time. Specifically, the rate of change of each of the audio level measurement data 18 ₁ , 18 ₂ ,... 18 _N is measured, and the individual video conference terminals 11 ₁ , 11 ₂ ,.
The presence or absence of a speech is detected at each cycle by detecting the rise and fall of the audio level for each 11 _N and determining the start and end states of the speech.

Here, each sound level measurement data 18 ₁ ,
18 ₂ ,..., 18 _N , when no change is observed, the sound level measurement data 18 ₁ , 18 ₂ ,.
8 _N and background level measurement data 23 ₁ , 23
₂ ,... _23N are compared, and the TV conference terminal 11 having a level difference between them in one cycle is determined to be a speech state, and if there is no level difference, it is determined to be a state where no speech is made. .

When one TV conference terminal 11 detects the rise and fall of a plurality of speech states in this cycle, it is determined whether the integrated value of the time during speech in one cycle exceeds a predetermined threshold. Make a determination. As a result, when this threshold is exceeded, it is determined that there was a utterance in that cycle. If it does not exceed the threshold, it may be determined as non-utterance, but in consideration of the case where the utterance section straddles the break of the cycle, when it is in the utterance state at the timing of the cycle change, the cycle Is determined to have made a comment.

In this way, each of the utterance state detectors 22 ₁ ,
The speech result data 31 ₁ , 31 ₂ ,... 31 _N for each cycle detected by 22 ₂ ,..., 22 _N are commonly input to the audio level measurement cycle control unit 16. On the other hand, the speaker determination unit 26 detects one or a plurality of teleconference terminals 11 that are speaking using the speech result data 31 ₁ , 31 ₂ ,..., 31 _N obtained from the voice level measurement cycle control unit 16.
Next, the voice level data 25 is compared between the teleconferencing terminals 11 that are speaking, and the largest one is determined as the speaker. As described above, the speaker determination result is output as the speaker determination result data 27.

The audio level measurement cycle control unit 16
The speaker discrimination data is acquired from the speaker discrimination unit 26 and stored, and it is discriminated on a cycle-by-cycle basis whether the speaker is the same or changes. As a result, (a) when it is determined that the speaker changes, control is performed by the measurement cycle control signal 17 so as to shorten the measurement cycle of the voice level by a predetermined time from the current value. The history may be made somewhat longer, the frequency at which the speaker changes may be calculated, and a value for shortening the cycle may be controlled using this frequency information. In addition, ROM (read only
The values for such addition and subtraction may be stored as a table in the memory), and the interval of one cycle may be set while reading the addition and subtraction values according to the situation.

(B) When it is determined that the speaker is constant, it is determined whether or not there is any speaker other than the speaker. This determination is performed by calculating the number of data at the start and during the utterance from the remaining utterance result data from which the utterance result data determined to be the speaker is deleted, and determining whether there is a person other than the speaker who is speaking. Done by As a result, when it is determined that “there is a speaker in addition to the speaker”, a measurement cycle control signal 17 for shortening the measurement cycle of the audio level is sent to each of the audio level measurement units 15 ₁ , 15 ₂ , 15. ......
15 _N. At this time, the larger the number of data, that is, the number of speakers, the shorter the measurement cycle.

[0030] On the contrary, if it is determined that "other is not speaker to speaker" is measured period control signal 17 to the sound level measuring unit 15 ₁ as a longer measurement period of the speech level , 15 ₂ ,..., 15 _N.
This state means that the same speaker continues as a speaker. Therefore, the speaker discrimination result data 27
, The time as a speaker is determined, and the longer the time, the longer the measurement cycle. Note that the measurement cycle is set using one section time during which one speaker is scheduled to speak.

In this manner, each video conference terminal 1
1 ₁ , 1 _{1 2} ... 1 _{1 N} identified as the speaker in _N
A person who speaks is highlighted on the screen, and even when a change of speakers occurs, the speaker whose screen is highlighted is accurately replaced. As a method of emphasizing a speaker, a method of displaying only a person who is determined to be a speaker on the entire screen, or a method in which a small screen of all speakers is set and a screen of a person who is determined to be a speaker is displayed by another screen. It is possible to use various known methods such as a method of enlarging the screen larger than the screen of the other person and a method of changing the frame of the screen of the person determined to be the speaker to a color different from the frame of the screen of another person. it can.

Second Embodiment

FIG. 2 shows an outline of a video conference system using a video conference system speaker identification device according to a second embodiment of the present invention. The same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate. From the communication interface 13, each video conference terminal 1
1 _1, 11 _2, audio information 14 for each ...... 11 _{N 1,} 14
_2, ...... 14 _N is extracted, the speech level measuring unit 61 corresponding _1, 61 _2, are input to the ...... 61 _N.
Here the voice information 14 _1, 14 _2, ...... 14 _N is the information of each television conference terminal 11 _1, 11 _2, and detected by the microphone (not shown) of the ...... 11 _N sound, voice and other background Also includes sound.

Audio level measuring section 61₁, 61_Two…… 6
1_NMeasured sound level data 18₁, 18_Two,
... 18_NIs a teleconference terminal 11₁, 11_Two, ……
11 _NAudio level detector 19 arranged corresponding to₁,
19_Two............ 19_NAnd background level detector 2
1₁, 21_Two, ... 21_NAnd speech state detection unit 2
2 ₁, 22_Two, ... 22_NSupplied to Audio level detection
Departure 19₁, 19_Two............ 19_NAre each entered
Audio level measurement data 18₁, 18_Two, ... 18 _NWhen
Background level measurement data 23₁, 23_Two…
… 23_NTo remove each of the background levels
Audio level data 2 representing the removed audio level
5₁, 25_Two, ... 25_NCreate and interact with these
It is supplied to the detection unit 62 and the speaker determination unit 63. Speaker discriminator 6
3 is the dialogue state data 64 detected by the dialogue state detector 62
And each audio level data 25₁, 25_Two, ... 25_NFor
To determine one or more speakers and determine the result
The data is sent to the image processing unit 66 as data 65. Image processing
The unit 66 emphasizes one or more determined speakers.
For image processing. The video signal 67 after this image processing
Means that each video conference terminal 1
1₁, 11_Two, ... 11_NSent to each tele
An image is displayed on the screen.

In such a video conference system speaker discriminating apparatus, the dialogue state detecting section 62 outputs each voice level data 2
5 _1, 25 _2, enter the ...... 25 _N, sound level measurement unit 61 _1, 61 _2, each of the integral of the sound level at each measuring period ...... 61 _N to measure the television conference terminal 1
1 _1, 11 _2, is calculated for each ...... 11 _N. Then, two types, ie, the maximum audio level as the integrated value and the next audio level, are extracted.

However, depending on the device, when there are two or more sound levels which are close to the maximum sound level to some extent, these may be extracted together. This is because, if the measurement cycle is set relatively long, a phenomenon may occur in which three or more persons have a quick conversation during one cycle. In such a case, the measurement period control signal 17 is transmitted from the dialogue state detection unit 62 to each of the audio level measurement units 61 ₁ , 61 ₂ ,..., 6 in the same manner as in the first embodiment.
It is also effective to adopt a configuration for transmitting to 1 _N and control the measurement cycle. In the present embodiment, in order to simplify the explanation, the explanation is limited to the dialogue between two parties.

The dialogue state detecting section 62 compares the integrated values of the voice levels of the two parties, and when these are small differences within a predetermined value range, it is determined that the currently held video conference is in a dialogue state. Is determined. In other cases, that is, when there is a great difference, it is determined that the currently held video conference is in a non-interactive state. Further, the dialogue state detection unit 62 similarly determines that the user is in the non-conversational state when there is only one speaker or when there is no speaker.

Dialogue state data 6 indicating the presence or absence of a dialogue state
4 is each audio level data 25 ₁ , 25 ₂ ,..., 25 _N
Is input to the speaker discriminating section 63 together with the information. When the speaker determination unit 63 dialog state data 64 indicates a non-interactive state, each of the sound level data _{25 1, 25 2, ...... 25} N
Are compared, and the one of the video conference terminal 11 representing the maximum audio level is determined as the “speaker A pattern”.

When the conversation state data 64 indicates the conversation state, the one of the video conference terminal 11 similarly indicating the maximum voice level is determined as the "speaker A pattern", and is in the conversation state. The one of the video conference terminal 11 representing the next highest voice level as another speaker is determined as a “speaker B pattern”.
These determination results are sent to the image processing unit 66.

The image processing unit 66 includes a CPU (Central Processing Unit) not shown, and a ROM (not shown)
The control program stored in the (read only memory) performs an emphasis process on the screen in accordance with the presence or absence of a dialogue state and the presence or absence of a speaker.

FIG. 3 shows how the image processing unit is controlled. The image processing unit 66 includes speaker identification data 65
And checks whether a speaker is present (step S101). When the "speaker A pattern" exists, a speaker exists. If no speaker is present (N), video processing is performed to display the video of all participants in the video conference in small screens (step S102).
3, each of the video conference terminals 11 ₁ , 11 ₂ ,.
Send to _N.

If there is a speaker and the "speaker B pattern" exists (step S103; Y), the screen is set to 2
A video signal 67 for displaying both the speaker A and the speaker B is created by division (step S104), and this is transmitted to the communication interface 13. If the speaker B does not exist (step S103; N), a video signal 67 that displays only the speaker A on the entire screen is created (step S105) and transmitted to the communication interface 13. become.

The above screen display mode is checked and switched for each measurement cycle. It should be noted that there may be various other emphasis modes. For example, the screen of all participants is displayed in a small frame, and only the person or speaker who is interacting is displayed in a frame of a different color from the others,
These persons 'frames may be displayed relatively larger than other persons' frames.

Although the first and second embodiments have been described above, the present invention is not limited to these embodiments. For example, it is also possible to incorporate a voice level measurement cycle control unit for changing the measurement cycle of the first embodiment into the second embodiment to realize a video conference system speaker discriminating apparatus on the premise of dialogue. is there.

[0045]

As described above, according to the first aspect of the present invention, when a speaker speaks for a long time alone, or when another speaker starts speaking in a manner of interrupting, etc. By changing the cycle for determining the speaker in response to the change of the speaker, it is possible to prevent the speaker from erroneously recognizing the change of the unnecessary voice, When there is an interrupt or a plurality of speakers exist, each speaker can be accurately determined.

According to the second aspect of the present invention, it is determined whether there are two speakers who are speaking by extracting the speech level of the largest speech level and the speech level of the next speech level and comparing the difference between these levels. Since the discrimination is made, when there are two speakers, by highlighting them on the screen, the sense of reality of the dialogue can be enhanced.

Further, according to the third aspect of the present invention, since the speech state of the speaker is detected and the cycle as a unit for switching the screen display is changed, the speech is displayed even when the image is displayed in an interactive manner. The person can be appropriately specified.

[0048]

[0049]

[Brief description of the drawings]

FIG. 1 is a block diagram showing a video conference system speaker identification device according to a first embodiment of the present invention and each video conference terminal connected thereto.

FIG. 2 is a block diagram showing a video conference system speaker identification device according to a second embodiment of the present invention and each video conference terminal connected thereto;

FIG. 3 is a flowchart illustrating a state of control of an image processing unit according to a second embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 11 Video conference terminal 15, 61 Audio level measurement unit 16 Audio level measurement cycle control unit 19 Audio level detection unit 21 Background level detection unit 22 Speaking state detection unit 25 Audio level data 26, 63 Speaker discrimination unit 28, 66 Image processing Unit 62 Dialogue state detection unit

Claims (3)

(57) [Claims] 1. An audio level detecting means for detecting an audio level of each video conference terminal participating in a video conference for each terminal, and detecting whether a speaker is speaking at each of the video conference terminals in a predetermined cycle unit. Utterance state detecting means, speaker deciding means for discriminating a speaker at each cycle based on the sound level detected by the sound level detecting means, detection results of the utterance state detecting means and speaker discrimination A speaker change situation determining means for determining a change situation of the speaker by using the determination result of the means; and when the speaker change situation determining means detects the change of the speaker, the period is set shorter than a current value. A video conference system speaker discriminating device comprising: a period variable means for setting the period to be longer than a current value when a section from setting to replacement is long. 2. An audio level detecting means for detecting an audio level of each video conference terminal participating in a video conference for each terminal; a maximum audio level for each terminal detected by the audio level detecting means; Voice level comparing means for comparing loud ones; dialog determining means for determining that the two are engaged in dialogue when the difference between the voice levels compared by the voice level comparing means is within a predetermined range; A video conferencing system comprising: display emphasis means for emphasizing, on a screen display, two speakers who have determined that they are interacting with each other compared with other video conference participants. 3. The video conference system speaker discriminating apparatus according to claim 2, wherein a speech state of the speaker is detected and a cycle as a unit for switching screen display is varied.