JP4709734B2 - Speaker selection device, speaker selection method, speaker selection program, and recording medium recording the same - Google Patents

Description

  The present invention relates to a speaker selection technique. More specifically, the present invention relates to a speaker selection technique in voice mixing.

  Multipoints (three or more points are referred to as multipoints, the same shall apply hereinafter) are connected via a packet communication network, which is a digital communication network, and multipoint voice communication (for example, multipoint audio conference) is performed by voice packet communication. Doing more is increasing.

FIG. 1 shows an outline of a configuration example in which voice packets sent from three points A, B, and C are mixed by a voice mixing device and the mixed voice packets are sent to a point D.
The voice packet transmission unit (900A) at the point A collectively refers to the input voice signal [the acoustic signal such as the discretized voice signal or the music signal at the point A. ] Are converted into voice packets A and sent to the voice mixing device (970) via the packet communication network (950). Similarly, the voice packet transmitting unit (900B) at the point B converts the input voice signal at the point B into the voice packet B and sends it to the voice mixing device (970) via the packet communication network (950). The voice packet transmitting unit (900C) converts the input voice signal at the point C into the voice packet C and sends it to the voice mixing device (970) via the packet communication network (950). The voice mixing device (970) creates a voice packet for one point from the voice packet A, voice packet B, and voice packet C, and sends it out to the point D via the packet communication network (950). The voice packet receiving unit (900D) at the point D converts the received voice packet into an output voice signal.
Although FIG. 1 shows an example in which the input is three points and the output is one point for simplicity of explanation, the input may be any number of points. Moreover, in normal multipoint voice communication use, the input point and the output point are the same, and if the input is the points A, B, and C, the output is also required for three points A, B, and C. D may be read as an example of any one of A, B, and C (the same applies hereinafter).

Conventionally, in such multipoint voice communication applications, ITU-T (International Telecommunication Union-Telecommunication Standardization Sector) is used as a voice encoding method. 711 was mostly used.
The reason is that the audio mixing device once decodes the audio code included in the audio packet sent from each point, performs the mixing process at the PCM (Pulse Code Modulation) signal level, and then performs the encoding process again. It is necessary to generate a voice packet for each point. This is because an encoding method other than 711 places a great load on the audio mixing device.

In such a situation, prior to the present invention, the present inventors have used G.D. While using a wideband speech coding method with better sound quality than 711, a multi-point mixing method that does not place a heavy load on the speech mixing device was realized (see Patent Document 1).
2 and 3 show an example of the multipoint mixing method disclosed in Patent Document 1. FIG. However, FIG. 2 shows an example of the functional configuration of the voice packet transmitting unit for one point in FIG. 1. In FIG. 3, only the point A is shown as the transmitting side, and only the point D is shown as the receiving side, and the points B and C are omitted. The processing at points B and C is the same as the processing at point A.

  In the multipoint mixing method disclosed in Patent Document 1, the speech waveform encoding unit (901) converts an input speech signal divided into predetermined time lengths (for example, about 10 ms to 20 ms) called frames into speech codes. And output this. Further, the voice section detection unit (902) outputs a determination result (hereinafter referred to as a VAD flag) indicating whether the voice section is for a voice section or a non-voice section for each frame. Further, the voice information acquisition unit (904) acquires voice information that is an acoustic feature amount of the input voice signal. As the audio information, for example, the correlation value of the power or pitch of the input audio signal can be used. As a method for acquiring speech information, the power may be obtained by calculating the sum of squares of the speech waveform of the input speech signal, or the correlation value of the pitch of the speech waveform of the input speech signal may be obtained. The packet construction unit (903) incorporates the VAD flag and the voice information into the voice packet together with the voice code, and sends it to the packet communication network (950).

In the voice mixing device (970) that has received the voice packet via the packet communication network (950), the packet decomposing unit (971) extracts the voice code, the VAD flag, and the voice information from the voice packet.
Further, the speech code decomposition unit (972) decomposes the speech code into a low frequency code and a high frequency code, the low frequency code is converted to the low frequency mixing unit (973), and the high frequency code is converted to the high frequency switching unit (974). Sent. Here, the high frequency code is one of enhancement layer codes, and the high frequency code may be read as an arbitrary enhancement layer code (the same applies hereinafter).

  The low-frequency mixing unit (973) receives and decodes the low-frequency code from each point, generates mixing speech for each point to be output, converts it to a low-frequency code, and combines this low-frequency code with a voice code combination Part (975).

  The speaker selection unit (976) refers to the VAD flag and voice information from each point, performs automatic determination processing to determine which point is the main speaker, and then inputs the speaker input point. A speaker index (hereinafter referred to as speaker number) is output.

  The high-frequency switching unit (974) receives the high-frequency code from each point, selects one high-frequency code for each point to be output using the speaker number, and sends it to the speech code combining unit (975). .

  The speech code combining unit (975) combines the low frequency code and the high frequency code to generate a speech code, and sends this speech code to the packet configuration unit (977). The packet construction unit (977) generates a voice packet from the received voice code and outputs the voice packet.

In Patent Document 1, as a speaker selection method of the speaker selection unit (976), a high-frequency code (enhancement layer information) of the ground having the highest average power per sample and determined as a speech section is selected. And if the speaker switches too frequently, there is a possibility that an audible noise may be heard or an unnatural reproduction voice, so that the speaker is not used for a certain period of time (for example, 40 ms to 200 ms). It is stated not to switch.
JP 2005-229259 A

  In general human conversation, about 30% to 50% include non-speech intervals. Nevertheless, when the input voice signal on the transmitting side contains human voice as background noise or in the case of two-way communication, the voice at another point is played back from the acoustic speaker, which becomes an echo. If you enter the microphone from the microphone, the background noise or echo may be determined as the input voice signal as the input voice signal even though the point is not actually speaking. Cannot judge. As a result, the speaker selection unit cannot correctly determine which of the plurality of speakers (speaking points) is the speaker (speaking point), and the automatic determination of the main speaker is incorrect, or the voice is erroneously spoken. There is a concern that code switching occurs and the quality of speech and sound after mixing deteriorates.

  In response to such concerns, as a speaker selection method, voice information of each point and a VAD flag indicating whether or not it is determined as a voice interval are used, and a speaker (point) determined as a voice interval is used. In the method of using the point with the highest voice power as the speaker (speaking point), the speaker (speaking point) is accurately selected when the input voice level fluctuates or when the input voice level varies from point to point. It cannot be determined and it cannot always be said that the above concerns can be sufficiently eliminated.

  In addition, as a method for selecting a speaker, when the speaker (speaking point) is not switched for a certain period of time, if the actual speaker (speaking point) changes in a short time, the speaker (speaking point) is accurately determined. Cannot be determined, and it cannot always be said that the above concerns can be sufficiently eliminated.

  Therefore, the present invention provides a speaker selection device, method, program, and recording medium for reducing deterioration in call quality and sound quality after mixing even when the input voice level fluctuates or the input voice level varies from point to point. The purpose is to provide.

In order to solve the above-described problem, in the present invention, a speaker in voice mixing that performs speaker selection for mixing voice packets sent from three or more points (hereinafter referred to as input points). Whether the voice code of one frame (hereinafter referred to as the current frame) included in a voice packet sent from one input point is a voice section or a non-voice section Using a determination result (hereinafter referred to as a VAD flag) indicating which one is used, the continuation state index that is an index indicating the continuation state of speech at the input point is updated, and the past and present sent from one input point are updated. Are stored in the speech information storage unit, and the speech information stored in the speech information storage means is stored as one speech information. Information (hereinafter referred to as selected voice information) is selected, the continuation state index of each input point, the voice information corresponding to the current frame included in each voice packet sent from each input point, and one frame When the continuation status index at the input point indicated by the speaker index indicates that the utterance is continuing, using the speaker index indicating the input point of the previous speaker and the selected voice information The voice information of an input point other than the input point indicated by the speaker index and the continuation state index indicating that the utterance is continuing (hereinafter referred to as another utterance continuation other point), and the talk Each of the input points indicated by the speaker index is compared with the selected voice information. Voice To determine the input point of the speaker of the current frame, and when the selected voice information is more eligible for the speaker than the voice information of the other point where the utterance continues, the input point of the speaker of the current frame is determined. When it is determined as an input point indicated by the speaker index and the continuation state index of the input point indicated by the speaker index indicates the end of the utterance, the voice information of each input point is compared. It is also possible to determine the input point of the speaker of the current frame. The updating of the continuation state index, the accumulation of audio information, and the selection of selected audio information are performed in units of the input points.

  As described above, even when the continuation state index of the point determined to be the utterance point until one frame ago indicates that the utterance is continuing, the point other than the point determined to be the utterance point until one frame before is From the selected voice information, the voice information of the point where the continuation state index indicates that the utterance is being continued is compared with the selected voice information selected from the voice information of the point determined to be the utterance point one frame before. However, if the voice information at other points is more eligible for the speaker, the voice information at each point is compared to determine a new speaker point. The speaker selection can be optimized for a situation in which the VAD flag corresponding to the first speaker continues to indicate the voice section. Since voice power is not a direct indicator in speaker selection, speaker selection is not directly affected by differences or fluctuations in the input voice level on the transmission side.

  In such a configuration, the continuation state index of each input point, the voice information corresponding to the current frame included in each voice packet sent from each input point, and the main speaker one frame before A first speaker index that indicates an input point of a certain first speaker, a second speaker index that indicates an input point of a second speaker who is a main speaker except for the first speaker one frame before; In the case where the continuation state index of the input point indicated by the first speaker index indicates that the utterance is continuing using the selected voice information (hereinafter referred to as the first case). The voice at an input point other than the input point indicated by the first speaker index and whose continuation state index indicates that the utterance is continuing (hereinafter referred to as another utterance continuation other point). Indicated by the information and the first speaker indicator When the selected voice information is more qualified for the speaker than the voice information at the other point where the utterance is continued, the input point of the first speaker in the current frame is If it is determined as the input point indicated by the first speaker indicator and the continuation state indicator of the input point indicated by the second speaker indicator indicates that the utterance is continuing, the second speaker in the current frame Is determined as the input point indicated by the second speaker indicator, and if the continuation state indicator of the input point indicated by the second speaker indicator indicates the end of the utterance, The voice information of each input point other than the input point indicated by the speaker index is compared to determine the input point of the second speaker in the current frame, and the voice information of the other point where the utterance continues and the first When the selected voice information at the input point indicated by the speaker index is compared with the selected voice information, the voice information at the other point where the utterance is continued is more suitable for the speaker than the selected voice information and in the first case. Or, when the continuation state index of the input point indicated by the first speaker indicator indicates the end of the utterance, the continuation state indicator of the input point indicated by the second speaker indicator is the utterance. If it indicates continuation, the input point of the first speaker in the current frame is determined as the input point indicated by the second speaker indicator, and other than the input point indicated by the second speaker indicator The voice information at each input point is compared to determine the second speaker's input point in the current frame, and the continuation status indicator at the input point indicated by the second speaker indicator indicates the end of speech. If so, the voice information of each input point is compared to determine the first speaker's input point in the current frame, and the voice information of each input point other than the determined first speaker's input point To determine the input point of the second speaker in the current frame.

  Continue for the first and second speakers, even when multiple speakers are selected, such as voice packet communication for real natural conversations where multiple people speak randomly By performing speaker selection using state indicators, the speaker selection is not directly affected by differences or fluctuations in the input voice level on the transmission side, and background noise is included in the input voice signal. Thus, the speaker selection can be optimized for a situation where the VAD flag corresponding to the first speaker continues to indicate the voice section.

  The audio information is a correlation value of audio power or pitch, and the selected audio information is the maximum audio power value or the audio pitch of the audio information stored in the audio information storage means. The maximum correlation value can be assumed.

  Regarding the update of the continuation state index, the continuation state index is an integer value, the continuation state index that is greater than or equal to a predetermined threshold value indicates that utterance is ongoing, and the continuation state index that is less than or less than the threshold value. Represents the end of utterance, and if the VAD flag represents a speech interval, the continuation status indicator is updated by adding a predetermined positive value to the current continuation status indicator, and the VAD flag indicates a non-speech interval. In the case of expressing, the continuation state index can be updated by subtracting a predetermined positive value that is the same as or different from the predetermined positive value from the current continuation state index.

  In this way, the numerical continuation state index is incremented or decremented by the VAD flag from time to time, and this value is used to determine whether the continuation state index represents utterance continuation or utterance end. If the utterance is ongoing, the speaker will continue to be the speaker, but if that is the end of the utterance, the speaker will be searched using voice information, so that the actual speaker ( Smooth speaker switching even when the (speaking point) changes, etc., and it is possible to reduce deterioration in call quality and sound quality after mixing.

  In addition, an upper limit and a lower limit may be provided for the continuous state index, and the continuous state index may not be added and updated if the upper limit is exceeded, and the continuous state index may not be subtracted and updated if the lower limit is exceeded. .

  The computer can be operated as a speaker selection device by the speaker selection program that causes the computer to function as the speaker selection device of the present invention. Then, a computer-readable program recording medium in which the speaker selection program is recorded can cause another computer to function as a speaker selection device or distribute the speaker selection program.

  According to the present invention, using the continuation state index updated by the VAD flag, it is determined whether the continuation state index represents utterance continuation or utterance end, and the utterance is ongoing. If so, the speaker will continue to be the speaker, but if that is the end of the utterance, the speaker will be searched using speech information, so the power of speech is not a direct indicator in speaker selection. In the selection, a speaker can be selected so as not to be directly affected by the difference or fluctuation of the input voice level on the transmission side, and to reduce the deterioration of the speech quality and sound quality after mixing.

  In addition, even when the continuation state index of the point determined to be the utterance point until one frame ago indicates that the utterance is continuing, whether or not the first speaker needs to be re-searched based on the voice information of each point is determined. Judgment (monitoring function) optimizes speaker selection even in situations where the VAD flag corresponding to the first speaker continues to indicate the voice interval due to background noise included in the input voice signal, etc. can do. Further, the numerical continuous state index is adjusted by the VAD flag from time to time, and it is determined whether the continuous state index represents the utterance continuation or the utterance end by the value of the continuation state index. If the utterance is ongoing, the speaker will continue to be the speaker, but if that is the end of the utterance, the speaker will be searched using voice information. ) Changes smoothly, etc. (buffer function). In this way, even if the input speech signal contains background noise or the actual speaker (speaking point) changes within a short period of time, it has resistance to these situations by the monitoring and buffering function, and mixing Speaker selection can be performed to reduce deterioration of later call quality and sound quality.

  Further, such an effect can be obtained with a small amount of calculation such that the continuation state index is updated to determine whether the continuation state index represents the utterance continuation or the utterance end. Therefore, there is no possibility of degrading the quality of voice mixing by applying the speaker selection of the present invention.

<< First Embodiment >>
1st Embodiment of the speaker selection apparatus and method which is this invention is described.
The speaker selection device (1 00 ) according to the first embodiment of the present invention is generally present as an entity constituting a speech mixing device that performs, for example, speech mixing, rather than being independently present alone. . As far Furthermore, the speaker selection device (1 00) is not an entity which constitutes a readily detachably voice mixing device the sound mixing apparatus, one-sided manner and focusing on some of the functions of the audio mixing device itself It can also be said that it was evaluated. In short, the speaker selection device (1 00 ) is generally a voice mixing device itself. Specifically, the speaker selection device (1 00 ) can be realized by mounting the function of the speaker selection device (1 00 ) on a digital signal processor or a dedicated LSI.
However, this does not mean that it exists as a single independent entity and that it is an entity constituting the audio mixing device that can be easily separated from the audio mixing device. For example, for the purpose of speaker selection itself, there is no obstacle to realizing the speaker selection device (1 00 ) as a single independent entity.
Here sound mixing apparatus, for example, to be realized by a computer such as a general purpose machine, such as a dedicated special-purpose machine or a personal computer configured with hardware, to implement the speaker selection device as a single independent entity (1 00) The same applies to the case.

  Hereinafter, the speaker selection device will be described as an entity constituting the speaker mixing device (500) when realized by a computer (general-purpose device). At this time, the speaker selecting device is referred to as a speaker selecting unit, considering that the speaker selecting device is a functional entity constituting the speaker mixing device (500). Note that, from the viewpoint of clarifying the feature of speaker selection according to the present invention, functions and processing related to speaker selection will be mainly described.

FIG. 4 shows a hardware configuration example of the speaker mixing device (500).
As illustrated in FIG. 4, the speaker mixing device (500) includes an input unit (11) to which a keyboard, a pointing device, and the like can be connected, an output unit to which a liquid crystal display, a CRT (Cathode Ray Tube) display, and the like can be connected. 12) a communication unit (13) to which a communication device (for example, a communication cable, a LAN card, a router, a modem, etc.) capable of communicating outside the speaker mixing device (500) can be connected, a DSP (Digital Signal Processor) (14) [ A CPU (Central Processing Unit) may be used. Further, a cache memory, a register (19), and the like may be provided. ] RAM (15) as a memory, ROM (16), hard disk, optical disk, external storage device (17) as a semiconductor memory, etc., and their input unit (11), output unit (12), communication unit (13) , A DSP (14), a RAM (15), a ROM (16), and a bus (18) connected so that data can be exchanged between the external storage devices (17). If necessary, the speaker selection device (1) may be provided with a device (drive) capable of reading and writing a storage medium such as a CD-ROM (Compact Disc Read Only Memory) and a DVD (Digital Versatile Disc).

  The external storage device (17) of the speaker mixing device (500) stores and stores a program for selecting a speaker and a general program necessary for voice mixing processing (not limited to an external storage device). For example, the program may be stored in a ROM that is a read-only storage device. ]. Further, data obtained by the processing of these programs is appropriately stored and stored in a RAM or an external storage device.

  More specifically, in the external mixing device (17) of the speaker mixing device (500) [or ROM, etc.], a program for updating the continuation state of utterance at each point, mainly speaking at all points. The utterance continuation state of the speaker at the point (first speaker) and the utterance continuation state of the speaker (second speaker) at the point where the speaker is mainly speaking out of the points other than the first speaker point are verified. In addition, a program for selecting the first speaker and the second speaker, and a general program required for voice mixing processing are stored and stored. In addition, a control program for controlling processing based on these programs may be stored as appropriate.

Data necessary for the speaker selection process is a VAD flag attached to the voice packet for each speaker point and voice information associated with the VAD flag. These data can be obtained from the voice packet by the decomposition action of the packet decomposition unit. As described above, the voice information may be, for example, a correlation value between the power and pitch of the voice of the voice code. For example, when the power is voice information, the voice information acquisition method may refer to the power code if the power code includes the power code, or the power code may not be included. May decode the speech code once and calculate the power by calculating the sum of squares of the speech waveform in the frame.
These data obtained from the voice packet are stored in an appropriate volatile memory, for example. The voice information associated with the VAD flag is generally associated with the voice code included in the voice packet.

  In the speaker mixing apparatus (500) according to the first embodiment, each program and data (VAD flag, voice information) necessary for processing each program are read into the RAM (15) as necessary, and the DSP is used. Interpretation is executed and processed in (14). As a result, the DSP (14) realizes predetermined functions (speaker selection unit, packet decomposing unit, speech code decomposing unit, low-frequency mixing unit, high-frequency switching unit, speech code combining unit, packet composing unit). Voice mixing including performing speaker selection according to the present invention is realized. The speaker selection unit of the present invention, which is a gist of the speaker mixing device (500), includes a continuation state update unit and a speaker switching determination unit.

  Next, the flow of speaker selection processing in the audio mixing device (500) will be sequentially described with reference to the drawings.

First, FIG. 5 shows a functional configuration example of the speaker selection unit (100) of the first embodiment. Note that, as described above, in the first embodiment, the input is performed at three points as an example, and the number of points may actually be any number as long as it is a plurality of points.
In addition, in the drawing, the reference numerals of the functional components corresponding to each point are distinguished including points A, B, and C. In the following description, unless otherwise specified, the functional component corresponding to a certain point. This will be explained as a representative. That is, for example, continues the state updating unit in figure (101A) (101 B) (101C), is described as a representative to the particular continuation state updating unit unless otherwise indicated (101) in the following description (same hereinafter).

The speaker selection unit (100) includes at least the same number of continuous state update units (101) as the number of input points and one speaker switching determination unit (102).
The speaker selection unit (100) receives the VAD flag and the voice information included in the voice packet at each point (step S301). For example, it is assumed that 0 is assigned to the non-voice section and 1 is assigned to the voice section as the value of the VAD flag. As described above, the voice information can be a correlation value of voice power and pitch.
The VAD flag is input to the continuation state update unit (101) corresponding to each point in the speaker selection unit (100), and the voice information is input to the speaker switching determination unit (102 in the speaker selection unit (100). ).

The continuation state update unit (101) receives the time series of the VAD flag obtained from the voice packets from each point sequentially input to the voice mixing device (500), and updates the value of the current continuation state index. The updated continuation state index is sent to the speaker switching determination unit (102) (step S302).
The continuation state index is an index indicating the continuation state of speech at each point, and the continuation state index can be defined according to a predetermined rule. In this embodiment, it is assumed that the continuation state index is defined as an integer value, a positive integer indicates that the utterance is continuing, and 0 or a negative integer indicates the end of the utterance.

A specific processing example for updating the value of the continuation state index is as follows.
If the initial value of the continuous state index is 0 and the most recently input VAD flag is 1, 1 is added to the current continuous state index. If the most recently input VAD flag is 0, the current continuous state index is 1 is subtracted. Addition and subtraction may be different, such as 2 for addition and 1 for subtraction. Further, when the value of the continuation state index is N (N is a predetermined integer of 1 or more) or more, M (M is a predetermined integer of 1 or more) is added, and when it is less than N, L (L is The number to be added may be varied depending on the current value of the continuation state index, such as addition in increments of a predetermined integer of 1 or more. Similarly, in the case of subtraction, the number to be subtracted may be varied depending on the current value of the continuation state index.
Although the continuous state index is defined as an integer value, a positive integer indicates that the utterance is continuing, and 0 or a negative integer indicates the end of the utterance. The utterance may be continued when the utterance is continuing above the threshold (or greater than the threshold) and less than the threshold (or below the threshold).

  The upper limit of the continuous state index [more precisely, the upper limit is made larger than the threshold value. ] Is determined in advance, and it is preferable not to add more than that. The lower limit of the continuation state index is the above threshold or a predetermined value smaller than the threshold [in the above example, it is 0 or a predetermined negative value. It is preferable not to subtract below that.

  If the upper limit value of the continuation state index is set large, the time until the high frequency point is switched by the high frequency switching unit (504) to be described later after the utterance is actually finished becomes longer. Conversely, if the upper limit value of the continuation state index is set to be small, it is possible to shorten the time from when the utterance is actually finished until the high frequency point is switched. If the frequency is switched too frequently, the sound quality after mixing deteriorates, and if the switching time is too long, the sound becomes unnatural when switching, so the upper limit value is, for example, the switching time limit of 40 ms to 200 ms described in Patent Document 1 above. It should be a value corresponding to.

  As for the lower limit value, if the lower limit value of the continuation state index is set to a small value such as -3, for example, the time until the high frequency point is switched after the utterance is actually started becomes longer. Conversely, if the lower limit value of the continuation state index is set to a large value such as −1 or 0, for example, it is possible to shorten the time from when the utterance actually starts until the high frequency point is switched. It should be noted that it is determined that the sound is suddenly generated for a temporary sound (for example, noise generated when an object falls behind a speaker who is not speaking). From the viewpoint of preventing speaker switching, the lower limit value is set to, for example, −1 [more precisely, a value slightly smaller than the threshold value. ] Should be set. That is, the lower limit value set to be slightly smaller than the threshold value plays a role in buffering speaker switching caused by the temporary sound as described above.

  The speaker switching determination unit (102) receives the continuation state index of each point output from the continuation state update unit (101) and the voice information of each point input to the speaker selection unit (100). A speaker index indicating the location of the speaker (hereinafter referred to as the first speaker number) and a speaker index indicating the location of the second speaker (hereinafter referred to as the second speaker number) are output (steps). S303).

FIG. 6B shows a flowchart of the speaker selection process (details of step S303) in the speaker switching determination unit (102). The information processing subject of each process below is a speaker switching determination unit (102).
Here, the first speaker number and the second speaker number are numbers assigned to the respective input points. For example, the point A is 0, the point B is 1, and the point C is 2.
The initial values of the first speaker number and the second speaker number may be at arbitrary points. However, the same value shall not be assumed. For example, the initial value of the first speaker number is 0, and the initial value of the second speaker number is 1.

<Step S1>
First, the previous frame [Note that one frame corresponds to the voice code included in the voice packet. ] (The first speaker number one frame before is appropriately stored in a memory such as a RAM. If there is no previous frame, the initial value of the speaker number is used.) Just do it.) This is defined as T1 (k-1). The argument k represents the current frame number, and k-1 means the frame number one frame before.

<Step S2>
From the continuation state index of the point indicated by the first speaker number T1 (k-1), the utterance continuation state of the point designated as the first speaker is determined. For example, if the continuation state index is positive, the utterance is continued, and if it is 0 or negative, the utterance is ended.

<Step S3>
If the point indicated by the first speaker number T1 (k-1) is continuing the utterance as a result of the continuation state determination in the process of step S2, the process of step S4 is performed, and if the utterance is ended, the process of step S5 is performed. Do.

<Step S4>
If it is determined that the point indicated by the first speaker number T1 (k-1) is continuing utterance, the re-search flag for the first speaker is reset to zero. The value 0 of the re-search flag indicates that re-search is not necessary.

<Step S5>
When it is determined that the point indicated by the first speaker number T1 (k-1) is the end of the utterance, the re-search flag of the first speaker is set to 1. The value 1 of the re-search flag indicates that re-search is necessary.

<Step S6>
Next, the second speaker number one frame before is acquired in the same manner as the process of step S1 (the second speaker number one frame before is appropriately stored in a memory such as a RAM. If there is no frame, the initial value of the speaker number may be used.) This is T2 (k−1).

<Step S7>
The utterance continuation state at the point designated as the second speaker is determined from the continuation state index at the point indicated by the second speaker number T2 (k-1). As in the case of the process of step S2, the determination method is that the utterance is continued if the continuation state index is positive, and the utterance is ended if it is 0 or negative. However, the judgment criteria may be different between the first speaker point and the second speaker point.

<Step S8>
If the point indicated by the second speaker number T2 (k-1) is continuing the utterance as a result of the continuation state determination in step S7, the process of step S9 is performed, and if the utterance is ended, the process of step S10 is performed. .

<Step S9>
If it is determined that the point indicated by the second speaker number T2 (k-1) is continuing utterance, the second speaker re-search flag is reset to zero. The value 0 of the re-search flag indicates that re-search is not necessary.

<Step S10>
When it is determined that the point indicated by the second speaker number T2 (k-1) is the end of the utterance, the second speaker re-search flag is set to 1. The value 1 of the re-search flag indicates that re-search is necessary.

<Step S11>
If the value of the first speaker re-search flag is 0, the process of step S12 is performed. If the value of the first speaker re-search flag is 1, the process of step S15 is performed.

<Step S12>
When the re-search flag of the first speaker is 0, the first speaker number T1 (k-1) in the previous frame is set to the first speaker number T1 (k) in the current frame.

<Step S13>
Next, the value of the second speaker's re-search flag is checked. If the value of the second speaker's re-search flag is 0, the process of step S14 is performed. Step S18 is performed.

<Step S14>
When the second speaker re-search flag is 0, the second speaker number T2 (k-1) one frame before is set to the second speaker number T2 (k) in the current frame.

<Step S15>
In the process of step S11, if the re-search flag of the first speaker is 1, if the re-search flag value of the second speaker is 0 according to the re-search flag value of the second speaker, step S16 If the value of the second speaker re-search flag is 1, the process of step S17 is performed.

<Step S16>
When the second speaker re-search flag is 0, the second speaker number T2 (k-1) one frame before is set to the first speaker number T1 (k) in the current frame. Subsequently, the process of step S18 is performed.

<Step S17>
If the second speaker re-search flag is 1, the first speaker point is searched. The search method may be based on, for example, a method in which the point where the voice information--for example, the correlation value of power and pitch--is the maximum is set as the first speaker point from the points where the continuation state index indicates that the utterance is continuing. . Subsequently, the process of step S18 is performed.

<Step S18>
If the second speaker re-search flag is 1 in the process of step S13, and if the process of either step S16 or step S17 is completed, the point indicated by the first speaker number T1 (k) of the current frame The second speaker point is searched from points other than. This specific search procedure may be the same as the search for the first speaker point in the process of step S17.

  As described above, in the first embodiment, if the first speaker is continuing to speak at the stage of the previous frame, the first speaker at the stage of the previous frame is directly used as the first speaker of the current frame. In this case, if the utterance of the second speaker is ongoing at the previous frame, the second speaker at the previous frame is used as the second speaker of the current frame, and the utterance ends at the previous frame. If so, the second speaker of the current frame is searched from among the speakers at each point except the point of the first speaker. Further, if the first speaker ends the utterance at the previous frame stage, the processing is developed depending on the utterance continuation state of the second speaker at the previous frame stage. In other words, if the second speaker at the previous frame stage is still speaking, the second speaker at the previous frame stage is set as the first speaker of the current frame, and is designated as the first speaker here. The second speaker of the current frame is searched from the speakers at each point except the point. If the second speaker in the previous frame stage is the end of the utterance, first the first speaker of the current frame is searched from among the speakers at each point, and then the point designated as the first speaker here The second speaker of the current frame is searched from the speakers at each point except for.

  Hereinafter, with reference to FIG. 7, a description will be given of the speech mixing processing in the speech mixing device (500) to which the speaker selection unit (100) of the first embodiment is applied. Here, a description will be given using an example of three-point input and three-point output.

  The packet decomposing unit (501) extracts a voice code, a VAD flag, and voice information from the voice packet. The speech code is input to the speech code decomposition unit (502). The VAD flag and voice information are input to the speaker selection unit (100).

  The speech code decomposition unit (502) decomposes the speech code into a low frequency code and a high frequency code, the low frequency code is sent to the low frequency mixing unit (503), and the high frequency code is sent to the high frequency switching unit (504). . As described above, the high frequency code is one of enhancement layer codes, and the high frequency code may be read as an arbitrary enhancement layer code.

  The low frequency mixing unit (973) includes a low frequency decoding unit (5031), a mixing unit (5032), and a low frequency encoding unit (5033). First, a low frequency decoding unit (5031) receives a low frequency code from each point and decodes it to obtain a low frequency signal. Next, the mixing unit (5032) adds the low-frequency signals of all points to obtain a total low-frequency signal, and excludes the low-frequency signals of each point from the total low-frequency signal for each point. Generates a low-frequency signal. For example, the low-frequency encoding unit for point A (5033A) encodes the low-frequency signal for point A to generate a low-frequency code (for point A), and converts the low-frequency code into the speech code combining unit (505A). ) The same applies to the low-frequency encoding unit for point B (5033B) and the low-frequency encoding unit for point C (5033C).

  As described above, the speaker selection unit (100) refers to the VAD flag and voice information from each point, and outputs the first speaker number and the second speaker number. The first speaker number and the second speaker number are input to the high frequency switching unit (504).

The high-frequency switching unit (504) receives the high-frequency code from each point and, upon receiving the first speaker number and the second speaker number, switches the high-frequency code to switch each speech code combining unit (505A) ( 505B) (505C).
An example in which the point indicated by the first speaker number is the point A and the point indicated by the second speaker number is the point B will be specifically described. For the voice of the first speaker, the first switching unit (5041) selects the high-frequency code of the point A, and the second switching unit (5042) is a point other than the point indicated by the first speaker number--that is, Point B and destination C are selected as output destinations, and the high-frequency code at point A is sent to the speech code combining unit (505B) and the speech code combining unit (505C). As for the voice of the second speaker, the first switching unit (5041) selects the high band code of the point B, and the second switching unit (5042) selects the point A which is the point indicated by the first speaker number. The high frequency code of point B is selected as the output destination and sent to the speech code combining unit (505A).

  The speech code combining unit (505) combines the low frequency code and the high frequency code to generate a speech code, and sends this speech code to the packet configuration unit (507). The packet construction unit (507) generates a voice packet using the received voice code, and outputs the voice packet to a target point.

<< Modification of First Embodiment >>
Next, a modification of the first embodiment will be described.
In the first embodiment, the speaker selection unit (100) outputs the first speaker number and the second speaker number. On the other hand, the modification of 1st Embodiment is a form which outputs only a 1st speaker number as a simple form. Here, a different part from 1st Embodiment is demonstrated.

FIG. 8 shows a flowchart (corresponding to step S303) of speaker selection processing in a modification of the first embodiment. The information processing subject of each process below is a speaker switching determination unit (102).
Each processing from step S1 to step S5 is the same as that of the first embodiment. In the modification of the first embodiment, the process of step S11 of the first embodiment is performed following the process of step S4 or step S5. Each processing from Step S6 to Step S10 in the first embodiment is not necessary.
In the modification of the first embodiment, after the process of step S11, if the value of the re-search flag for the first speaker is 0, the process of step S12 is performed. And it is complete | finished without passing through each process of said step S13 and step S14. After the process of step S11, if the re-search flag value of the first speaker is 1, the process of step S17 is performed. And it is complete | finished without passing through the process of said step S18. In addition, each process of step S15 and step S16 in 1st Embodiment is unnecessary.

  Note that, as is apparent from the flowchart shown in FIG. 8, the processes of step S4, step S5, and step S11 are substantially unnecessary. Accordingly, if the point indicated by the first speaker number T1 (k-1) is continuing the utterance as a result of the determination of the continuation state in the process of step S2, the process of step S3 is performed, and the process of step S12 is performed. If so, the process of step S17 may be performed.

As the modification of the first embodiment briefly shows, the basic feature of the present invention is that the continuation state index is sequentially updated according to the time series of the VAD flag, and the continuation state index indicates that the utterance is continuing. Or the end of the utterance, and if the utterance is ongoing, the first speaker will continue to be the first speaker. To search for one speaker.
The first embodiment described above can be described as performing this basic feature in a combination of the first speaker and the second speaker.

  The speech mixing process in the speech mixing device (500) to which the speaker selection unit (100) of the modification of the first embodiment is applied will be described. Here, only the parts different from the case of the first embodiment will be described.

  As described above, the speaker selection unit (100) refers to the VAD flag and voice information from each point and outputs only the first speaker number. The first speaker number is input to the high frequency switching unit (504).

The high frequency switching unit (504) receives the high frequency code from each point and, upon receiving the first speaker number, switches the high frequency code to each of the voice code combining units (505A) (505B) (505C). Output.
An example in which the point indicated by the first speaker number is the point A will be specifically described. For the voice of the first speaker, the first switching unit (5041) selects the high-frequency code of the point A, and the second switching unit (5042) is a point other than the point indicated by the first speaker number—that is, the point The address B and the address C are selected as output destinations, and the high frequency code at the point A is sent to the speech code combining unit (505B) and the speech code combining unit (505C). At this time, it is only necessary that a high-frequency silence code is sent to the speech code combining unit (505A) in which the point indicated by the first speaker number corresponds to the point A.

<< Second Embodiment >>
Hereinafter, a second embodiment of the present invention will be described.
FIG. 9 shows a functional configuration example of the speaker selecting unit (200) in the second embodiment. Similar to the description, in the first embodiment, the input is performed at three points, and in practice, the number of points may be any number as long as it is a plurality of points.

  In the second embodiment, even when the continuation state index of the point determined to be the utterance point until one frame ago indicates that the utterance is continuing, the points other than the point determined to be the utterance point until one frame ago Of the selected voice information, the voice information at the point where the continuation state index indicates that the utterance is being continued is compared with the selected voice information selected from the voice information at the point determined to be the utterance point one frame before. If the voice information at other points is more eligible for the speaker than the voice information at the other points, the voice information at each point is compared to determine a new speaker point. This second embodiment is based on the voice information at each point in a situation where the VAD flag corresponding to the first speaker continues to indicate the voice section because the input voice signal includes background noise. In addition, the selection of the speaker is made appropriate by determining whether the first speaker needs to be searched again.

  In the second embodiment, a buffer (103) and a voice information search unit (104) are added to the functional configuration of the speaker selection unit (100) of the first embodiment. In order to simplify the description, the description of the same part as in FIG. 5 is omitted, and the difference from the first embodiment will be described.

  The voice information input to the speaker selection unit (200) is sent to the speaker switching determination unit (102) and also sent to the buffer (103) and stored therein (step S3021). The buffer (103) is storage means capable of accumulating audio information in past frames for a predetermined number of frames. The buffer (103) can be realized by a RAM (15), a register (19) or the like, and may be a shift buffer, for example.

  The voice information search unit (104) searches the buffer (103), selects one piece of voice information (selected voice information) from voice information within a predetermined past predetermined time, and selects a speaker switching judgment unit. (102) (step S3022). The selection method of the selected voice information can be appropriately determined according to the type of the voice information, and the voice information (selected voice information) indicating the speaker eligibility is selected from the voice information stored in the buffer (103). For example, if the voice information is a correlation value of voice power or pitch, the maximum value can be used as the selected voice information. Note that the selection of the selected voice information may be performed by the voice information search unit (104) corresponding to the point designated as the first speaker one frame before.

  The speaker switching determination unit (102) receives the continuation state index and voice information for each point, and the selected voice information of the first speaker, and outputs the first speaker number and the second speaker number (step S303 ′). ).

FIG. 10-2 shows a flowchart of the speaker selection process (details of step S303 ′) in the speaker switching determination unit (102) of the second embodiment. The information processing subject of each process below is a speaker switching determination unit (102).
The flowchart of the speaker selection process in the second embodiment is a process in which the processes in step S31 and step S32 are added to that in the first embodiment. In the description of the speaker selection process in the second embodiment, the description of the same part as the speaker selection process in the first embodiment is omitted.

<Step S3>
If the point indicated by the first speaker number T1 (k-1) is continuing the utterance as a result of the continuation state determination in the process of step S2, the process of step S31 is performed. Do.

<Step S31>
When it is determined that the point indicated by the first speaker number T1 (k-1) is continuing the utterance, the selected voice information of the point indicated by the first speaker number T1 (k-1) and the voice of another point Determine whether forced disconnection is necessary from the information. Here, forced disconnection means that the value of the re-search flag is forcibly set to 1—that is, re-search is required—even if the continuation state index indicates that the utterance is continuing.
For example, if the voice information is a voice power value, the selected voice information (for example, the maximum voice power value) at the point indicated by T1 (k−1) is smaller than a predetermined threshold value and T1 (k− The continuation state index of the point where the selected voice information of the point indicated by 1) is smaller than any of the voice information of other points and the voice information is larger than the selected voice information of the point indicated by T1 (k-1) If it indicates that the utterance is continuing, it is determined that forced disconnection is necessary, and if not, it is determined that forced disconnection is unnecessary.

<Step S32>
Depending on whether forced disconnection is necessary or not, step S4 is performed if forced disconnection is not required, and step S5 is performed if forced disconnection is necessary.

<Step S4>
If the forced disconnection is unnecessary, the first speaker re-search flag is reset to zero.

<Step S5>
If it is determined in the process of step S3 that the utterance has ended as a result of the continuation determination, or if it is determined in the process of step S32 that the forced disconnection is necessary, the re-search flag for the first speaker is set to 1. To do.

  The processing after step S6 is the same as the speaker selection processing in the first embodiment.

<< Modification of Second Embodiment >>
Next, a modification of the second embodiment will be described.
In the second embodiment, the speaker selection unit (100) outputs the first speaker number and the second speaker number. On the other hand, the modification of 2nd Embodiment is a form which outputs only a 1st speaker number as a simple form. That is, the modification of the second embodiment is similar to the modification of the first embodiment. A different part from 2nd Embodiment is demonstrated.

FIG. 11 shows a flowchart of speaker selection processing in a modification of the second embodiment (corresponding to step S303 ′). The information processing subject of each process below is a speaker switching determination unit (102).
Each process of step S1 to step S5 including step S31 and step S32 is the same as that of 2nd Embodiment. In the modification of the second embodiment, the process of step S11 of the second embodiment is performed following the process of step S4 or step S5. Each processing from Step S6 to Step S10 in the second embodiment is not necessary.
In the modification of the second embodiment, after the process of step S11, if the value of the re-search flag of the first speaker is 0, the process of step S12 is performed. And it is complete | finished without passing through each process of said step S13 and step S14. After the process of step S11, if the re-search flag value of the first speaker is 1, the process of step S17 is performed. And it is complete | finished without passing through the process of said step S18. In addition, each process of step S15 and step S16 in 2nd Embodiment is unnecessary.

  Note that, as is apparent from the flowchart shown in FIG. 11, the processes of step S4, step S5, and step S11 are substantially unnecessary. Therefore, if it is determined in step S32 that forced disconnection is not necessary, step S12 is performed. In step S3, if it is determined that the utterance is ended by the continuation determination result, If it is determined in the forced disconnection determination that forced disconnection is necessary, the process of step S17 may be performed.

  Since the audio mixing process in the audio mixing device (500) to which the speaker selection unit (100) of the modification of the second embodiment is applied is the same as that of the modification of the first embodiment, the description thereof is omitted.

  In each of the above embodiments, the speaker selection device has been described as a functional entity (speaker selection unit) of the voice mixing device. However, the speaker selection device is realized as a single independent entity, for example, by a computer (general-purpose device). Even so, there is no substantial difference between the functional configuration of the speaker selection unit described in the above embodiment and the hardware configuration of a computer (general-purpose device) that realizes the functional configuration.

  In addition to the above embodiment, the speaker selection device / method according to the present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present invention. Further, the processing described in the above speaker selection device / method is not only executed in time series in the order described, but also executed in parallel or individually as required by the processing capability of the device that executes the processing. It may be.

  When the processing functions in the speaker selection device are realized by a computer, the processing contents of the functions that the speaker selection device should have are described by a program. Then, by executing this program on a computer, the processing functions in the speaker selection device are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  Further, the speaker selection device may be configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

  The present invention can be applied to the case where the background noise is included in the voice included in the voice packet transmitted from each point, when the input voice level fluctuates, when the input voice level at each point is different, or for a short time. Even if the actual speaker (speaking point) changes during this period, it is possible to select a more accurate speaker, and as a result, it is possible to obtain a stable mixed voice with little deterioration in sound quality after mixing. It is useful for multipoint voice communication by voice packet communication.

The figure which shows the system structural example of multipoint audio | voice communication. The figure which shows the function structural example of an audio | voice packet transmission part. The figure which shows the function structural example of an audio | voice mixing apparatus. The figure which shows the hardware structural example of an audio | voice mixing apparatus. The figure which shows the function structural example of the speaker selection apparatus of 1st Embodiment. The figure which shows the flow of the speaker selection process in 1st Embodiment. The figure which shows the detailed process flow of a speaker switching judgment part in the speaker selection process in 1st Embodiment. The figure which shows the function structural example of the audio | voice mixing apparatus to which the speaker selection apparatus (speaker selection part) of 1st Embodiment is applied. The figure which shows the detailed process flow of a speaker switching judgment part in the speaker selection process in the modification of 1st Embodiment. The figure which shows the function structural example of the speaker selection apparatus of 2nd Embodiment. The figure which shows the flow of the speaker selection process in 2nd Embodiment. The figure which shows the detailed process flow of a speaker switching judgment part in the speaker selection process in 2nd Embodiment. The figure which shows the detailed process flow of a speaker switching judgment part in the speaker selection process in the modification of 2nd Embodiment.

Explanation of symbols

100 speaker selection unit 101 continuation state update unit 102 speaker switching determination unit 103 buffer 104 voice information maximum value search unit 200 speaker selection unit 500 voice mixing device 501A, 501B, 501C packet decomposition unit 502A, 502B, 502C voice code decomposition Unit 503 low frequency mixing unit 5031A, 5031B, 5031C low frequency decoding unit 5032 mixing unit 503 3 A, 503 3 B, 503 3 C low frequency encoding unit 504 high frequency switching unit 505A, 505B, 505C speech code combining unit 507A, 507B, 507C Packet component

Claims (12)

A speaker selection device in voice mixing that performs speaker selection and mixing for voice packets sent from three or more points (hereinafter referred to as input points),
Corresponding to each input point,
Whether the voice code of one frame (hereinafter referred to as the current frame) included in a voice packet sent from one input point is a voice section or a non-voice section Continuation state updating means for updating a continuation state index that is an index indicating the utterance continuation state of the input point using a determination result (hereinafter referred to as a VAD flag),
Speech information storage means for storing acoustic feature quantities (hereinafter referred to as speech information) of speech codes for each frame included in each past and present speech packet sent from one input point;
Voice information search means for selecting one voice information (hereinafter referred to as selected voice information) from the voice information stored in the voice information storage means;
The continuation state index of each input point, the voice information corresponding to the current frame included in each voice packet sent from each input point, a speaker index indicating the input point of the speaker one frame before, With the selected voice information as input,
When the continuation status index at the input point indicated by the speaker index indicates that the utterance is ongoing,
The voice information of an input point other than the input point indicated by the speaker index and the continuation state index indicating that the utterance is continuing (hereinafter referred to as another utterance continuation other point), and the speaker Compare the selected voice information of the input point indicated by the index,
If the voice information at the other point where the utterance continues than the selected voice information is eligible for the speaker, the voice information of each input point is compared to determine the input point of the speaker of the current frame,
If the selected voice information is more eligible for the speaker than the voice information of the utterance continuation other point, the input point of the speaker of the current frame is determined as the input point indicated by the speaker index,
When the continuation status indicator at the input point indicated by the speaker indicator indicates the end of utterance,
A speaker selection device comprising speaker switching judgment means for comparing the voice information of each input point to determine the input point of the speaker of the current frame. The speaker switching judgment means is
The continuation state index of each input point, the voice information corresponding to the current frame included in each voice packet transmitted from each input point, and the input point of the first speaker who is the main speaker one frame before The first speaker indicator to be instructed, the second speaker indicator to instruct the input point of the second speaker who is the main speaker except the first speaker one frame before, and the selected voice information are input. ,
When the continuation state index of the input point indicated by the first speaker index indicates that the utterance is continuing (hereinafter referred to as the first case), and the first speaker index The voice information of the input point other than the input point indicated by the input point and the continuation state index indicating that the utterance is continuing (hereinafter referred to as another point of utterance continuation) and the first speaker index When the selected voice information is compared with the selected voice information at the designated input point, and the selected voice information is more eligible for the speaker than the voice information at the other point where the utterance continues,
Determining the input point of the first speaker in the current frame as the input point indicated by the first speaker index;
If the continuation status indicator at the input point indicated by the second speaker indicator indicates that the utterance is continuing, the input indicated by the second speaker indicator at the input point of the second speaker in the current frame As a point,
If the continuation status indicator at the input point indicated by the second speaker indicator indicates the end of utterance, the voice information at each input point other than the input point indicated by the first speaker indicator is compared. To determine the input point of the second speaker in the current frame,
The voice information of the utterance continuation other point is compared with the selected voice information by comparing the voice information of the utterance continuation other point with the selected voice information of the input point indicated by the first speaker index. In the first case, or when the continuation status indicator at the input point indicated by the first speaker indicator indicates the end of speech,
If the continuation status indicator at the input point indicated by the second speaker indicator indicates that the utterance is continuing, the input indicated by the second speaker indicator at the input point of the first speaker in the current frame Determine the point of input of the second speaker in the current frame by comparing the voice information of each input point other than the input point indicated by the second speaker index,
If the continuation status indicator at the input point indicated by the second speaker indicator indicates the end of utterance, the voice information at each input point is compared to determine the input point of the first speaker in the current frame Then, the voice information of each input point other than the determined input point of the first speaker is compared with that for determining the input point of the second speaker in the current frame. Item 2. The speaker selection device according to Item 1 . The voice information is assumed to be a voice power or pitch correlation value,
The selection audio information, among the audio information stored the voice information in the storage unit, according to claim 1 or claim characterized in that it is a maximum value of the correlation value of the maximum value or audio pitch of the speech power 2. The speaker selection device according to 2. The continuation state index is an integer value, the continuation state index that is greater than or equal to a predetermined threshold represents continuation of utterance, and the continuation state index less than or less than the threshold represents utterance end,
When the VAD flag represents a voice interval, the continuation state update means updates the continuation state indicator by adding a predetermined positive value to the current continuation state indicator, and the VAD flag indicates a non-voice interval. to represent the one of claims 1, characterized in that updating the persistent state indication by the current of the continuous state index reduces the same or different predetermined positive value to the predetermined positive value of claim 3 The speaker selection device according to the above. An upper limit and a lower limit are set for the continuous state index,
The continued state updating means, exceed the upper limit without the addition updates the persistent state indicator, also to claim 4 is below the above lower limit, characterized in that not the subtraction update the persistent state indicator The speaker selection device described. A speaker selection method in voice mixing in which a speaker is selected and mixed with respect to a voice packet transmitted from three or more points (hereinafter referred to as input points),
Corresponding to each input point,
Whether the voice code of one frame (hereinafter referred to as the current frame) included in a voice packet sent from one input point is a voice section or a non-voice section. A continuation state update step of updating a continuation state index that is an index indicating the utterance continuation state of the input point using a determination result (hereinafter referred to as a VAD flag) indicating
A voice information storage step in which voice information storage means stores an acoustic feature quantity (hereinafter referred to as voice information) of a voice code of a frame included in a voice packet sent from one input point;
A voice information search means for selecting one voice information (hereinafter referred to as selected voice information) from the voice information stored in the voice information storage means;
The speaker switching determination means determines the continuation state index of each input point, the voice information corresponding to the current frame included in each voice packet transmitted from each input point, and the speaker's input point one frame before. With the speaker index to be instructed and the selected voice information as input,
When the continuation status index at the input point indicated by the speaker index indicates that the utterance is ongoing,
The voice information of an input point other than the input point indicated by the speaker index and the continuation state index indicating that the utterance is continuing (hereinafter referred to as another utterance continuation other point), and the speaker Compare the selected voice information of the input point indicated by the index,
If the voice information at the other point where the utterance continues than the selected voice information is eligible for the speaker, the voice information of each input point is compared to determine the input point of the speaker of the current frame,
If the selected voice information is more eligible for the speaker than the voice information of the utterance continuation other point, the input point of the speaker of the current frame is determined as the input point indicated by the speaker index,
When the continuation status indicator at the input point indicated by the speaker indicator indicates the end of utterance,
A speaker selection method, comprising: a speaker switching determination step of determining the input point of the speaker of the current frame by comparing the voice information of each input point. The above speaker switching judgment step
The speaker switching determination means is the continuation state index of each input point, the voice information corresponding to the current frame included in each voice packet transmitted from each input point, and the main speaker one frame before A first speaker index indicating the input point of the first speaker; a second speaker index indicating the input point of the second speaker who is the main speaker except for the first speaker one frame before; With selected voice information as input,
When the continuation state index of the input point indicated by the first speaker index indicates that the utterance is continuing (hereinafter referred to as the first case), and the first speaker index The voice information of the input point other than the input point indicated by the input point and the continuation state index indicating that the utterance is continuing (hereinafter referred to as another point of utterance continuation) and the first speaker index When the selected voice information is compared with the selected voice information at the designated input point, and the selected voice information is more eligible for the speaker than the voice information at the other point where the utterance continues,
Determining the input point of the first speaker in the current frame as the input point indicated by the first speaker index;
If the continuation status indicator at the input point indicated by the second speaker indicator indicates that the utterance is continuing, the input indicated by the second speaker indicator at the input point of the second speaker in the current frame As a point,
If the continuation status indicator at the input point indicated by the second speaker indicator indicates the end of utterance, the voice information at each input point other than the input point indicated by the first speaker indicator is compared. To determine the input point of the second speaker in the current frame,
The voice information of the utterance continuation other point is compared with the selected voice information by comparing the voice information of the utterance continuation other point with the selected voice information of the input point indicated by the first speaker index. In the first case, or when the continuation status indicator at the input point indicated by the first speaker indicator indicates the end of speech,
If the continuation status indicator at the input point indicated by the second speaker indicator indicates that the utterance is continuing, the input indicated by the second speaker indicator at the input point of the first speaker in the current frame Determine the point of input of the second speaker in the current frame by comparing the voice information of each input point other than the input point indicated by the second speaker index,
If the continuation status indicator at the input point indicated by the second speaker indicator indicates the end of utterance, the voice information at each input point is compared to determine the input point of the first speaker in the current frame Then, the voice information of each input point other than the determined input point of the first speaker is compared with that for determining the input point of the second speaker in the current frame. Item 7. The speaker selection method according to Item 6 . The voice information is assumed to be a voice power or pitch correlation value,
The selection audio information, among the audio information stored the voice information in the storage unit, according to claim 6 or claim, characterized in that the maximum value of the correlation value of the maximum value or audio pitch of the speech power 8. The speaker selection method according to 7 . The continuation state index is an integer value, the continuation state index that is greater than or equal to a predetermined threshold represents continuation of utterance, and the continuation state index less than or less than the threshold represents utterance end,
The continuation state update step updates the continuation state index by adding a predetermined positive value to the current continuation state index when the VAD flag represents a speech period, and the VAD flag indicates a non-speech period. to represent the one of claims 6, characterized in that updating the persistent state indication by the current of the continuous state index reduces the same or different predetermined positive value to the predetermined positive value of claims 8 The speaker selection method described in Crab. An upper limit and a lower limit are set for the continuous state index,
The continued state updating step, exceed the upper limit without the addition updates the persistent state indicator, also to claim 9 is below the above lower limit, characterized in that not the subtraction update the persistent state indicator The speaker selection method described. A speaker selection program for causing a computer to function as the speaker selection device according to any one of claims 1 to 5 . A computer-readable recording medium on which the speaker selection program according to claim 11 is recorded.

Images