JP2006252458A - Voice signal processor for processing voice signals of a plurality of speakers, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a means for activating a conference and improving the progress of proceedings without increasing participants' burdens in an audio conference system. <P>SOLUTION: A voice signal receiving section 141 of a server device 14 receives voice signals of users 19 from respective terminal units 11. A response voice signal selecting section 1455 of the server device 14 determines whether conditions included in a response condition database 1444 are met, and when the conditions are met, the response voice signal selecting section 1455 reads a response voice signal associated with the conditions, from a response voice signal group 1445 and delivers it to a response voice signal mixing section 1457. The response voice signal mixing section 1457 mixes the response voice signal delivered from the response voice signal selecting section 1455, with a voice signal indicating the speech of each user 19 mixed by an spoken voice signal mixing section 1456, and then transmits the mixed signals to each terminal unit 11. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an audio signal processing technique for conducting an audio conference using an audio device.

In order to activate the telephone call, a sound effect signal is sent to the telephone of the other user who is making a call by one user making a call operating the operation unit provided on the telephone. There are techniques that allow transmission. For example, Patent Literatures 1 to 3 disclose such prior art.
JP-A-11-55379 JP 2002-51116 A JP 2004-112109 A

  In an audio conference system that can implement an audio conference using an acoustic device such as a headset, activation of conversation in the conference can be promoted by using the above-described conventional technology. However, according to the above-described conventional technology, it is necessary to perform an operation for transmitting a sound effect signal suitable for any user participating in the conference to a terminal device or the like of another user. Therefore, there are inconveniences such as users who are not able to concentrate on the conference.

  Also, in audio conferencing, usually when multiple users speak at once, to control it, or to encourage participants to organize the content of the meeting when the scheduled meeting end time approaches However, there are inconveniences such as users who are not able to concentrate on the meeting.

  In view of the above situation, an object of the present invention is to provide means for enabling activation of a conference and improvement of proceedings without increasing the burden on participants in an audio conference system.

  In order to achieve the above object, the present invention provides a storage unit that stores a response voice signal indicating a response voice, an input unit that receives a voice signal from each of a plurality of terminal devices, and the input unit. Detecting means for detecting a signal indicating mute from the sound signal based on the level of each part of the sound signal received by the means, and time measuring means for measuring the duration of the signal indicating mute detected by the detecting means; An audio signal processing apparatus comprising: selection means for selecting a response audio signal corresponding to the time measured by the time measuring means; and output means for outputting the response audio signal selected by the selection means. provide.

  According to the audio signal processing apparatus having such a configuration, a suitable response audio signal is automatically output according to the length of the silent period, and the activation of the audio conference and the progress of the proceedings are improved.

  Further, in the voice signal processing device, the storage means further stores key phrase data indicating a key phrase, and is indicated by the voice signal by performing voice recognition processing on the voice signal received by the input means. Voice recognition means for specifying a phrase included in the utterance; and detection means for detecting a key phrase indicated by the key phrase data from the phrase specified by the voice recognition means; and the selection means includes the detection means The response voice signal corresponding to the key phrase data indicating the key phrase detected by the above may be selected.

  According to the audio signal processing apparatus having such a configuration, a suitable response audio signal is automatically output in response to a specific statement, so that the audio conference can be activated and the progress of proceedings can be improved.

  In the audio signal processing device, the selection unit is configured to select a response audio signal corresponding to a terminal device that is an output source of the audio signal received by the input unit within a predetermined period in the past. Also good.

  According to the audio signal processing apparatus having such a configuration, a response audio signal suitable for each participant is automatically output, which is more preferable.

  The audio signal processing device further includes a time measuring unit that measures an elapsed time from a specific time point, and the selection unit selects a response audio signal corresponding to the time measured by the time measuring unit. May be.

  According to the audio signal processing device having such a configuration, for example, it is possible to perform a process such as automatically outputting a response audio signal for notifying a participant to that effect a predetermined time before the scheduled conference end time, and the convenience is enhanced.

  The audio signal processing apparatus further includes time measuring means for measuring a time duration of the audio signal output from the one terminal device received by the input means, and the selecting means is measured by the time measuring means. The response voice signal corresponding to the set time may be selected.

  According to the audio signal processing apparatus having such a configuration, for example, it is possible to perform a process such as encouraging a participant who speaks for a long time to suppress the speech and prompt other participants to speak, thereby improving convenience.

  Further, in the audio signal processing device, the selection means is an audio signal whose output source is another terminal device when an audio signal whose output source is one terminal device is continuously received by the input means. May be configured to select a predetermined response audio signal.

  According to the audio signal processing apparatus having such a configuration, when a plurality of participants speak at the same time, it is possible to perform processing such as organizing the progress of the conference after temporarily controlling those comments, and the convenience is enhanced.

  Further, in the audio signal processing apparatus, when the audio signal is received by the input unit when the response audio signal is output by the output unit, the output unit is not output among the response audio signals being output. You may comprise so that at least one part of a part may not be output.

  According to the audio signal processing apparatus having such a configuration, the problem that the speech of the participant is hindered by the response audio signal that is automatically output is avoided.

  Further, in the audio signal processing device, the output unit is configured to output a response audio signal using only a terminal device that is an output source of the audio signal received by the input unit within a predetermined period in the past as a transmission destination. May be.

  According to the audio signal processing device having such a configuration, the response audio signal avoids the inconvenience that a person other than the current speaker is disturbed.

  The audio signal processing apparatus further includes a mixing unit that mixes the audio signal received by the input unit and the response audio signal selected by the selection unit, and the output unit includes the mixing unit. An audio signal obtained as a result of mixing may be output.

  According to the audio signal processing device having such a configuration, the audio signal indicating the speech of the conference participant and the automatically selected response audio signal are simultaneously transmitted to the participant's terminal device and are output separately. Compared to the case, a simple system can be constructed.

  Further, in the voice signal processing device, the storage means stores response text data indicating a message character for response in addition to the response voice signal or instead of the response voice signal, and the selection means includes: In addition to the response voice signal corresponding to the time measured by the time measuring means, or instead of the response voice signal corresponding to the time measured by the time measuring means, the response text corresponding to the time measured by the time measuring means The data may be selected, and the output unit may be configured to output response text data selected by the selection unit.

  According to the voice signal processing apparatus having such a configuration, it is possible to notify the participant by the character of the response message that is automatically selected, which is advantageous in some cases as compared with the notification of the response message by voice.

  In the audio signal processing device, the storage unit stores response identification data for identifying one response audio signal from another response audio signal, instead of the response audio signal, and the selection unit includes: When the audio signal received by the input means satisfies a predetermined condition, instead of the response audio signal corresponding to the condition, response identification data corresponding to the condition is selected, and the output means is the selection means May be configured to output the response identification data selected by.

  According to the audio signal processing device having such a configuration, the audio signal processing device transmits response identification data having a smaller amount of data compared to the response audio signal to the terminal device or the like, so that the response audio signal is stored in advance or acquired from others. By combining with a possible terminal device, automation of the proceedings of the audio conference is realized, which is convenient.

  The present invention also provides a program that causes a computer to execute processing performed by the above-described audio signal processing apparatus.

[Embodiment]
[1. Configuration of audio conference system]
The audio conference system 1 according to the embodiment of the present invention is a system that allows users in different places to hold a conference by audio. Furthermore, the audio conference system 1 has a function of automatically transmitting a response audio signal such as an audio signal indicating a conflict or an audio signal prompting the progress of the conference to each of the terminal devices of users participating in the conference. ing.

  FIG. 1 is a block diagram showing the configuration of the audio conference system 1. The audio conference system 1 includes a network 10 that interconnects a plurality of communication devices, a plurality of terminal devices 11 that are connected to the network 10, a headset 13 that is connected to each of the terminal devices 11, and a network 10. A connected server device 14 is provided.

  Each of the plurality of terminal devices 11 and the headset 13 is used by each of the users 19 of the audio conference system 1. The number of users who can participate in the conference using the audio conference system 1, that is, the number of the terminal devices 11 and the headsets 13 can be arbitrarily changed, and even if the configuration of the users participating during the conference is changed, Good.

  As shown in FIG. 1, when it is necessary to distinguish different users 19 and the terminal devices 11 and headsets 13 used by the users 19 from each other, the users 19-n, the terminal devices 11-n and the headsets, respectively. As in 13-n, “-n” is added to the end to distinguish them. However, “n” is an arbitrary natural number. Further, when there is no need to distinguish between the different user 19 and the terminal device 11 and the headset 13 used by the user 19, they are simply referred to as the user 19, the terminal device 11 and the headset 13, respectively.

  The network 10 includes one or more relay devices interconnected by wire or wireless, and relays data between different communication devices. The network 10 may be an open network that does not limit users such as the Internet, or may be any of an intranet, a LAN (Local Area Network) using a communication protocol other than the Internet protocol, and the like.

  The terminal device 11 transmits an audio signal indicating the voice of the user 19 to the server device 14 and is selected by the server device 14 from the server device 14 according to the audio signal indicating the audio of the other user 19 and the audio signal. This is a device for receiving a synthesized voice signal in which a response voice signal is mixed.

  The terminal device 11 amplifies an audio signal indicating a user's speech input from the microphone unit of the headset 13, and an A / A that converts the audio signal output from the microphone amplifier 111 from an analog signal to a digital signal. The D converter 112 and the audio signal transmission unit 113 that transmits the audio signal output from the A / D converter 112 to the server device 14 via the network 10 are provided.

  In addition, the terminal device 11 receives an audio signal receiving unit 114 that receives a synthesized audio signal from the server device 14 via the network 10, and converts the audio signal output from the audio signal receiving unit 114 from a digital signal to an analog signal D. / A converter 115 and a headphone amplifier 116 that amplifies the audio signal output from D / A converter 115 and outputs the amplified signal to the headphone section of headset 13.

  Further, the terminal device 11 transmits a control signal generated by the control unit 118 to the server device 14 via the network 10 and a control unit 118 that controls each component of the terminal device 11. A display unit 119 for notifying the user of a message or the like by displaying characters or graphics, an operation unit 120 for the user to perform various operations on the terminal device 11, and a control program for designating processing of the control unit 118 And a storage unit 121 that stores application programs and is used as a work area for other components. The storage unit 121 stores in advance a terminal ID that is an identifier for identifying the terminal device 11 in the network 10.

  The headset 13 includes a microphone unit that generates an analog audio signal indicating the speech of the user 19 and outputs the analog audio signal to the terminal device 11, and a headphone unit that converts the analog audio signal input from the terminal device 11 into sound and generates a sound. ing.

  The server device 14 receives an audio signal from each of the plurality of terminal devices 11, selects a response audio signal corresponding to the condition when the content of the received audio signal satisfies a predetermined condition, and receives it from the terminal device 11. This is a device that generates a synthesized speech signal by mixing the selected speech signal and the selected response speech signal, and then transmits the synthesized speech signal to each terminal device 11.

  The server device 14 receives an audio signal reception unit 141 that receives an audio signal indicating the speech of the user 19 from each terminal device 11 via the network 10, and receives various control signals from each terminal device 11 via the network 10. A control signal receiving unit 142, a voice signal transmitting unit 143 that transmits the generated synthesized voice signal to each terminal device 11 via the network 10, and a control program or application program that specifies processing of the control unit 145 and various data And a storage unit 144 that is also used as a work area for other components and a control unit 145 that controls each component of the server device 14.

  The storage unit 144 of the server device 14 stores in advance a server ID that is an identifier for identifying the server device 14 in the network 10. The storage unit 144 further includes a conference DB 1441 that is a database (hereinafter referred to as “DB”) that stores a plurality of conference data indicating conference schedules, participants, and the like, and user data that indicates user information of the audio conference system 1 A user DB 1442 storing a plurality of key phrases, a key phrase DB 1443 storing a plurality of key phrase data indicating key phrases used in response voice signal selection processing (described later), and a plurality of response condition data indicating selection conditions of response voice signals The response condition DB 1444 and the response sound signal group 1445 including a plurality of response sound signals are stored in advance.

  FIG. 2 is a diagram illustrating the contents of the conference DB 1441. The conference data included in the conference DB 1441 includes the conference date, the conference time, the conference agenda, the user ID group indicating the conference participant, and the response audio signal indicating the conflict, all of the participants' terminals. Each field includes a conflict mode indicating whether to transmit to the device or only to the terminal device of the current speaker, and a response style indicating the style of the response voice signal.

  The user ID group included in the conference data includes a plurality of user IDs of user data included in the user DB 1442, thereby indicating that the user indicated by the corresponding user data is a participant in the conference indicated by the conference data. Yes. The conference mode included in the conference data indicates that the response voice signal indicating the conference is transmitted to the terminal devices of all the participants, or “same to all” is transmitted, or is transmitted only to the terminal device of the current speaker. It is either “Speak to the speaker”. The response style included in the conference data is, for example, “formal”, “Frank”, etc., and indicates which of the response voice signals that indicate the same response message but have different wordings.

  FIG. 3 is a diagram illustrating the contents of the user DB 1442. The user data included in the user DB 1442 includes fields for a user ID for identifying each user, a user name, a user title, a user password, and a terminal ID of the terminal device 11 currently used by the user. ing. However, the terminal ID included in the user data is blank while the user 19 indicated by the user data is not using the voice conference system 1, and a process for establishing connection between the terminal device 11 and the server device 14 (described later). The server device 14 stores the terminal ID acquired from the terminal device 11 in the corresponding user data.

  FIG. 4 is a diagram illustrating the contents of the key phrase DB 1443. The key phrase data included in the key phrase DB 1443 has fields for the contents of the key phrase and the type of the key phrase. The content of the key phrase indicates the content of the statement such as “~”. The type is one of “confirmed”, “opinion”, “question”, etc., and what the key phrase is for the speaker Indicates whether it is of the kind that is said in

  FIG. 5 is a diagram showing the contents of the response condition DB 1444. The response condition data included in the response condition DB 1444 includes a condition for selecting a response voice signal, response identification data for specifying a response voice signal to be selected when the condition is satisfied, and response identification data. "Consultation" indicating whether or not the response content specified in the above means that it should be informed, even if the response content specified in the response identification data is being spoken by any user 19 And "XXX" indicating the contents of the wild card when the response identification data includes a wild card.

  For example, the condition “keyphrase: type = confirmed and silent duration = 2 seconds” included in the response condition data indicates that the key phrase DB 1443 has a keyphrase whose content of the field “type” is “conclusive”. It is detected during the utterance, and the silent state continues for 2 seconds thereafter. The condition of the response condition data includes, for example, “speech duration” indicating the duration of continuous speech by the same user 19, “conference duration” indicating the duration of the entire conference, and “meeting schedule” indicating the scheduled duration of the entire conference. Various parameters such as “time”, “speaking user: job title” indicating the title of the user 19 who has spoken last, and “number of simultaneous speakers” indicating the number of users 19 who are currently speaking can be included.

  Further, for example, the response identification data “007” included in the response condition data is a response specified by the message ID “007” among the response audio signals included in the response audio signal group 1445 when the condition is satisfied. It shows that an audio signal is selected. For example, the response identification data “RANDOM (001, 002, 003)” included in the response condition data is specified by each of the message IDs “001”, “002”, and “003” when the condition is satisfied. It shows that any one of the response voice signals to be selected is randomly selected.

  Further, for example, response identification data “XXX: full name + (if XXX: title = general staff then 017, otherwise 018)” included in the response condition data whose content of the field “XXX” is “random user” One of a response voice signal indicating the name of the user 19 selected at random among the plurality of participating users 19 and a response voice signal specified by each of the message IDs “017” and “018” are selected. And connect them in that order. In this case, if the position of the user 19 selected at random is a general employee, the response voice signal specified by the message ID “017” is selected. In other cases, the response voice specified by the message ID “018” is selected. The signal will be selected.

  In the field “XXX” included in the response condition data, when there are a plurality of users 19 who are currently speaking, “first earliest statement” indicating the user 19 who has started speaking first among the plurality of statements. Various things, such as “user”, “random user” indicating a randomly selected user 19, and “shortest speaking user” indicating a user 19 having the shortest accumulated speech time, are conceivable.

  FIG. 6 is a diagram showing a configuration of response audio signals included in response audio signal group 1445. However, FIG. 6 shows the configuration for convenience. Actually, the response voice signal group 1445 has a message ID or a user ID for the response message shown in FIG. 6 or the voice waveform data indicating the name. A plurality of response voice signals associated with each other are included.

  The response voice signal is roughly divided into a response voice signal indicating a response message and a response voice signal indicating the name of each user 19. The response voice signal indicating the response message is further divided into groups such as response styles “formal” and “frank”. These groups are groups specified in the field “response style” of the conference DB 1441. Response voice signals indicating response messages are distinguished from each other by a message ID. In the case of the example of FIG. 6, for example, the message ID “001” corresponds to a response voice signal indicating the voice of the response message “Yes, yeah”.

  The response voice signals indicating the name of the user 19 are distinguished from each other by the user ID. This user ID indicates a user ID in the user DB 1442. In the case of the example of FIG. 6, for example, the user ID “0425” corresponds to a response voice signal indicating a voice calling the name “Koji Sasaki”.

  Returning to FIG. 1, other data stored in the storage unit 144 of the server device 14 will be described. When an audio conference using the audio conference system 1 is started, the storage unit 144 stores conference duration data 1446 indicating the time from the start of the audio conference to the current time, and from the start of the audio conference for each user 19. Utterance duration data 1447 indicating the history of the utterance time until the end, utterance duration data 1448 indicating the time during which the silent state continues after any of the users 19 uttered, and the utterances of each user 19 Is stored in the data buffer 1449 for temporarily storing the audio signals individually for a predetermined time in the past. These data are temporarily stored in the storage unit 144 during the continuation of the audio conference, and are deleted from the storage unit 144 when the audio conference ends.

  The control unit 145 of the server device 14 detects a part indicating the speech of the user 19 from the voice signal by determining whether or not the level of the voice signal of each user 19 exceeds a predetermined threshold, for example, and detects the detection result. The speech signal detection unit 1451 delivered to the time measurement unit 1453, and the speech signal level of all users 19 is determined to determine whether or not the speech signal level is below a predetermined threshold, for example, thereby detecting a speechless state in the audio conference and measuring the detection result 1453. Mute signal detection unit 1452 to be handed over, the elapsed time from the start of the audio conference, the duration of each user 19's speech, and the duration of the silent state where none of the users 19 speak, Recorded in the storage unit 144 as conference duration data 1446, speech duration data 1447, and speech duration data 1448 A timer unit 1453 which includes a key phrase detection unit 1454 for detecting a portion indicating a key phrase from the audio signal of each user 19.

  In order to detect the key phrase, the key phrase detection unit 1454 performs, for example, an FFT (Fast Fourier Transform) process on the voice signal of the user 19, and extracts a frequency having a peak in the frequency distribution of the voice signal as a feature amount. Equally, each voice content included in the voice signal is recognized by comparing with the same kind of feature quantity of the voice waveform stored in advance. Subsequently, the key phrase detection unit 1454 detects the key phrase by comparing the recognized voice content with the data of the field “content” included in the key phrase DB 1443 and determining a match between them. The above is an example of the case where the key phrase detection unit 1454 uses the speech recognition method based on the feature quantity in the frequency domain, but the speech recognition method that the key phrase detection unit 1454 uses for detecting the key phrase is not limited to this, For example, any other method such as a speech recognition method based on a feature amount in the time-frequency domain or a speech recognition method based on a probability model may be used.

  The control unit 145 further determines whether the time measured by the time measuring unit 1453, the key phrase detected by the key phrase detection unit 1454, and the like satisfy the conditions included in the response condition DB 1444, and the response conditions satisfying the conditions The response audio signal selection unit 1455 which selects the response audio signal specified by the response identification data of the data from the response audio signal group 1445 and delivers it to the response audio signal mixing unit 1457, and each user 19 stored in the data buffer 1449 A speech audio signal mixing unit 1456 that mixes the audio signal and outputs it to the response audio signal mixing unit 1457 and the audio signal transmission unit 143, and a response audio signal delivered from the response audio signal selection unit 1455 and a speech audio signal mixing unit 1456 Mixing the synthesized voice signal And a response voice signal mixing unit 1457 to output No. to the transmitting unit 143.

[2. Operation of the audio conference system]
Hereinafter, the operation of the audio conference system 1 when a plurality of users 19 conduct an audio conference using the audio conference system 1 will be described. First, the user 19 participating in the audio conference operates his / her own terminal device 11 to establish a communication connection between the terminal device 11 and the server device 14. In establishing the communication connection, the user 19 inputs his / her user ID and password to the terminal device 11, and the control unit 118 of the terminal device 11 generates a control signal including the user ID and password input by the user and transmits the control signal. The data is transmitted to the server device 14 via the unit 117. The control signal includes the terminal ID of the terminal device 11.

  When receiving the control signal from the terminal device 11, the control unit 145 of the server device 14 searches the user DB 1442 (see FIG. 3) for user data including the user ID and password included in the received control signal, and In the field “terminal ID”, the terminal ID included in the control signal is stored. If the control unit 145 fails to retrieve user data, the server device 14 transmits a control signal instructing the terminal device 11 to display a message prompting the user ID and password to be re-input, and the correct combination of the user ID and password. The following operation is not performed until a control signal including is received from the terminal device 11.

  As described above, when a communication connection is established between the terminal device 11 and the server device 14 and the user device 19 authenticates the user 19 based on the user ID and the password, the user signal 19 indicates his / her speech. Is transmitted from the terminal device 11 to the server device 14, and at the same time, an audio signal indicating the speech of the user 19 of another terminal device 11 can be received from the server device 14.

  For example, a voice indicating the speech of the user 19-1 is converted into a voice signal by the microphone unit of the headset 13-1, and the voice signal is transmitted via the microphone amplifier 111, the A / D converter 112, and the voice signal transmission unit 113 of the terminal device 11-1. To the server device 14. The audio signal receiving unit 141 of the server device 14 receives the audio signal of the user 19-1 from the terminal device 11-1, and also receives the audio signals of other users 19 participating in the audio conference from the terminal device 11 of those users 19. And temporarily store them individually in the data buffer 1449. The audio signal temporarily stored in the data buffer 1449 is mixed by the speech audio signal mixing unit 1456 and output to the audio signal transmission unit 143 as a synthesized audio signal, and all users who participate in the audio conference from the audio signal transmission unit 143 It is transmitted to 19 terminal devices 11.

  The synthesized audio signal transmitted from the server device 14 is received by the audio signal receiving unit 114 in the terminal device 11-1, and the headphone unit of the headset 13-1 via the D / A converter 115 and the headphone amplifier 116. Is output. The headphone unit of the headset 13-1 converts the synthesized voice signal received from the terminal device 11-1 into voice and outputs it. As a result, the user 19-1 can transmit his / her speech to the other users 19 participating in the audio conference and can listen to the speech of the other user 19 to establish the audio conference.

  By the way, in addition to the above processing, the server device 14 further mixes an appropriate response voice signal as necessary with respect to the synthesized voice signal indicating the voice of the user 19 generated by the voice signal mixing unit 1456, and performs terminal processing. It has a function of transmitting to the device 11. Hereinafter, the operation of the server device 14 related to the function will be described.

  When the start time of the audio conference indicated by the fields “date” and “time zone” of the conference data included in the conference DB 1441 (see FIG. 2) is reached, the timer unit 1453 measures the elapsed time using the start time as a reference time. The measurement results are sequentially stored in the storage unit 144 as conference duration data 1446.

  Further, the speech signal detection unit 1451 constantly monitors the voice signal newly stored in the data buffer 1449, and after the level of the voice signal of a certain user 19 continues below a predetermined threshold for a predetermined time or more. When the predetermined threshold value is continuously exceeded for a predetermined time, speech start data indicating the start of speech is generated and delivered to the time measuring unit 1453. In addition, the speech signal detection unit 1451 generates speech end data indicating the end of speech when the level of the audio signal continuously falls below a predetermined threshold for a predetermined time or more after generation of the speech start data. To hand over. When the timing unit 1453 receives the speech start data and the speech end data from the speech signal detection unit 1451, the timer unit 1453 transmits the data indicating the speech time zone of the user 19 based on those data for each terminal device 11 that has transmitted the voice signal. The data is stored in the storage unit 144 as the duration data 1447. By the way, the method by which the speech signal detection unit 1451 identifies the start timing and the end timing of the speech signal portion of the audio signal is not limited to the above-described method. For example, the threshold value and time used for the determination are made variable according to a predetermined rule. Various methods are conceivable.

  The silent signal detection unit 1452 constantly monitors the audio signal newly stored in the data buffer 1449, and the audio signals of all the users 19 participating in the audio conference continue for a predetermined threshold for a predetermined time or longer. If it falls below, silent start data indicating the start of the silent state is generated and handed over to the timing unit 1453. In addition, the mute signal detection unit 1452 generates mute end data indicating the end of the mute state when the sound signal of any user 19 continuously exceeds a predetermined threshold for a predetermined time or longer after the mute start data is generated. Generated and handed over to the timer unit 1453. When the time counting unit 1453 receives the mute start data and mute end data from the mute signal detection unit 1452, the data indicating the elapsed time from the last time the mute start data is received to the present time based on the data is used. 1448 is stored in the storage unit 144. The point that the silence signal detection unit 1452 specifies the start timing and the end timing of the portion indicating silence in the audio signal is not limited to the above-described one, as in the case of the speech signal detection unit 1451 described above.

  The response voice signal selection unit 1455 constantly monitors the conference duration data 1446, the speech duration data 1447, and the silent duration data 1448 that are sequentially updated by the timing unit 1453 as described above, and these data are updated. Then, based on the updated data, search is made for response condition data included in the response condition DB 1444 (see FIG. 5) that satisfies the condition indicated by the field “condition”. When the search is successful, the response audio signal selection unit 1455 reads out the corresponding response audio signal from the response audio signal group 1445 based on the message ID and the user ID indicated by the response identification data included in the searched response condition data. . At that time, the response audio signal selection unit 1455 selects the field “response” of the conference data (see FIG. 2) corresponding to the currently held audio conference among the response audio signals included in the response audio signal group 1445 (see FIG. 6). The response audio signal is read from the group corresponding to the style indicated by “style”.

  On the other hand, the key phrase detection unit 1454 also constantly monitors the voice signal newly stored in the data buffer 1449, and as described above, the key phrase DB 1443 (FIG. 4) is detected. The key phrase indicated by the key phrase data included in (4) is detected. When the key phrase is detected by the key phrase detection unit 1454, the condition indicated by the field “condition” in the response condition data included in the response condition DB 1444 based on the type of the key phrase data corresponding to the detected key phrase. Search for items that satisfy. When the search is successful, the response audio signal selection unit 1455 reads out the corresponding response audio signal from the response audio signal group 1445 based on the message ID and the user ID indicated by the response identification data included in the searched response condition data. . At that time, the response audio signal selection unit 1455 reads the response audio signal from the group corresponding to the style indicated by the field “response style” of the conference data.

  For example, when the audio conference indicated by the first row conference data included in the conference DB 1441 illustrated in FIG. 2 is being performed, the second row included in the response condition DB 1444 illustrated in FIG. When the condition of the response condition data is satisfied, the response audio signal selection unit 1455 uses the message ID “007” from the response audio signal group included in the response style “formal” group in the response audio signal group 1445. The response voice signal to be identified, that is, the voice signal indicating the message "If you can answer the current question, please speak."

  When the response audio signal is read according to the response condition data (see FIG. 5) in which the field “interrupt” is “Yes”, the response audio signal selection unit 1455 responds to the read response audio signal. Delivery to the audio signal mixing unit 1457 is immediately started. On the other hand, when the response voice signal is read according to the response condition data field “interrupt” being “No”, the response voice signal selection unit 1455 is in a silent state based on the silence duration data 1448. When it is determined that the response voice signal is determined to be silent, when it is determined that it is not the silent state, it waits for the silent state to be entered, and the delivery of the read response voice signal to the response voice signal mixing unit 1457 is started. As a result, for example, a response message such as a summon is transmitted while none of the users 19 is speaking, while a response message that requires interruption, for example when controlling simultaneous speaking by multiple people, is any It is transmitted even if the user 19 is speaking.

  When the response audio signal mixing unit 1457 receives the response audio signal from the response audio signal selection unit 1455 as described above, the response audio signal mixing unit 1457 mixes the received response audio signal with the synthesized audio signal received from the speech audio signal mixing unit 1456, and as a result. The resultant synthesized speech signal is output to the speech signal transmission unit 143. As described above, the synthesized voice signal output from the response voice signal mixing unit 1457 is an elapsed time from the start of the conference and an accumulated speech time of each user 19 with respect to the voice signal indicating the speech of the user 19 participating in the audio conference. A voice signal indicating a message having a suitable content is mixed according to the duration of the silent state, the number of simultaneous speakers, and the like.

  By the way, when the field “conference mode” included in the conference data (see FIG. 2) corresponding to the currently performed audio conference is “conference to all”, the audio signal transmitting unit 143 is all participating in the audio conference. To the terminal device 11 of the user 19, the synthesized voice signal received from the response voice signal mixing unit 1457, that is, the voice signal indicating the conference voice mixed with the response voice signal is transmitted.

  On the other hand, when the field “conference mode” included in the conference data corresponding to the current audio conference is “conference to speaker”, the audio signal transmission unit 143 displays the field of response condition data (see FIG. 5). While the response audio signal is delivered from the response audio signal selection unit 1455 to the response audio signal mixing unit 1457 in accordance with what “Yes” is “Yes”, the terminal device of the user 19 who made the last statement by the message duration data 1447 11, the synthesized voice signal received from the response voice signal mixing unit 1457 is transmitted to the terminal device 11 of the other user 19, while the synthesized voice signal received from the voice signal mixing unit 1456, that is, the response voice. Send an unmixed audio signal. However, while the response audio signal is delivered from the response audio signal selection unit 1455 to the response audio signal mixing unit 1457 according to the response condition data field “Consideration” being “No”, the audio signal transmission unit 143 is used by all users. The synthesized speech signal received from the response speech signal mixing unit 1457 is transmitted to the 19 terminal devices 11. As a result, only the user 19 who is speaking (accurately, the user 19 who put “between” in the middle of speech) hears the voice of the other party, and other users 19 are disturbed by the voice of the other party. There is nothing.

  Further, the response audio signal selection unit 1455 prevents any of the users 19 from starting to speak during the playback of the speech such as a match so that the speech is not easily heard by the listener due to the speech such as the match. If necessary, playback of the response audio signal is stopped. More specifically, the response audio signal selection unit 1455 delivers the response audio signal to the response audio signal mixing unit 1457 according to the field “interrupt” of the response condition data (see FIG. 5) being “No”. When it is determined that the utterance duration data 1447 is updated and the utterance is started by any of the users 19, the response voice signal delivered at that time is not delivered to the response voice signal mixing unit 1457 thereafter.

  However, instead of immediately stopping delivery of the response voice signal when the response voice signal selection unit 1455 determines to start speaking, the response voice signal selection unit 1455 continues to deliver, for example, until the next syllable delimiter of the voice indicated by the response voice signal. Thereafter, the delivery may be stopped. In that case, for example, when the voice signal up to the first “yes” of the response message “I see, it is right” is delivered to the response voice signal mixing unit 1457 by the response voice signal selection unit 1455, If there is, the response voice signal selection unit 1455 hands over the voice signals up to “about” to the response voice signal mixing unit 1457, and then cancels the delivery of the remaining “sure” voice signals. As a result, it is possible to avoid the inconvenience that the response sound such as a conflict is unnaturally interrupted.

  On the other hand, when the response voice signal is delivered to the response voice signal mixing unit 1457 according to the field “interrupt” of the response condition data (see FIG. 5) being “Yes”, the user 19 starts to speak. If it is determined that the response audio signal is selected, the response audio signal selection unit 1455 passes the response audio signal to the response audio signal mixing unit 1457 to the end without stopping the transfer of the response audio signal that has been transferred at that time. Therefore, for example, an inconvenience that one of the users 19 starts to speak while the important response message is being transmitted to the user 19 and the response message is not transmitted to the end is avoided.

  As described above, according to the audio conference system 1, the response audio signal automatically transmitted from the server device 14 to each terminal device 11 without increasing the burden on any user 19 participating in the audio conference. , Revitalize meetings and improve the progress of proceedings.

  In the voice conference system 1 described above, the server apparatus 14 has been described as transmitting a synthesized voice signal obtained by mixing all voice signals received from each terminal apparatus 11 to each terminal apparatus 11. For the terminal device 11-n, only the audio signal received from the terminal device 11 other than the terminal device 11-n may be mixed and transmitted. In that case, there is no inconvenience that the speech of the user 19-n is pronounced like an echo from the headphone unit of the headset 13-n.

  In the audio conference system 1 described above, the conflict mode is specified by the conference data, and it has been described that the conflict mode cannot be made different for each terminal device 11. However, the conflict mode is determined according to each terminal device 11. Can be changed. Further, in the audio conference system 1 described above, the response style is also specified by the conference data, and it has been described that the response style cannot be made different for each terminal device 11, but the response style depends on each terminal device 11. Can be changed.

  Further, in the above audio conference system 1, the contents of the key phrase DB 1443 and the response condition DB 1444 can be changed by each user 19 or the administrator of the audio conference system 1. When 19 speaks, the response message corresponding to the comment may be reproduced. In that case, for example, the user 19 of the proceeding facilitator makes the voice conference system 1 make a speech for proceeding the proceeding that is difficult to say as his speech by speaking a key phrase registered in advance. Is possible and convenient.

  In the above audio conference system 1, the terminal device 11 and the server device 14 have been described as being connected to each other via the network 10. For example, the terminal device 11 and the server device 14 are connected to each other through a dedicated line. You may make it do. In the above audio conference system 1, the signal transmitted / received between the terminal device 11 and the server device 14 has been described as a digital signal, but is transmitted / received between the terminal device 11 and the server device 14. The signal may be an analog signal.

  In the voice conference system 1 described above, the server apparatus 14 is described as transmitting a response message to the terminal apparatus 11 as an audio signal. For example, the server apparatus 14 transmits the response message to the terminal apparatus 11 as text data. You may do it. In this case, the terminal device 11 that has received the text data of the response message can notify the user 19 of the response message by causing the display unit 119 to display the characters indicated by the text data. Further, the terminal device 11 that has received the text data of the response message generates a voice signal of the response message indicated by the text data by voice synthesis processing, and outputs the generated voice signal to the headset 13, thereby enabling the user 19. A response message may be notified.

  Further, for example, the response audio signal group 1445 is stored in each terminal device 11 instead of being stored in the server device 14, and the response audio signal selection unit 1455 of the server device 14 selects the response audio signal instead of selecting the response audio signal. The contents of the field “response identification data” included in the data specifying the audio signal, that is, the response condition data (see FIG. 5) satisfying the condition may be transmitted to the terminal device 11 as a control signal. In that case, the terminal device 11 reads the response audio signal from the response audio signal group 1445 stored in the storage unit 121 according to the response identification data included in the control signal received from the server device 14 and outputs the response audio signal to the headset 13. become.

  Further, in the audio conference system 1 described above, the server apparatus 14 has been described as providing a response audio signal transmission service for each terminal apparatus 11, but each terminal apparatus is not provided with, for example, a server apparatus. In a state in which the terminal devices 11 are peer-to-peer connected, each terminal device 11 has the same function as the server device 14 and synchronizes necessary data with each other so that each terminal device 11 selects and reproduces the response voice signal. You may make it perform.

  In the audio conference system 1 described above, the server apparatus 14 has been described as mixing the audio signal indicating the speech of each user 19 and the response audio signal, and then transmitting them to the terminal apparatus 11. A group of audio signals that can be distinguished from each other may be transmitted from 14 to the terminal device 11. In that case, since the audio signal received from the server device 14 is mixed in the terminal device 11, the user 19 changes the mixing balance between the audio signals in each terminal device 11 according to preference. A flexible audio signal can be used. The server device 14 may not include the audio signal received from the terminal device 11-n in the audio signal group transmitted to the terminal device 11-n.

  The terminal device 11 and the server device 14 may be realized by dedicated hardware, or may be realized by causing a general-purpose computer capable of inputting / outputting audio signals to execute processing according to an application program. .

It is the block diagram which showed the structure of the audio conference system concerning embodiment of this invention. It is the figure which illustrated the contents of conference DB concerning the embodiment of the present invention. It is the figure which illustrated the contents of user DB concerning the embodiment of the present invention. It is the figure which illustrated the contents of key phrase DB concerning the embodiment of the present invention. It is the figure which showed the content of response condition DB concerning embodiment of this invention. It is the figure which showed the structure of the response audio | voice signal contained in the response audio | voice signal group concerning embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Voice conference system, 10 ... Network, 11 ... Terminal device, 13 ... Headset, 14 ... Server apparatus, 111 ... Microphone amplifier, 112 ... A / D converter, 113 * 143 ... Voice signal transmission part, 114 * 141 ... Audio signal receiving unit, 115 ... D / A converter, 116 ... headphone amplifier, 117 ... control signal transmission unit, 118/145 ... control unit, 119 ... display unit, 120 ... operation unit, 121/144 ... storage unit, 142 ... Control signal receiving unit, 1441 ... conference DB, 1442 ... user DB, 1443 ... key phrase DB, 1444 ... response condition DB, 1445 ... response voice signal group, 1446 ... conference duration data, 1447 ... speech duration data, 1448 ... Mute duration data, 1449 ... Data buffer, 1451 ... Speech signal detector, 1452 ... Mute No. detection unit, 1453 ... the timing unit, 1454 ... key phrase detection unit, 1455 ... response audio signal selection unit, 1456 ... speaking voice signal mixing section, 1457 ... response voice signal mixing section.

Claims (10)

Storage means for storing a response voice signal indicating a response voice;
Input means for receiving an audio signal whose output source is each of a plurality of terminal devices;
Detecting means for detecting a signal indicating silence from the audio signal based on the level of each portion of the audio signal received by the input means;
A time measuring means for measuring a time during which a signal indicating mute detected by the detecting means continues;
Selecting means for selecting a response voice signal corresponding to the time measured by the time measuring means;
An audio signal processing apparatus comprising: output means for outputting a response audio signal selected by the selection means. The storage means further stores key phrase data indicating a key phrase,
Voice recognition means for specifying a phrase included in the speech indicated by the voice signal by performing voice recognition processing on the voice signal received by the input means;
Detecting means for detecting a key phrase indicated by the key phrase data from a phrase specified by the voice recognition means; and
The audio signal processing apparatus according to claim 1, wherein the selection unit selects a response audio signal corresponding to key phrase data indicating the key phrase detected by the detection unit. 2. The audio signal processing according to claim 1, wherein the selection unit selects a response audio signal corresponding to a terminal device that is an output source of the audio signal received by the input unit within a predetermined period in the past. apparatus. It further includes a time measuring means for measuring the elapsed time from a specific time point,
The audio signal processing apparatus according to claim 1, wherein the selection unit selects a response audio signal corresponding to the time measured by the time measuring unit. Further comprising time measuring means for measuring a time during which an audio signal whose output source is one terminal device received by the input means continues,
The audio signal processing apparatus according to claim 1, wherein the selection unit selects a response audio signal corresponding to the time measured by the time measuring unit. When the voice signal whose output source is one terminal apparatus is continuously received by the input means and the voice signal whose output source is another terminal apparatus is received by the input means, the selection means receives a predetermined response The audio signal processing apparatus according to claim 1, wherein an audio signal is selected. When a voice signal is received by the input means while a response voice signal is being output by the output means, the output means does not output at least a part of a non-output portion of the response voice signal being output. The audio signal processing apparatus according to claim 1. The voice according to claim 1, wherein the output means outputs a response voice signal using only a terminal device that is an output source of the voice signal received by the input means within a predetermined period in the past as a transmission destination. Signal processing device. The storage means stores response text data indicating a message character for response in addition to the response voice signal or instead of the response voice signal,
In addition to the response voice signal corresponding to the time measured by the time measuring means, or in place of the response voice signal corresponding to the time measured by the time measuring means, the selection means is a time measured by the time measuring means. Select response text data corresponding to,
The audio signal processing apparatus according to claim 1, wherein the output unit outputs response text data selected by the selection unit. Processing for storing a response voice signal indicating a response voice;
A process of receiving an audio signal having each of a plurality of terminal devices as an output source;
A process of detecting a signal indicating silence from the audio signal based on the level of each part of the received audio signal;
A process of measuring the duration of the detected signal indicating silence,
A process of selecting a response audio signal corresponding to the measured time;
A program for causing a computer to execute a process of outputting a selected response voice signal.

Images