JP2007298876A - Voice data recording and reproducing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus by which specified speaker's remark is efficiently heard, and utterance contents of the others to reach the remark are understood, from voice data in which conversation of a plurality of speakers is recorded. <P>SOLUTION: A network interface (I/F) 4 of a recording server 101 acquires the voice data of multi-point voice conference communicated in a network 100, and a recording section 3 records the voice data as minutes. The recording section 3 records a phoneme feature amount of each conference attendant. A feature amount data extracting section 2 extracts the phoneme feature amount and compares it with a phoneme feature amount of each conference attendant, when receiving a reproduction request from a personal computer 102. As a result, a recorded voice data of the specified speaker is extracted, and utterance speed conversion (high speed reproduction) of data other than the voice data of the speaker, is performed, and a digest data is generated and a streaming data is distributed. The personal computer 102 receives the streaming data and reproduces it as the digest data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an apparatus for recording and using audio such as conference audio.

  Conventionally, in a conference, it is common to create a minutes that records the speech of each speaker. When confirming such minutes, it takes time to listen back to all the statements, so it is often desirable to listen to only the important part (so-called digest).

  Therefore, a device that identifies each speaker from recorded voice data and converts it into text data by voice recognition processing (see, for example, Patent Document 1), and a device that creates a summary from this text (for example, Patent Document 2, Patent Document 3). Have been proposed).

  In addition, a recording information processing apparatus that divides voice data for each speaker and topic and performs a search process using these speakers and topics as a key search has been proposed (for example, see Patent Document 4). According to this recorded information processing apparatus, only a specific speaker's speech can be heard.

In addition, an information reproduction apparatus (see, for example, Patent Document 5) that rewinds for a specified time and reproduces at a slower speed than before when the content of a statement cannot be heard has been proposed.
Japanese Patent Laying-Open No. 2005-43628 JP 2004-20739 A JP 2002-99530 A Japanese Patent Laid-Open No. 2004-23661 JP-A-8-147794

The devices described in Patent Documents 1 to 3 perform speech recognition processing and extract text, but such processing requires high-level speech recognition technology.
Since the apparatus described in Patent Document 4 is configured to listen only to the speech of a specific speaker, it is difficult to hear the speech of others before and after, and it is difficult to grasp the outline of the entire conference.
The device described in Patent Document 5 has to be manually designated by the user when low speed reproduction is desired.

  In view of the above problems, the present invention can efficiently listen to a specific speaker's remarks from voice data in which conversations of a plurality of speakers are recorded, and the remarks of others up to that remark. An object of the present invention is to provide an apparatus capable of grasping the contents.

  An audio data recording / reproducing apparatus according to the present invention includes audio recording data recording pronunciations of a plurality of speakers and audio features of a specific speaker, audio feature extraction means for extracting audio features, The speaker that compares the voice feature amount of the specific speaker with the voice feature amount extracted by the voice feature amount extraction unit, and extracts the voice data recording section of the specific speaker from the voice data Extraction means, speech speed conversion means for performing processing for compressing voice data in a section other than the voice data recording section of the specific speaker on the time axis, and outputting voice data including the compressed section to the outside And an output means.

  In the present invention, the voice feature (formant, etc.) of a specific speaker is recorded in the recording means. The voice feature amount extraction unit extracts a voice feature amount from the recorded voice data. The speaker extraction unit compares the voice feature amount of a specific speaker recorded in advance with the extracted voice feature amount, and extracts a speech section of the specific speaker. For the sections other than the extracted speech section of the specific speaker, the speech data is compressed on the time axis to perform speech speed conversion (speeding up).

  Further, according to the present invention, the speech speed converting means compresses voice data in a section other than the voice data recording section of the specific speaker at a lower compression ratio in a section closer to the voice data recording section of the specific speaker. It is characterized by doing.

  In the present invention, the speech speed conversion means converts the speech speed by lowering the compression rate as it is closer to the utterance section of the specific speaker. Thereby, it is possible to grasp the content of another person's utterance more accurately until the specific speaker reaches the utterance.

  In addition, the present invention is characterized in that the speech speed conversion means performs a process of extending the voice data recording section of the specific speaker along the time axis.

  In the present invention, speech speed conversion (slowering) is performed by expanding the voice data on the time axis for the extracted speech section of the specific speaker. Since the voice of the specific speaker is played back slowly, it becomes easier to understand the speech of the specific speaker.

  The present invention further comprises the data acquisition means for acquiring voice data and speaker identification data for identifying a speaker of the voice data over time, and the recording means includes the voice data and the speaker identification data. And the voice feature amount of the specific speaker is recorded, and the speaker extracting means is configured to record the voice of the specific speaker based on at least one of the speaker identification data and the voice feature amount comparison result. A voice data recording section is extracted.

In the present invention, voice data and speaker identification data are acquired over time by the data acquisition means and recorded in the recording means. The speaker extracting means extracts the utterance section of a specific speaker from the speaker identification data. This makes it possible to more accurately extract the utterance section of a specific speaker. The present invention is an audio data recording / reproducing apparatus according to claim 4, which is connected to a sound emitting and collecting apparatus including a microphone array. The sound emission and collection device forms a plurality of sound collection beam signals having strong directivities in different directions based on the sound collection sound signals of the microphones of the microphone array, and the plurality of sound collection devices. Compare the beam signals, select the sound collecting beam signal with the strongest signal intensity, detect the direction corresponding to the selected sound collecting beam signal, the selected sound collecting beam signal as audio data, The detected direction is output as speaker identification data.

  In the present invention, the audio data recording / reproducing apparatus is connected to the sound emitting and collecting apparatus. This sound emission and collection device forms multiple sound collection beam signals from the sound collection sound signals of each microphone in the microphone array, selects the sound collection beam signal with the highest signal intensity, and supports the sound collection beam signals Detecting the direction to perform. The sound emitting and collecting apparatus outputs the selected sound collecting beam signal as voice data and outputs the detected direction as speaker identification data.

  According to the present invention, the speech feature amount of a specific speaker recorded in advance and the speech feature amount extracted from the speech data are compared to extract the speech segment of the specific speaker. The speech rate conversion (speeding up) of this section makes it unnecessary to perform advanced processing such as speech recognition (text extraction), while listening to the speech of a specific speaker, The contents can also be grasped. Further, it is not necessary for the user to manually specify the utterance section of a specific speaker.

With reference to the drawings, a minutes recording and reproducing system according to an embodiment of the present invention will be described.
FIG. 1 is a diagram showing a configuration of a minutes recording / reproducing system according to the present embodiment. The minutes recording / reproducing system includes an audio conference device 111, an audio conference device 112, a recording server 101, and a personal computer 102 connected to the network 100.

  The audio conference apparatus 111 and the audio conference apparatus 112 are respectively arranged at a point a and a point b that are separated from each other. A voice conference device 111 is arranged at the point a, and seven speakers A to G are seated in directions Dir11 to Dir16 and Dir18 with respect to the voice conference device 111 so as to surround the voice conference device 111, respectively. is doing. At the point b, the audio conference device 112 is arranged, and the five participants H to L surround the audio conference device 112 with respect to the audio conference device 112 in directions Dir21, Dir22, Dir24, Dir26, I am present at Dir28.

FIG. 2 is a block diagram showing the main configuration of the audio conference apparatus 111 of this embodiment. Note that the audio conference apparatus 112 has the same configuration as the audio conference apparatus 111, and a description thereof will be omitted.
The audio conference apparatus 111 includes a control unit 11, an input / output I / F 12, a D / A converter 14, a sound emission amplifier 15, a speaker SP1, a microphone MIC 101, a sound collection amplifier 16, an A / D converter 17, an echo cancellation circuit 20, and an operation. A unit 31 and a display unit 32 are provided.

  The control unit 11 controls the audio conference apparatus 111 in an integrated manner. The input / output I / F 12 is connected to the network 100, converts voice data from the partner apparatus input via the network 100 from network format data to a general voice signal, and passes through the echo cancellation circuit 20. Output to the D / A converter 14.

  The D / A converter 14 converts the digital sound output sound signal into an analog form, the sound output amplifier 15 amplifies the sound output sound signal and applies it to the speaker SP1, and the speaker SP1 converts the sound output sound signal into sound. And release it. Thereby, the voice of the conference person of the other party apparatus connected with the network is emitted to the conference person of the own apparatus.

  The microphone MIC101 picks up surrounding sound (direction Dir11 to direction Dir18) including the utterance sound of the conference person of its own device, converts it into an electric signal, and generates a collected sound signal. The sound collecting amplifier 16 amplifies the collected sound signal, and the A / D converter 17 converts the collected sound signal in an analog format into a digital format.

  The echo cancellation circuit 20 generates a pseudo regression signal based on the input voice signal by the adaptive filter 21 and subtracts the pseudo regression signal from the collected voice signal by the post processor 22. Thereby, the wraparound sound from the speaker SP to the microphone MIC is suppressed. The input / output I / F 12 converts the collected sound signal from the echo cancellation circuit 20 into sound data having a predetermined data length in a network format, and attaches sound collection time data obtained from the control unit 11 to the network 100. Output.

  With such a configuration, the multipoint conference can be performed by the audio conference apparatuses 111 and 112 connected to the network 100.

FIG. 3 is a block diagram showing the configuration of the recording server 101.
The recording server 101 includes a control unit 1, a feature data extraction unit 2, a recording unit 3, and a network I / F 4. The recording server 101 may be arranged at the same place as one of the audio conference apparatuses 111 and 112 or at a place completely different from these.

The control unit 1 performs overall control of the recording server 101 such as network communication control for the network I / F 4 and recording control for the recording unit 3.
The feature data extraction unit 2 extracts the voice feature amount of each conference participant from the voice data. The voice feature amount typically represents each speaker's formant, pitch, and the like, and is extracted from a frequency spectrum (power spectrum) obtained by Fourier transforming the voice data, and a cepstrum obtained by logarithmically transforming the power spectrum and then performing inverse Fourier transform. Prior to the conference, the feature data extraction unit 2 extracts the voice feature amount of each speaker and records it in the recording unit 3 as the voice feature amount (feature data) of each speaker. The conference participant (chairperson) registers in advance the identification information of each speaker (that is, which speaker each characteristic data belongs to). For example, when registering the voice feature amount of the speaker A in the recording unit 3, the chairperson asks the speaker A to speak and uses the operation unit 31 of the audio conference apparatus 111 to provide information on the speaker A (personal name, etc.). ) Is recorded in the recording unit 3. If the minutes recording / playback system of this embodiment is used in-house, if the meeting participants do not change, etc., the voice feature amount of each employee is recorded in the recording unit 3 in advance. Also good.

  Further, the feature data extraction unit 2 generates voice status data for identifying attributes of each part of the input voice data during the meeting. Here, the attribute includes a transmission source device of the audio data, a sound collection time at the device, and the like.

  The recording unit 3 is composed of a large-capacity magnetic disk or the like, and functionally includes an audio data recording unit 301, an audio status data recording unit 302, and a feature data recording unit 303. The audio data recording unit 301 sequentially records audio data input via the network I / F 4. At this time, a recording area for the audio conference apparatus 111 and a recording area for the audio conference apparatus 112 are prepared in the audio data recording unit 301, and audio data is recorded in the corresponding areas. The voice situation data recording unit 302 records voice situation data input from the feature data extraction unit 2, that is, information such as a voice data transmission source device and a sound collection time. The feature data recording unit 303 records the audio feature amount of the conference participant extracted by the feature data extraction unit 2 prior to the conference.

  Further, when the feature data extraction unit 2 is instructed to reproduce the voice data recorded from the personal computer 102, the feature data extraction unit 2 extracts a voice feature amount from the voice data recorded in the voice data recording unit 301, and records the feature data. The voice feature quantity recorded in the unit 303 is compared. As a result, it is possible to extract a recording section of voice data based on a speech of a specific speaker (for example, a chairperson). The recording server 101 uses the voice of the extracted recording section as it is, converts the voice other than this section to the speech speed (high speed playback), and distributes the stream as the conference digest data to the personal computer 102. Thereby, the personal computer 102 can perform digest reproduction of the minutes.

FIG. 4 is a block diagram showing the configuration of the personal computer 102.
The personal computer 102 includes a CPU 121, a storage unit 122 such as a hard disk, a display unit 123, an operation input unit 124, a network I / F 125, and a speaker 126.

  The CPU 121 controls the processing of a normal personal computer and functions as a voice data playback unit by reading and executing a playback application program (hereinafter referred to as a playback application) stored in the storage unit 122. The playback application transmits a playback request for the recorded audio data to the recording server 101 in response to the user's minutes playback request. Also, the playback application receives the user's speaker designation, designates a specific speaker from the voice data recorded in the recording server 101, and receives and plays the digest data. As a result, digest reproduction that can efficiently listen to the speech of the designated speaker is realized. Note that the user can also specify a plurality of speakers. In this case, the voice sections of the plurality of designated speakers are kept as they are, and the other sections are reproduced at high speed.

  The storage unit 122 includes a magnetic disk, a semiconductor memory, and the like, and stores a playback application and is used as a working unit when the CPU 121 executes each function.

  The display unit 123 is configured by a liquid crystal display or the like, and when a playback application is executed by the CPU 121, the playback application is activated and display screen information is given from the CPU 121 to display various screens.

  The operation input unit 124 includes a keyboard and a mouse, receives a user operation input, and gives it to the CPU 121. For example, by moving the cursor on the display screen with the mouse and clicking the mouse at the corresponding position, the click information is given to the CPU 121, and the CPU 121 determines the operation input content from the click position and the click situation and performs a predetermined reproduction process. I do.

  The network I / F 125 connects the personal computer 102 to the network 100 and receives a control signal from the CPU 121 and audio data (streaming data) from the recording server 101 in accordance with communication control from the CPU 121.

  The speaker 126 emits sound data according to the control of the CPU 121.

Next, the recording flow of the recording server 101 will be described with reference to FIG.
FIG. 5 is a flowchart showing a recording process flow of the recording server 101. Note that the audio feature amount of each conference participant is registered in the recording unit 3 before the recording process flow is performed.
The recording server 101 monitors voice data communication over the network 100. The recording server 101 starts recording upon detecting the conference start trigger (S1 → S2). At this time, the conference start trigger can be obtained by detecting that audio data is communicated to the network 100, or each audio conference device 111, 112 can generate a conference start pulse by pressing the conference start switch. It can be obtained by detection. Further, if the recording server 101 is provided with a recording start switch, it can be detected by pressing the recording start switch.

  When recording is started, the recording server 101 (control unit 1) acquires the recording start time by a built-in timer or the like, and gives it to the feature data extraction unit 2. The feature data extraction unit 2 stores this recording start time as a title of one audio data file (S3).

  The network I / F 4 acquires audio data communicated through the network 100 and provides it to the control unit 1, the feature data extraction unit 2, and the recording unit 3, and the recording unit 3 sequentially stores the audio data (S4).

  At this time, the control unit 1 acquires device data and time data from the information added to the audio data acquired by the network I / F 4 (S5), and gives the device data to the recording unit 3. The recording unit 3 sequentially records audio data in the audio data recording unit 301 for each device in accordance with the device data acquired from the control unit 1.

  In addition, the control unit 1 acquires device data and time data from the audio data, and gives them to the feature data extraction unit 2 (S5). The feature data extraction unit 2 generates voice status data from the device data and the time data, and gives the voice status data to the recording unit 3. The recording unit 3 records the voice situation data from the feature data extraction unit 2 in the voice situation data recording unit 302 (S6).

  Such generation / recording processing of voice status data and recording processing of voice data are repeated until a recording end trigger is detected. When a recording end trigger or a recording pause trigger is detected (S7), the control unit 1 gives a recording end control instruction to the feature data extraction unit 2. The recording end trigger is obtained by detecting pressing of the conference end switch or powering off of the audio conference apparatuses 111 and 112 connected to the network 100. The feature data extraction unit 2 generates and records final voice situation data, and also groups grouping instruction data for grouping each voice situation data recorded in advance in the voice situation data recording unit 302 with a title acquired at the start of recording. Generated and recorded in the voice status data recording unit 302 (S8).

By performing such a configuration and processing, the audio data recording unit 301 records audio data that is continuous over time for each device, and records the minutes.
Next, the playback flow of the recording server 101 and the personal computer 102 will be described with reference to the flowcharts of FIGS.
FIG. 6 is a flowchart showing a playback processing flow of the personal computer 102.
First, the CPU 121 determines whether or not the user has input a minutes search keyword using the operation input unit 124 (S21). The minutes search keyword is, for example, a meeting date and time, a meeting name, a device name, or the like. The CPU 121 transmits the input keyword to the recording server 101 (S22). In the recording server, the minutes are searched from the keyword, and the corresponding result is received by the personal computer 102 (S23). The CPU 121 displays the received result on the display unit 123 (S24). Thereby, the user can check the list of recorded minutes and can designate the minutes to be reproduced.

  Thereafter, it is determined whether or not the user has input a reproduction request (S25). If no reproduction request is input, the process is repeated from S21. The reproduction request is input when the user designates the minutes to be reproduced with a mouse or the like from the search result displayed on the display unit 123. The playback request includes designation information for normal playback (no speech speed conversion) and digest playback. In the case of digest reproduction, speaker designation information indicating which speaker's utterance is preferentially listened (when there are a plurality of speaker designations, priority order information of a plurality of speakers is also included) is input.

  When the reproduction request is input, the CPU 121 transmits the reproduction request and the speaker designation information to the recording server 101 (S26). The recording server 101 reads the designated minutes, performs analysis processing (described later) based on the speaker designation information, and generates streaming data. This streaming data is received by the personal computer 102 (S27). The CPU 121 reproduces the received streaming and emits sound from the speaker 126 (S28).

  Thereafter, the CPU 121 determines whether or not the user has input a reproduction change instruction (S29). The reproduction change instruction includes, for example, a pause instruction, a fast-forward instruction, and the like. The CPU 121 performs a reproduction change process in response to the input reproduction change instruction (S30). For example, if a pause instruction is input, playback of streaming data is paused and sound emission is stopped.

  Thereafter, the CPU 121 determines whether or not the user has input an end instruction (S31). If there is an end instruction, the end instruction information is transmitted to the recording server 101 (S32). If there is no end instruction, the process is repeated from the reception of the streaming data. Unlike the pause instruction described above, the end instruction in S32 is an instruction to stop receiving streaming data and end this reproduction processing flow.

FIG. 7 is a flowchart showing the playback processing flow of the recording server 101.
The flow in FIG. 6A is triggered by the transmission of the minutes search keyword from the personal computer 102. When receiving the minutes search keyword from the personal computer 102, the control unit 1 searches the voice status data recording unit 302 of the recording unit 3 for the minutes corresponding to the keyword (S51). As described above, the minutes search keyword is a meeting date and time, a device name, and the like. The control unit 1 returns the meeting date and time, meeting name, device name, etc. of the minutes corresponding to the keyword as a search result (S52). As a result, the search result is displayed on the display unit 123 of the personal computer 102.

  The flow in FIG. 5B is triggered by the transmission request and speaker designation information transmitted from the personal computer 102. In the case of not digest playback (normal playback), the minutes are simply read out from the recording unit 3 and distributed (streaming distribution), and the description thereof is omitted. When the control unit 1 receives the reproduction request and the speaker designation information from the personal computer 102, the control unit 1 gives them to the feature data extraction unit 2. The feature data extracting unit 2 reads out the feature data of the designated speaker from the speech feature amounts of the conference participants recorded in the feature data recording unit 303 of the recording unit 3 from the speaker designation information (S71). . The feature data extraction unit 2 reads the minutes specified in the reproduction request from the audio data recording unit 301 of the recording unit 3 (S72). A voice feature is extracted from the read voice data, and analysis processing is performed for comparison with the feature data of the specific speaker read in S71 (S73). Note that the minutes are read for a predetermined time length (for example, 2 to 3 seconds), and analysis processing for only a few seconds is first performed.

  The feature data extraction unit 2 extracts a section having a speech feature amount matching the feature data of a specific speaker from the read speech data by a method such as pattern matching. This section is set as an utterance section of a specific speaker. The feature data extraction unit 2 converts the speech speed of sections other than the speech section of the specific speaker (S74), and generates streaming data (S75). The conversion speed is, for example, double speed. Further, the conversion speed may be variable. For example, a portion close to the utterance section of a specific speaker is set to a low speed (such as 1.5 times) or a normal speed, and is set to a high speed as the distance from the utterance section of the specific speaker increases.

  In addition, the speech speed conversion process is performed not only by reading and outputting the voice data at double speed but also as follows. That is, in the speech speed conversion process, the voice data (voice signal) is cut into one-cycle waveform, and the number of periodic waveforms of the voice signal is reduced by generating a new periodic waveform by synthesizing one section before and after each periodic waveform. Thus, the signal is compressed while maintaining the pitch.

  FIG. 8A is a flowchart showing the procedure of speech speed conversion processing. FIG. 2B is a diagram for explaining a compression method. In FIG. 9A, first, the number of samples in one cycle of the head portion of the audio signal (for example, sampling frequency × 1 / signal frequency) is detected (S91). Two periodic waveforms, which are sample data for one period, are taken out and, as shown in FIG. 5B, an attenuation wave is created by multiplying the first periodic waveform A by an attenuation gain coefficient, An increasing wave is created by multiplying the second periodic waveform B by an increasing gain coefficient (S92). Then, by adding and synthesizing these, a periodic waveform having an intermediate shape between A and B is synthesized (S93). This synthesized waveform is output in place of the periodic waveforms A and B (S94). Instead of the periodic waveform A and the periodic waveform B, an acoustically natural time axis compression is performed by outputting a composite waveform.

  Also, the conversion speed can be made variable by defining the cycle for performing the speech speed conversion processing. For example, as shown in FIG. 9A, it can be converted to double speed by synthesizing two periodic waveforms every two cycles, and every three cycles as shown in FIG. 9B. By combining two waveforms, it can be converted to 1.5 times speed. Note that in the process of S73 of FIG. 7, the speech speed may be converted for the utterance section of a specific speaker. In this case, in order to help the speaker understand the content of the utterance, a process of expanding the voice may be performed. In the case of decompression processing, the above-described synthesized waveform may be inserted between the periodic waveform A and the periodic waveform B in FIG. 8B to increase the number of periodic waveforms of the audio signal. In this case, only the head portion (for example, several hundred msec) of the section may be expanded and the subsequent portion may be output at the normal speed, and may not be expanded more than necessary. Further, the head portion may be expanded and the subsequent portion may be compressed.

  Note that sections other than a specific speaker's speech section may be skipped (compressed to infinite magnification). Further, as feature data, a voice feature amount of a meaningless utterance (for example, a utterance such as “e ~ a ~ a”) is recorded in the recording unit 3, and in the process of S72, this utterance section is recognized by voice recognition. It is also possible to extract only meaningless utterances.

  Through the above processing, the streaming data after the speech speed conversion is generated. In FIG. 7B, the feature data extraction unit 2 transmits the streaming data after the speech speed conversion to the personal computer 102 (S76). The processes of S72 to S76 are repeated until the end instruction information is received from the personal computer 102 (S77 → S72). If the end instruction information is received from the personal computer 102, the operation is finished (S77 → END). Since the processes of S72 to S76 are respectively performed for audio data of a predetermined time length (for example, 2 to 3 seconds), the personal computer 102 waits for reception until the first several seconds of streaming data is distributed. The streaming data is generated by a task different from the reproduction processing of the personal computer 102. Therefore, when the first distribution starts, the minutes digest can be listened to without waiting time thereafter.

  The minutes recording / reproducing system of the present invention can be applied as follows. FIG. 10 is a block diagram illustrating a configuration of an audio conference apparatus according to an application example. This audio conference apparatus is used in place of each of the audio conference apparatuses 111 and 112 in FIG. In addition, the same code | symbol is attached | subjected about the component which is common in the audio conference apparatus 111 shown in FIG. 2, and the description is abbreviate | omitted.

  The audio conference apparatus according to the application example includes a control unit 11, an input / output I / F 12, a sound output directivity control unit 13, a D / A converter 14, a sound output amplifier 15, speakers SP1 to SP16, microphones MICs 101 to 116, 201 to 216, a sound collection amplifier 16, an A / D converter 17, a sound collection beam generation unit 18, a sound collection beam selection unit 19, an echo cancellation circuit 20, an operation unit 31, and a display unit 32.

  The control unit 11 of the voice conference apparatus converts voice data from the partner apparatus input from the input / output I / F 12 from network format data to a general voice signal and emits the sound through the echo cancel circuit 20. While outputting to the directivity control part 13, the azimuth | direction data attached to the input audio | voice signal is acquired, and sound emission control with respect to the sound emission directivity control part 13 is performed.

  The sound emission directivity control unit 13 generates sound emission sound signals for the speakers SP1 to SP16 according to the sound emission control contents. The sound output sound signal to the speakers SP1 to SP16 is formed by performing signal control processing such as delay control and amplitude control on the input sound data. The D / A converter 14 converts the sound output sound signal in the digital format into an analog format, the sound output amplifier 15 amplifies the sound output sound signal and applies it to the speakers SP1 to SP16, and the speakers SP1 to SP16 output the sound output sound signal. The sound is converted to sound. Thereby, the voice of the conference person of the other party apparatus connected with the network is emitted to the conference person of the own apparatus.

  The microphones MIC 101 to 116 and 201 to 216 collect ambient sounds including the utterance sound of the conference person of the own device, convert them into electric signals, and generate collected sound signals.

  The collected sound beam generator 18 performs delay processing on the collected signals of the microphones MICs 101 to 116 and 201 to 216, and generates collected sound beam sound signals MB1 to MB8 having strong directivity in a predetermined direction. The collected sound beam audio signals MB1 to MB8 are set so as to have strong directivity in different directions. If the voice conference apparatus of FIG. 10 is replaced with the voice conference apparatus 111 of FIG. 1, MB1 is set to the direction Dir11, MB2 is set to the direction Dir12, MB3 is set to the direction Dir13, MB4 is set to the direction Dir14, and MB5 is set to the direction Dir15. In addition, MB6 is set to the direction Dir16, MB7 is set to the direction Dir17, and MB8 is set to the direction Dir18. On the other hand, if the voice conference apparatus of FIG. 10 is replaced with the voice conference apparatus 112 of FIG. 1, MB1 is set to the direction Dir21, MB2 is set to the direction Dir22, MB3 is set to the direction Dir23, MB4 is set to the direction Dir24, and MB5 is set. In the direction Dir25, MB6 is set in the direction Dir26, MB7 is set in the direction Dir27, and MB8 is set in the direction Dir28.

  The collected sound beam selection unit 19 compares the signal strengths of the collected sound beam sound signals MB1 to MB8, selects the collected sound beam sound signal having the highest intensity, and outputs it to the echo cancel circuit 20 as the collected sound beam sound signal MB. To do. The collected sound beam selection unit 19 detects the direction Dir corresponding to the selected collected sound beam sound signal MB and supplies the detected direction to the control unit 11. The input / output I / F 12 converts the collected sound beam sound signal MB from the echo cancel circuit 20 into sound data having a predetermined data length in a network format, and attaches the azimuth data and sound collection time data obtained from the control unit 11. And output to the network 100.

Next, a recording flow of the recording server 101 according to the application example will be described with reference to FIG.
FIG. 11 is a flowchart showing a recording process flow of the recording server 101.
The recording server 101 monitors voice data communication over the network 100. When the recording server 101 detects the conference start trigger, the recording server 101 starts recording (S101 → S102).

  When recording is started, the recording server 101 (control unit 1) acquires the recording start time and gives it to the feature data extraction unit 2. The feature data extraction unit 2 stores the recording start time as one audio status data title (S103).

  The network I / F 4 acquires the audio data communicated through the network 100 and gives it to the recording unit 3, which sequentially stores the audio data (S104).

  At this time, the control unit 1 acquires device data, azimuth data, and time data from the audio data acquired by the network I / F 4 and provides the device data to the recording unit 3. The recording unit 3 sequentially records audio data in the audio data recording unit 301 for each device in accordance with the device data acquired from the control unit 1.

  In addition, the control unit 1 acquires device data, direction data, and time data from the voice data, and gives them to the feature data extraction unit 2 (S105). The feature data extraction unit 2 temporarily stores the device data, the orientation data, and the time data.

  This process is repeated until the control unit 1 detects a change in azimuth data for each device. When the control unit 1 detects a change in azimuth data (S106), the control unit 1 performs session end process control to extract feature data. It gives to part 2 (S107). The feature data extraction unit 2 generates audio status data including device data, direction data, and start time data of the corresponding audio data group, and gives the audio data group to the recording unit 3 in order to associate the audio data group including the same direction data. . The recording unit 3 records the voice situation data from the feature data extraction unit 2 in the voice situation data recording unit 302 (S108). Such generation and recording processing of voice status data and recording processing of voice data are repeated until a recording end trigger is detected, and voice status data is generated and recorded every time the azimuth data changes.

  When the recording end trigger is detected (S110), the control unit 1 gives a recording end control instruction to the feature data extraction unit 2. The feature data extraction unit 2 generates and records final voice situation data, and also groups grouping instruction data for grouping each voice situation data recorded in advance in the voice situation data recording unit 302 with a title acquired at the start of recording. It is generated and recorded in the voice situation data recording unit 302 (S111).

  The sound data recording unit 301 records sound data that is continuous over time for each apparatus. At this time, the voice data is classified according to the direction data by the voice feature data recorded in the voice situation data recording unit 302. The conference participant (chairperson) registers in advance the relationship between the orientation data and each speaker (that is, which speaker exists in each orientation). As a result, the voice data is divided for each speaker.

  If it is voice data of point a, voice data of speaker A, voice data of speaker B, voice data of speaker C, voice data of speaker D, voice data of speaker E, voice data of speaker F , The voice data of the speaker G and the voice data of silence (noise) not designated by the speaker. Each segmented audio data is associated with segment start time data. If there is no speech, it is not recorded in the voice feature data.

  Similarly, if the voice data is at point b, the voice data of speaker H, the voice data of speaker I, the voice data of speaker J, the voice data of speaker K, the voice data of speaker L, and the speaker And voice data of silence (noise) that is not designated, and start time data for each section is associated. Also in this case, if there is no speech, it is not recorded in the voice feature data.

  In this way, by using the configuration and processing of the application example, it is possible to record the minutes with the audio feature data corresponding to each conference participant. Since the time data is also associated, the minutes can be recorded including the speech status of each conference participant. Thereby, when performing digest reproduction | regeneration processing, the speech of the designated speaker can be extracted accurately.

  In the application example, the recording server 101 extracts a voice feature amount from the read voice data at the time of the analysis process (S73) in FIG. 7B, and performs a specific process according to the speaker classification recorded in the voice data. Determine the speaker's utterance interval. For a specific speaker's utterance section extracted from the speech feature, it is further determined whether the speaker section recorded in the speech data matches the specific speaker's section. Determined as the speaker's utterance interval. If they do not match, the speech section of another speaker is set. Thereby, the utterance section of a specific speaker is extracted with high accuracy.

  In this embodiment, a case where a multipoint conference is performed with a plurality of audio conference apparatuses connected to the network is shown, but the same operation and effect can be obtained even when only a single audio conference apparatus is used. be able to.

  In the present embodiment, an example of recording audio data has been described. However, it is possible to further record image (still image, moving image) data by installing a camera or the like in the audio conference apparatus. The image data may be recorded in synchronization with the audio data, and at the time of reproduction, the image data may be displayed in synchronization with the audio data.

Configuration diagram of a minutes recording and playback system according to an embodiment of the present invention The block diagram which shows the main structures of the audio conference apparatus 111 of this embodiment. The block diagram which shows the main structures of the recording server 101 of this embodiment Block diagram showing the main configuration of the personal computer 102 The flowchart which shows the recording processing flow of the recording server 101 The flowchart which shows the reproduction | regeneration processing flow of the personal computer 102 A flowchart showing a playback processing flow of the recording server 101 Diagram showing speech speed conversion processing Diagram showing speech speed conversion process for variable speed The block diagram which shows the structure of the audio conference apparatus which concerns on an application example The flowchart which shows the recording processing flow of the recording server 101 which concerns on an application example.

Explanation of symbols

100-network 101-recording server 1-control unit 2-characteristic data generation unit 3-recording unit 4-network I / F
111, 112-voice conference equipment

Claims (5)

Recording means for recording sound data of pronunciations of a plurality of speakers, and sound features of a specific speaker;
Voice feature quantity extraction means for extracting voice feature quantities;
Speaker extraction for comparing the voice feature quantity of the specific speaker with the voice feature quantity extracted by the voice feature quantity extraction unit and extracting the voice data recording section of the specific speaker from the voice data Means,
Speech speed conversion means for performing processing for compressing voice data of a section other than the voice data recording section of the specific speaker on a time axis;
Output means for outputting audio data including a compressed section to the outside;
An audio data recording / reproducing apparatus comprising:   The speech speed converting means compresses voice data in a section other than the voice data recording section of the specific speaker at a lower compression ratio in a section closer to the voice data recording section of the specific speaker. The audio data recording / reproducing apparatus according to claim 1.   The voice data recording / reproducing apparatus according to claim 1, wherein the speech speed conversion unit performs a process of extending a voice data recording section of the specific speaker along a time axis. Including the data acquisition means for acquiring voice data and speaker identification data for identifying a speaker of the voice data over time;
The recording means records the voice data, the speaker identification data, and the voice feature amount of the specific speaker,
The said speaker extraction means extracts the audio | voice data recording area of the said specific speaker based on at least any one of the said speaker identification data or the comparison result of an audio | voice feature-value. Or the audio | voice data recording / reproducing apparatus of Claim 3. The audio data recording / reproducing device according to claim 4, wherein the audio data recording / reproducing device is connected to a sound emission and collection device including a microphone array.
The sound emission and collection device forms a plurality of sound collection beam signals having strong directivities in different directions based on the sound collection sound signals of the microphones of the microphone array, and the plurality of sound collection beam signals In comparison, the sound collecting beam signal having the strongest signal intensity is selected, the direction corresponding to the selected sound collecting beam signal is detected, the selected sound collecting beam signal is set as audio data, and the detected signal is detected. An audio data recording / reproducing apparatus, characterized in that the direction is output as speaker identification data.

Images