JP6589040B1 - Speech analysis apparatus, speech analysis method, speech analysis program, and speech analysis system - Google Patents

Abstract

An object of the present invention is to provide a voice analysis device, a voice analysis method, a voice analysis program, and a voice analysis system that can output information for performing analysis based on temporal changes in the amount of speech of participants in a discussion. An audio analysis apparatus 100 according to an embodiment of the present invention includes an audio acquisition unit 112 that acquires audio uttered by a plurality of participants, and an analysis unit that specifies the amount of speech for each of a plurality of participants in the audio. 114, a section setting unit 115 that sets a section in speech based on an input from the user, a graph obtained by accumulating temporal changes in speech amounts of a plurality of participants, and information indicating the sections in the graph are output. And an output unit 116.

Description

  The present invention relates to a speech analysis apparatus, a speech analysis method, a speech analysis program, and a speech analysis system for analyzing speech.

  A Harkness method (also referred to as a Harkness method) is known as a method for analyzing discussions in group learning and meetings (see, for example, Non-Patent Document 1). In the Harkness method, the transition of each participant's speech is recorded as a line. As a result, each participant's contribution to the discussion and the relationship with others can be analyzed. The Harkness method can be effectively applied to active learning where students learn independently.

Paul Sevigny, “Extreme Discussion Circles: Preparing ESL Students for“ The Harkness Method ””, Polyglossia, Center for Language Education, Ritsumeikan Asia Pacific University, October 2012, No. 23, p. 181-191

  However, since the Harkness method shows the tendency of the speech of the whole period from the start to the end of the discussion, it cannot show the change of the speech amount of each participant along the time series. For this reason, there is a problem that it is difficult to analyze based on changes in the amount of speech of each participant.

  The present invention has been made in view of these points, and a speech analysis device, a speech analysis method, a speech analysis program, and speech analysis that can output information for performing analysis based on temporal changes in the amount of speech of participants in a discussion The purpose is to provide a system.

  The voice analysis device according to the first aspect of the present invention includes an acquisition unit that acquires voices uttered by a plurality of participants, and an analysis unit that specifies a speech amount for each of the plurality of participants in the voices. A section setting unit for setting a section in the voice based on an input from a user, a graph obtained by accumulating time changes in the amount of speech of the plurality of participants, and information indicating the section in the graph And an output unit for outputting.

  The output unit may output a position on the graph corresponding to a time switched between the two sections as information indicating the section.

  The section setting unit is based on at least one of an operation in a communication terminal that communicates with the voice analysis device, an operation in a sound collection device that acquires the voice, and a predetermined sound included in the voice, A section may be set.

  The output unit may output the graph obtained by accumulating temporal changes in the speech amount in order of increasing degree of variation in the speech amount calculated for each of the plurality of participants.

  The output unit outputs the graph obtained by accumulating temporal changes in the amount of speech for each section in descending order of the degree of variation in the amount of speech for each of the sections calculated for each of the plurality of participants. Also good.

  The output unit may output a plurality of graphs for the same section set for a plurality of the voices.

  In addition to the information indicating the graph and the section, information indicating an event that has occurred within the time of the sound may be output on the graph.

  The analysis unit may specify a value obtained by dividing the length of time during which a participant speaks within a predetermined time window by the length of the time window as the amount of speech.

  In the speech analysis method according to the second aspect of the present invention, the processor acquires the speech uttered by a plurality of participants, and specifies the amount of speech for each of the plurality of participants in the speech. And a step of setting a section in the voice based on an input from a user, a graph in which temporal changes in the amount of speech of the plurality of participants are stacked, and information indicating the section in the graph And executing a step.

  The speech analysis program according to the third aspect of the present invention includes a step of acquiring, in a computer, speech uttered by a plurality of participants, and a step of specifying a speech amount for each of the plurality of participants in the speech. And a step of setting a section in the voice based on an input from a user, a graph in which temporal changes in the amount of speech of the plurality of participants are stacked, and information indicating the section in the graph And executing a step.

  A speech analysis system according to a fourth aspect of the present invention includes a speech analysis device and a communication terminal capable of communicating with the speech analysis device, and the communication terminal includes a display unit that displays information, and the speech The analysis device is based on an acquisition unit that acquires voices uttered by a plurality of participants, an analysis unit that specifies a speech amount for each of the plurality of participants in the voice, and an input from a user, An output unit configured to display on the display unit a section setting unit that sets a section in the voice, a graph obtained by accumulating temporal changes in the amount of speech of the plurality of participants, and information indicating the section in the graph And having.

  According to the present invention, it is possible to output a change in the amount of speech of each participant along a discussion time series.

It is a mimetic diagram of a voice analysis system concerning this embodiment. It is a block diagram of the speech analysis system concerning this embodiment. It is a schematic diagram of the audio | voice analysis method which the audio | voice analysis system which concerns on this embodiment performs. It is a front view of the display part of the communication terminal which is displaying the setting screen. It is a front view of the display part of the communication terminal which is displaying the message amount screen. It is a front view of the display part of the communication terminal which is displaying the message amount screen. It is a front view of the display part of the communication terminal which is displaying the message amount screen. It is a front view of the display part of the communication terminal which is displaying the area extraction screen. It is a front view of the display part of the communication terminal which is displaying the message amount screen. It is a sequence diagram of the audio | voice analysis method which the audio | voice analysis system which concerns on this embodiment performs.

[Outline of Speech Analysis System S]
FIG. 1 is a schematic diagram of a speech analysis system S according to the present embodiment. The voice analysis system S includes a voice analysis device 100, a sound collection device 10, and a communication terminal 20. The number of the sound collecting devices 10 and the communication terminals 20 included in the voice analysis system S is not limited. The voice analysis system S may include other servers and terminals.

  The voice analysis device 100, the sound collection device 10, and the communication terminal 20 are connected via a network N such as a local area network or the Internet. At least some of the voice analysis device 100, the sound collection device 10, and the communication terminal 20 may be directly connected without going through the network N.

  The sound collection device 10 includes a microphone array including a plurality of sound collection units (microphones) arranged in different directions. For example, the microphone array includes eight microphones arranged at equal intervals on the same circumference in a horizontal plane with respect to the ground. The sound collection device 10 transmits the sound acquired using the microphone array to the sound analysis device 100 as data.

  The communication terminal 20 is a communication device capable of performing wired or wireless communication. The communication terminal 20 is a portable terminal such as a smartphone terminal or a computer terminal such as a personal computer. The communication terminal 20 receives the analysis condition setting from the analyst and displays the analysis result by the voice analysis device 100.

  The voice analysis device 100 is a computer that analyzes the voice acquired by the sound collection device 10 by a voice analysis method described later. In addition, the voice analysis device 100 transmits the result of the voice analysis to the communication terminal 20.

[Configuration of speech analysis system S]
FIG. 2 is a block diagram of the speech analysis system S according to the present embodiment. In FIG. 2, arrows indicate main data flows, and there may be data flows not shown in FIG. In FIG. 2, each block represents a functional unit configuration, not a hardware (device) unit configuration. Therefore, the blocks shown in FIG. 2 may be implemented in a single device, or may be separately implemented in a plurality of devices. Data exchange between the blocks may be performed via any means such as a data bus, a network, a portable storage medium, or the like.

  The communication terminal 20 includes a display unit 21 for displaying various information and an operation unit 22 for receiving an operation by an analyst. The display unit 21 includes a display device such as a liquid crystal display and an organic light emitting diode (OLED) display. The operation unit 22 includes operation members such as buttons, switches, and dials. By using a touch screen capable of detecting the position of contact by the analyst as the display unit 21, the display unit 21 and the operation unit 22 may be configured integrally.

  The voice analysis device 100 includes a control unit 110, a communication unit 120, and a storage unit 130. The control unit 110 includes a setting unit 111, a sound acquisition unit 112, a sound source localization unit 113, an analysis unit 114, a section setting unit 115, and an output unit 116. The storage unit 130 includes a setting information storage unit 131, a voice storage unit 132, and an analysis result storage unit 133.

  The communication unit 120 is a communication interface for communicating between the sound collection device 10 and the communication terminal 20 via the network N. The communication unit 120 includes a processor, a connector, an electric circuit, and the like for executing communication. The communication unit 120 performs predetermined processing on a communication signal received from the outside to acquire data, and inputs the acquired data to the control unit 110. In addition, the communication unit 120 performs predetermined processing on the data input from the control unit 110 to generate a communication signal, and transmits the generated communication signal to the outside.

  The storage unit 130 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, and the like. The storage unit 130 stores a program executed by the control unit 110 in advance. The storage unit 130 may be provided outside the voice analysis device 100, and in that case, data may be exchanged with the control unit 110 via the communication unit 120.

  The setting information storage unit 131 stores setting information indicating analysis conditions set by the analyst in the communication terminal 20. The sound storage unit 132 stores the sound acquired by the sound collection device 10. The analysis result storage unit 133 stores an analysis result indicating a result of analyzing the voice. Each of the setting information storage unit 131, the voice storage unit 132, and the analysis result storage unit 133 may be a storage area on the storage unit 130, or may be a database configured on the storage unit 130.

  The control unit 110 is a processor such as a CPU (Central Processing Unit), for example, and by executing a program stored in the storage unit 130, a setting unit 111, a sound acquisition unit 112, a sound source localization unit 113, an analysis unit 114, It functions as the section setting unit 115 and the output unit 116. The functions of the setting unit 111, the sound acquisition unit 112, the sound source localization unit 113, the analysis unit 114, the section setting unit 115, and the output unit 116 will be described later with reference to FIGS. At least a part of the function of the control unit 110 may be executed by an electric circuit. Further, at least part of the functions of the control unit 110 may be executed by a program executed via a network.

  The voice analysis system S according to the present embodiment is not limited to the specific configuration shown in FIG. For example, the voice analysis device 100 is not limited to one device, and may be configured by connecting two or more physically separated devices in a wired or wireless manner.

[Description of voice analysis method]
FIG. 3 is a schematic diagram of a speech analysis method performed by the speech analysis system S according to the present embodiment. First, the analyst sets analysis conditions by operating the operation unit 22 of the communication terminal 20. For example, the analysis condition is information indicating the number of participants in the discussion to be analyzed and the direction in which each participant (that is, each of the plurality of participants) is located with reference to the sound collection device 10. The communication terminal 20 receives the analysis condition setting from the analyst and transmits it as setting information to the speech analysis apparatus 100 (a). The setting unit 111 of the voice analysis device 100 acquires setting information from the communication terminal 20 and stores the setting information in the setting information storage unit 131.

  FIG. 4 is a front view of the display unit 21 of the communication terminal 20 displaying the setting screen A. The communication terminal 20 displays the setting screen A on the display unit 21 and accepts the analysis condition setting by the analyst. The setting screen A includes a position setting area A1, a start button A2, and an end button A3. The position setting area A1 is an area for setting the direction in which each participant U is actually located with reference to the sound collection device 10 in the discussion of the analysis target. For example, the position setting area A1 represents a circle centered on the position of the sound collecting device 10 as shown in FIG. 4, and further represents an angle along the circle with reference to the sound collecting device 10.

  The analyst sets the position of each participant U in the position setting area A1 by operating the operation unit 22 of the communication terminal 20. In the vicinity of the position set for each participant U, identification information (here, U1 to U4) for identifying each participant U is assigned and displayed. In the example of FIG. 4, four participants U1 to U4 are set. The part corresponding to each participant U in the position setting area A1 is displayed in a different color for each participant. Thereby, the analyst can easily recognize the direction in which each participant U is set.

  The start button A2 and the end button A3 are virtual buttons displayed on the display unit 21, respectively. When the start button A2 is pressed by the analyst, the communication terminal 20 transmits a start instruction signal to the voice analysis device 100. When the end button A3 is pressed by the analyst, the communication terminal 20 transmits an end instruction signal to the voice analysis device 100. In the present embodiment, one discussion is from the start instruction to the end instruction by the analyst.

  When the voice acquisition unit 112 of the voice analysis device 100 receives a start instruction signal from the communication terminal 20, the voice acquisition unit 112 transmits a signal instructing voice acquisition to the sound collector 10 (b). When the sound collection device 10 receives a signal instructing acquisition of sound from the sound analysis device 100, the sound collection device 10 starts acquiring sound. In addition, when the voice acquisition unit 112 of the voice analysis device 100 receives an end instruction signal from the communication terminal 20, the voice acquisition unit 112 transmits a signal instructing the end of voice acquisition to the sound collector 10. When the sound collection device 10 receives a signal instructing the end of voice acquisition from the voice analysis device 100, the sound collection device 10 ends the voice acquisition.

  The sound collection device 10 acquires sound in each of the plurality of sound collection units, and records the sound therein as sound of each channel corresponding to each sound collection unit. Then, the sound collecting device 10 transmits the acquired sounds of the plurality of channels to the sound analyzing device 100 (c). The sound collecting device 10 may sequentially transmit the acquired sound, or may transmit sound of a predetermined amount or a predetermined time. In addition, the sound collection device 10 may collectively transmit sounds from the start to the end of acquisition. The voice acquisition unit 112 of the voice analysis device 100 receives the voice from the sound collection device 10 and stores it in the voice storage unit 132.

  The voice analysis device 100 analyzes the voice at a predetermined timing using the voice acquired from the sound collection device 10. The voice analysis device 100 may analyze voice when an analyst gives an analysis instruction by a predetermined operation on the communication terminal 20. In this case, the analyst selects the voice corresponding to the discussion to be analyzed from the voices stored in the voice storage unit 132.

  Moreover, the voice analysis device 100 may analyze the voice when the voice acquisition is completed. In this case, the voice from the start to the end of acquisition corresponds to the analysis target discussion. Further, the voice analysis apparatus 100 may analyze the voice sequentially (that is, by real-time processing) during the voice acquisition. In this case, in the voice analysis device 100, the voice for the past predetermined time (for example, 30 seconds) from the current time corresponds to the discussion of the analysis target.

  When analyzing the sound, the sound source localization unit 113 first performs sound source localization based on a plurality of channels of sound acquired by the sound acquisition unit 112 (d). The sound source localization is a process of estimating the direction of the sound source included in the sound acquired by the sound acquisition unit 112 every time (for example, every 10 milliseconds to 100 milliseconds). The sound source localization unit 113 associates the sound source direction estimated for each time with the participant direction indicated by the setting information stored in the setting information storage unit 131.

  The sound source localization unit 113 can use a known sound source localization method such as a MUSIC (Multiple Signal Classification) method or a beam forming method as long as the direction of the sound source can be specified based on the sound acquired from the sound collector 10. .

  Next, the analysis unit 114 analyzes the sound based on the sound acquired by the sound acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113 (e). The analysis unit 114 may set the entire completed discussion as an analysis target, or may set a part of the discussion as an analysis target in the case of real-time processing.

  Specifically, first, the analysis unit 114 performs analysis (for example, 10 milliseconds to 100 millimeters) in the analysis target based on the sound acquired by the sound acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113. Every second), it is determined which participant speaks (speaks). The analysis unit 114 specifies a continuous period from when one participant starts speaking until it ends as a speech period, and causes the analysis result storage unit 133 to store it. When a plurality of participants make a statement at the same time, the analysis unit 114 specifies a statement period for each participant.

  In addition, the analysis unit 114 calculates the amount of speech of each participant for each hour and stores it in the analysis result storage unit 133. Specifically, the analysis unit 114 calculates a value obtained by dividing the length of time during which a participant has made a speech by the length of the time window in a certain time window (for example, 5 seconds). (Also called degrees). Then, the analysis unit 114 calculates the amount of speech for each participant for each participant while shifting the time window by a predetermined time (for example, 1 second) from the start time of the discussion to the end time (current in the case of real-time processing). repeat.

  The section setting unit 115 sets one or more sections for the voice corresponding to the discussion to be analyzed based on the input from the user (participant or analyst). The section may be set for each subject subject of the discussion such as “Japanese”, “Science”, “Society”, etc., and discussions such as “Discussion”, “Idea”, and “Summary” are in progress. It may be set for each stage. The section setting unit 115 stores the section information indicating the section in the analysis result storage unit 133 in association with the setting target voice.

  The section information includes the name of the section and the time of the section (that is, the start time and end time of the section in the voice). The section setting unit 115 selects a section based on at least one of (1) an operation on the communication terminal 20, (2) an operation on the sound collection device 10, and (3) a predetermined sound acquired by the sound collection device 10. Set.

  When a section is set based on an operation on the communication terminal 20, a participant or an analyst is included in the section information by operating the operation unit 22 (for example, touch screen, mouse, keyboard, etc.) of the communication terminal 20. Enter the character string and time to be displayed. Participants or analysts may input the section information after the discussion ends, or may input the section information during the discussion. Then, the section setting unit 115 receives the section information specified in the communication terminal 20 via the communication unit 120 and stores it in the analysis result storage unit 133.

  When a section is set based on an operation in the sound collecting device 10, a participant or an analyst operates an operation unit such as a switch or a touch screen provided in the sound collecting device 10 at the time of switching the section. Set the interval. The operation of the operation unit of the sound collection device 10 is associated in advance with switching of a predetermined section (for example, switching from a “discussion” section to an “idea out” section). The section setting unit 115 receives information indicating an operation from the operation unit of the sound collecting device 10 via the communication unit 120, and identifies switching of a predetermined section at the timing of the operation. Then, the section setting unit 115 stores the specified section information in the analysis result storage unit 133.

  When the section is set based on the predetermined sound acquired by the sound collection device 10, the participant or the analyst uses a device capable of generating sound (for example, a portable terminal, a music playback device, etc.) A predetermined switching sound indicating switching is generated. The switching sound may be a sound wave that can be heard by humans or an ultrasonic wave that cannot be heard by humans. The switching sound indicates the switching of sections by, for example, a predefined frequency or an on / off pattern. The switching sound may be generated only at the switching timing of the section, or may be generated continuously in the section.

  A different sound can be used for each section as the switching sound. In this case, the section setting unit 115 detects a switching sound included in the sound acquired by the sound collection device 10. Then, the section setting unit 115 identifies switching from the section corresponding to the switching sound before the change to the section corresponding to the switching sound after the change at the timing when the switching sound changes. Then, the section setting unit 115 stores the specified section information in the analysis result storage unit 133.

  In addition, a sound indicating switching of a predetermined section (for example, switching from a “discussion” section to an “idea out” section) can be used as the switching sound. In this case, the section setting unit 115 detects a switching sound included in the sound acquired by the sound collection device 10. The section setting unit 115 identifies switching of a predetermined section at the timing when the switching sound is generated. Then, the section setting unit 115 stores the specified section information in the analysis result storage unit 133.

  The output unit 116 controls the display unit 21 to display the analysis result by the analysis unit 114 by transmitting the display information to the communication terminal 20 (f). The output unit 116 is not limited to display on the display unit 21, and the analysis result may be output by other methods such as printing by a printer and data recording in a storage device. An analysis result output method by the output unit 116 will be described below with reference to FIGS.

[Description of how to display the amount of speech for each section]
When displaying the analysis result, the output unit 116 of the voice analysis device 100 reads the analysis result by the analysis unit 114 and the section information by the section setting unit 115 on the discussion of the display target from the analysis result storage unit 133. The output unit 116 may display the discussion immediately after the analysis by the analysis unit 114 is completed, or may display the discussion designated by the analyst.

  FIG. 5 is a front view of the display unit 21 of the communication terminal 20 displaying the speech amount screen B. The utterance amount screen B is a screen that displays information indicating the temporal change of the utterance amount for each section, and includes a utterance amount graph B1, a section name B2, and a section switching line B3.

  When displaying the utterance amount screen B, the output unit 116 displays a time change of the utterance amount of each participant for each section based on the analysis result and the section information read from the analysis result storage unit 133. Generate information.

  Graph B1 is a graph which shows the time change of the speech amount of each participant U. The output unit 116 causes the display unit 21 to display the amount of speech for each time indicated by the analysis result for each participant U as a line graph, with the amount of speech (activity) on the vertical axis and time on the horizontal axis. At this time, the output unit 116 displays the value obtained by accumulating the speech amounts of the participants U at each time point, that is, the value obtained by sequentially summing the speech amounts of the participants U in order.

  In the example of FIG. 5, the speech amount of the participant U4 is the total value of the speech amounts of the participants U3 and U4, the speech amount of the participant U2 is the total value of the speech amounts of the participants U2, U3, and U4, The amount of speech of the participant U1 is a total value of the amount of speech of the participants U1, U2, U3, and U4. The output unit 116 may randomly determine the order in which the speech amount of the participant U is accumulated (summed up) or may be determined according to a predetermined rule.

  Accordingly, the output unit 116 can display the amount of speech of the entire discussion group in addition to the amount of speech of each participant U. The analyst can grasp the time change of the contribution of each participant U and at the same time grasp the time change of the excitement of the entire group of the participant U.

  The output unit 116 displays the region or line indicating the graph B1 for each participant U in a display mode such as a different color or pattern for each participant. In the example of FIG. 5, the graph B1 is displayed in a different pattern for each participant U, and a legend that associates the participant U with the pattern is displayed in the vicinity of the graph B1. Thus, the analyst can easily determine which participant U the graph B1 corresponds to.

  The section name B2 is a character string representing the section name. The section switching line B3 is a line indicating the switching timing of two sections. The output unit 116 displays each section indicated by the section information in the section name in the vicinity of the graph B1 in the time range corresponding to the section. Further, the output unit 116 specifies the timing of switching between the two sections based on the section time indicated by the section information. Then, the output unit 116 displays the switching line B3 at the time (horizontal axis) position of the graph B1 corresponding to the identified switching timing. Accordingly, the output unit 116 can display which section the speech amount graph B1 of each participant U corresponds to every time.

  In this way, the output unit 116 displays information indicating the section set in the discussion, superimposed on the time change of the speech amount of each participant U. Therefore, the analyst can grasp the time change of the speech amount of each participant U for each section.

  Since the graph B1 displays the total amount of speech of each participant U (in total), if the amount of speech of the participant U arranged below changes, the participant arranged above accordingly The amount of U's speech is also displayed as if it has changed. For this reason, the temporal change in the amount of speech of each participant U may be difficult to understand at a glance. Therefore, the output unit 116 can display the time change of the speech amount of each participant U in an easy-to-see manner by determining the order in which the speech amount of the participant U is accumulated based on the speech amount of each participant U in the graph B1. it can.

  FIG. 6 is a front view of the display unit 21 of the communication terminal 20 displaying the speech amount screen B. In the speech amount screen B of FIG. 6, the order in which the speech amount is accumulated is changed for each section, and the rest is the same as the speech amount screen B of FIG. The output unit 116 may switch and display the speech amount screen B in FIG. 5 and the speech amount screen B in FIG. 6 according to the operation of the analyst, or may display at least one predetermined.

  When changing the stacking order, the output unit 116, based on the analysis result and the section information read from the analysis result storage unit 133, the degree of variation in the amount of speech of each participant U in each section (for example, variance or standard deviation) ) Is calculated. And the output part 116 produces | generates graph B1 by accumulating the amount of participant U's utterances in order with a small degree of dispersion | variation for every area. The output unit 116 may determine the order of stacking based on the degree of variation in all sections, not for each section.

  Thus, by stacking from the bottom of the graph B1 in ascending order of the degree of variation in the amount of speech, the change in the amount of speech of the participant U arranged below becomes the apparent amount of speech of the participant U arranged above. The influence exerted can be reduced. Moreover, since the tendency of the speech amount of each participant U varies depending on the section, the temporal change in the speech amount can be displayed more easily by changing the stacking order for each section.

[Explanation of event display method]
The output unit 116 may display a predetermined event that occurred during the discussion (that is, within the time of the voice acquired by the voice acquisition unit 112) in the graph B1. Thus, the analyst can analyze the influence of the occurrence of the event on the speech amount of each participant U. The event is, for example, (1) approach to a group of discussion assistants (teacher, facilitator, etc.), or (2) a specific speech (word) of the assistant. The event shown here is an example, and the output unit 116 may display the occurrence of another event that can be recognized by the speech analysis apparatus 100.

  In order to detect the approach of the assistant to the group, the output unit 116 uses a signal exchanged between the sound collector 10 and the assistant. In this case, the assistant holds a transmitter that emits a predetermined signal by radio communication radio waves or ultrasonic waves such as Bluetooth (registered trademark), and the sound collecting device 10 includes a receiver that receives the signal. The output unit 116 indicates that the assistant has approached when the signal from the assistant's transmitter can be received by the receiver of the sound collector 10 or when the intensity of the received signal exceeds a predetermined threshold. Determine. Further, the output unit 116 is configured so that the assistant can leave when the receiver of the sound collecting device 10 cannot receive a signal from the assistant's transmitter or when the intensity of receiving the signal becomes less than a predetermined threshold. Determine what happened.

  Further, in order to detect the approach of the assistant to the group, the output unit 116 may use the assistant's voiceprint (that is, the frequency spectrum of the assistant's voice). In this case, the output unit 116 registers the assistant's voiceprint in advance, and detects the assistant's voiceprint in the voice acquired by the sound collecting device 10 during the discussion. The output unit 116 determines that the assistant has approached when the assistant's voiceprint is detected, and determines that the assistant has left when the assistant's voiceprint cannot be detected.

  In order to detect a specific word of the assistant, the output unit 116 performs voice recognition on the assistant's voice. In this case, the assistant holds a sound collector (for example, a pin microphone), and the output unit 116 receives the voice of the assistant acquired by the sound collector held by the assistant. By using the sound collecting device held by the assistant separately from the sound collecting device 10, the voice of the participant U and the voice of the assistant can be clearly distinguished.

  The output unit 116 converts the voice acquired from the sound collection device held by the assistant into a character string. The output unit 116 can use a known speech recognition method to convert speech into a character string. Then, the output unit 116 includes specific words (for example, words relating to the progress of discussion such as “first”, “summary”, “last”, and words such as “good” and “bad” in the converted character string. ) Is detected. The words to be detected are set in the speech analyzer 100 in advance. When the output unit 116 detects a specific word, the output unit 116 determines that the specific word has been issued.

  The output unit 116 may perform voice recognition only before and after the timing when the change in the amount of speech of each participant U is large. In this case, based on the analysis result read from the analysis result storage unit 133, the output unit 116 calculates the degree of change in the amount of speech for each time (for example, the amount or rate of change per unit time). The degree of change in the amount of speech may be calculated for each participant U or may be calculated as the sum of all participants U.

  The output unit 116 then obtains the sound acquired by the sound collector held by the assistant in a predetermined time range including the timing at which the degree of change is equal to or greater than the predetermined threshold (for example, from 5 seconds before to 5 seconds after the timing) Perform voice recognition. In general, speech recognition has a heavy processing load. Thus, by performing speech recognition only before and after the timing when the degree of change in the amount of speech is large, it is possible to analyze the words that caused the change in the amount of speech while reducing the processing load.

  And the output part 116 produces | generates the display information which linked | related the information which shows the event detected by the above method with the time in an audio | voice. FIG. 7 is a front view of the display unit 21 of the communication terminal 20 displaying the message amount screen B. FIG. In the message volume screen B of FIG. 7, event information B4 is displayed on the graph B1, and the rest is the same as the message volume screen B of FIG. The output unit 116 may switch and display the speech amount screen B of FIG. 5 and the speech amount screen B of FIG. 7 according to the operation of the analyst, or may display at least one predetermined.

  The event information B4 is information indicating the content and timing of the event. The event information B4 indicates the content of the event by, for example, a character string indicating that the assistant has approached or left, or a character string indicating the speech of the assistant detected by voice recognition. The event information B4 indicates the event timing by an arrow indicating the timing at which the event occurs on the graph B1.

  In this manner, the output unit 116 displays information indicating the content and timing of the event that occurred during the discussion, superimposed on the time variation of the speech amount of each participant U. Therefore, the analyst can analyze how the event that occurred during the discussion affected the temporal change in the amount of speech of each participant U. For example, when the amount of speech increases when the teacher approaches the group, the analyst can evaluate that the teacher has activated the discussion. Also, the analyst can evaluate that the words are effective words for activating the discussion when the amount of utterance increases when a specific word is uttered by the teacher, for example.

[Description of how to display the amount of speech in the same section]
The output unit 116 can extract and display a plurality of speech amount graphs in the same section. FIG. 8 is a front view of the display unit 21 of the communication terminal 20 displaying the section extraction screen C. The output unit 116 displays the section extraction screen C for the specified section when the analyst specifies the name B2 of any section on the speech amount screen B of FIGS. The section extraction screen C is a screen for displaying a result of extracting a speech amount graph in the same section, and includes a speech amount graph C1, a section name C2, and a group name C3.

  When displaying the section extraction screen C, the output unit 116 extracts the analysis results and section information of a plurality of groups for the specified section from the analysis result storage unit 133. The groups to be displayed may be different groups discussed at the same time, or the same or different groups discussed in the past. And the output part 116 produces | generates the display information for displaying the time change of each participant's speech amount about the some group in the designated area based on the extracted analysis result and area information.

  The speech amount graph C1 is a graph showing temporal changes in the speech amount of each participant U in a specified section for each of two or more groups. The display mode of the graph C1 is the same as that of the graph B1. The section name C2 is a character string indicating the name of the designated section.

  The group name C3 is a name for identifying a group to be displayed, and may be set by an analyst or may be automatically determined by the speech analysis apparatus 100. In the example of FIG. 8, the output unit 116 displays the graph C1 of two groups, but may display the graph C1 of three or more groups. The output unit 116 may display the names of one or more participants U belonging to the group instead of or in addition to the group name C3.

  In this way, the output unit 116 displays a plurality of graphs showing temporal changes in the amount of speech of each participant in different groups for the same section. Thereby, the analyst can compare and analyze temporal changes of the speech amount of different groups for the same section (for example, the same subject or the same stage in the discussion). For example, the analyst can grasp the tendency of the utterance amount for each group by comparing different groups discussed at the same time. Further, for example, the analyst can grasp a change in the tendency of the speech amount of the same group by comparing a plurality of past discussions in the same section for the same group.

[Explanation of how to display the heat map of speech volume]
The output unit 116 is not limited to the stacked graph as illustrated in FIG. 5, and may display a heat map indicating temporal changes in the amount of speech of each participant U. FIG. 9 is a front view of the display unit 21 of the communication terminal 20 displaying the message amount screen D. The utterance amount screen D includes a utterance amount heat map D1, a section name D2, and a section switching line D3. The section name D2 and the section switching line D3 are the same as the section name B2 and the section switching line B3 in FIG.

  The speech amount heat map D1 displays the amount of speech along time by color. FIG. 9 represents the difference in color by the density of the dots. For example, the higher the density of the dots, the darker the color, and the lower the density of the dots, the lighter the color. The output unit 116 takes time in a predetermined direction (for example, the horizontal direction in FIG. 9) and causes the display unit 21 to display a color area corresponding to the amount of speech for each participant U.

  As described above, the output unit 116 also displays the heat map instead of the graph, so that the analyst can grasp the time change of the speech amount of each participant U for each section. The output unit 116 may switch and display the graph of FIG. 5 and the heat map of FIG. 9 according to the operation of the analyst, or may display at least one predetermined.

[Speech analysis method sequence]
FIG. 10 is a sequence diagram of a speech analysis method performed by the speech analysis system S according to the present embodiment. First, the communication terminal 20 receives setting of analysis conditions from an analyst, and transmits it as setting information to the voice analysis device 100 (S11). The setting unit 111 of the voice analysis device 100 acquires setting information from the communication terminal 20 and stores the setting information in the setting information storage unit 131.

  Next, the voice acquisition unit 112 of the voice analyzer 100 transmits a signal instructing acquisition of voice to the sound collector 10 (S12). When the sound collection device 10 receives a signal instructing the acquisition of sound from the sound analysis device 100, the sound collection device 10 starts recording the sound using a plurality of sound collection units, and the sound analysis device 100 converts the recorded sound of the plurality of channels. (S13). The voice acquisition unit 112 of the voice analysis device 100 receives the voice from the sound collection device 10 and stores it in the voice storage unit 132.

  The voice analysis apparatus 100 starts voice analysis at any timing when an instruction is given by an analyst, when voice acquisition is completed, or while voice is being acquired (that is, real-time processing). When analyzing sound, the sound source localization unit 113 first performs sound source localization based on the sound acquired by the sound acquisition unit 112 (S14).

  Next, the analysis unit 114 determines which participant has made a speech for each participant based on the sound acquired by the sound acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113. The speech period and the speech amount are specified (S15). The analysis unit 114 stores the speech period and the speech amount for each participant in the analysis result storage unit 133.

  The section setting unit 115 sets one or more sections for the voice corresponding to the discussion to be analyzed (S16). At this time, the section setting unit 115 sets a section based on at least one of an operation on the communication terminal 20, an operation on the sound collection device 10, and a predetermined sound acquired by the sound collection device 10. The section setting unit 115 stores the section information indicating the section in the analysis result storage unit 133 in association with the setting target voice.

  The output unit 116 performs control to display the analysis result on the display unit 21 of the communication terminal 20 (S17). Specifically, the output unit 116 is a display for displaying the above-described speech amount screen B, the section extraction screen C, or the speech amount screen D based on the analysis result by the analysis unit 114 and the section information by the section setting unit 115. Information is generated and transmitted to the communication terminal 20.

  The communication terminal 20 displays the analysis result on the display unit 21 according to the display information received from the voice analysis device 100 (S18).

[Effect of this embodiment]
The Harkness method shows the tendency of the speech during the entire period from the start to the end of the discussion, so it cannot show the change in the amount of speech of each participant along the time series of the discussion. For this reason, there is a problem that it is difficult to analyze based on changes in the amount of speech of each participant. On the other hand, the speech analysis apparatus 100 according to the present embodiment displays the time change of the speech amount of each participant for each section. Thereby, the analyzer can grasp | ascertain the time change of the amount of speech of each participant for every area.

  In addition, the voice analysis device 100 automatically analyzes the discussions of a plurality of participants based on the voice acquired using the sound collection device 10 having a plurality of sound collection units. For this reason, unlike the Harkness method described in Non-Patent Document 1, it is not necessary for the recorder to monitor the discussion, and it is not necessary to arrange a recorder for each group.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment, A various deformation | transformation and change are possible within the range of the summary. is there. For example, the specific embodiments of device distribution / integration are not limited to the above-described embodiments, and all or a part of them may be configured to be functionally or physically distributed / integrated in arbitrary units. Can do. In addition, new embodiments generated by any combination of a plurality of embodiments are also included in the embodiments of the present invention. The effect of the new embodiment produced by the combination has the effect of the original embodiment.

  The speech analysis apparatus 100, the sound collection apparatus 10, and the processor of the communication terminal 20 are main components of each step (process) included in the speech analysis method shown in FIG. That is, the speech analysis device 100, the sound collection device 10, and the processor of the communication terminal 20 read a program for executing the speech analysis method shown in FIG. 10 from the storage unit, and execute the program to execute the speech analysis device 100, the collection device. The voice analysis method shown in FIG. 10 is executed by controlling each part of the sound device 10 and the communication terminal 20. Part of the steps included in the speech analysis method shown in FIG. 10 may be omitted, the order between steps may be changed, and a plurality of steps may be performed in parallel.

S voice analysis system 100 voice analysis device 110 control unit 112 voice acquisition unit 114 analysis unit 115 section setting unit 116 output unit 10 sound collector 20 communication terminal 21 display unit

Claims (11)

An acquisition unit that acquires audio generated by a plurality of participants belonging to a group in association with the group ;
An analysis unit for identifying a speech amount for each of the plurality of participants in the voice;
Based on the input from the user, the Keru your voice and interval setting unit for setting a name of at least a portion of section and the section,
An output unit that outputs a graph in which temporal changes in the amount of speech of the plurality of participants are stacked, and information indicating the section in the graph;
I have a,
The output unit associates the plurality of graphs of the section in which the same name is set in the plurality of voices associated with the plurality of groups with each of the plurality of groups corresponding to each of the plurality of graphs. Output voice analysis device. The speech analysis apparatus according to claim 1, wherein the output unit outputs a time position at a boundary between two consecutive sections as information indicating the section on the graph . The section setting unit, as the input from the user, the operation by the user in a communication terminal communicating with the sound analysis device, and the operation by the user in the sound collecting device for acquiring the voice, the user is generated The speech analysis apparatus according to claim 1, wherein the section is set based on at least one of the timings at which the acquisition unit acquires the predetermined sound.   The output unit determines an order in which the temporal changes of the speech amount are accumulated based on the speech amount of each of the plurality of participants, and outputs the graph in which the temporal changes of the speech amount are accumulated in the determined order. The speech analyzer according to any one of claims 1 to 3. The voice analysis according to claim 4 , wherein the output unit outputs the graph in which temporal changes in the amount of speech are accumulated in order of increasing degree of variation in the amount of speech calculated for each of the plurality of participants. apparatus. The output unit outputs the graph obtained by accumulating the temporal changes of the speech amount for each section in order of increasing degree of variation of the speech amount for each section calculated for each of the plurality of participants. The speech analysis apparatus according to claim 5 .   The voice analysis device according to any one of claims 1 to 6, wherein information indicating an event occurring within the time of the voice is output on the graph in addition to the information indicating the graph and the section.   8. The analysis unit according to claim 1, wherein the analysis unit specifies a value obtained by dividing the length of time during which a participant has made a speech within a predetermined time window by the length of the time window as the speech amount. The voice analysis device according to claim 1. Processor
Obtaining audio from a plurality of participants belonging to a group in association with the group ;
Identifying the amount of speech per time for each of the plurality of participants in the voice;
A step of, based on input from the user, to set the name of your Keru least some sections and the sections in the speech,
Outputting a graph obtained by accumulating time changes in the amount of speech of the plurality of participants, and information indicating the section in the graph;
The execution,
The outputting step associates the plurality of graphs of the section in which the same name is set in the plurality of voices associated with the plurality of groups with the plurality of groups corresponding to the plurality of graphs, respectively. Output voice analysis method. On the computer,
Obtaining audio from a plurality of participants belonging to a group in association with the group ;
Identifying the amount of speech per time for each of the plurality of participants in the voice;
A step of, based on input from the user, to set the name of your Keru least some sections and the sections in the speech,
Outputting a graph obtained by accumulating time changes in the amount of speech of the plurality of participants, and information indicating the section in the graph;
Was executed,
The outputting step associates the plurality of graphs of the section in which the same name is set in the plurality of voices associated with the plurality of groups with the plurality of groups corresponding to the plurality of graphs, respectively. voice analysis program that output Te. A voice analysis device, and a communication terminal capable of communicating with the voice analysis device,
The communication terminal has a display unit for displaying information,
The speech analyzer is
An acquisition unit that acquires audio generated by a plurality of participants belonging to a group in association with the group ;
An analysis unit for identifying a speech amount for each of the plurality of participants in the voice;
Based on the input from the user, the Keru your voice and interval setting unit for setting a name of at least a portion of section and the section,
An output unit that causes the display unit to display a graph in which temporal changes in the amount of speech of the plurality of participants are stacked, and information indicating the section in the graph;
I have a,
The output unit associates the plurality of graphs of the section in which the same name is set in the plurality of voices associated with the plurality of groups with each of the plurality of groups corresponding to each of the plurality of graphs. Output voice analysis system.

Images