JP7414319B2 - Speech analysis device, speech analysis method, speech analysis program and speech analysis system - Google Patents

Description

The present invention relates to a speech analysis device, a speech analysis method, a speech analysis program, and a speech analysis system for analyzing speech.

The Harkness method (also referred to as the Harkness method) is known as a method for analyzing discussions in group studies and meetings (see, for example, Non-Patent Document 1). In the Harkness method, the transition of each participant's utterances is recorded with lines. This makes it possible to analyze each participant's contribution to the discussion and their relationship with others. The Harkness method can also be effectively applied to active learning, where students learn independently.

Paul Sevigny, "Extreme Discussion Circles: Preparing ESL Students for "The Harkness Method"," Polyglossia, Ritsumeikan Asia Pacific University Language Education Center, October 2012, No. 23, p. 181-191

When conducting an analysis such as the Harkness method in a school or organization, multiple groups often engage in discussions at the same time. In this case, discussion assistants (teachers, facilitators, etc.) are assigned to multiple groups, which poses a problem in that it is difficult to grasp the status of each group's statements in the discussion.

The present invention has been made in view of these points, and an object of the present invention is to provide a speech analysis device, a speech analysis method, a speech analysis program, and a speech analysis system that can output information indicating the status of speech in a plurality of groups. do.

A voice analysis device according to a first aspect of the present invention includes an acquisition unit that acquires voices uttered by a plurality of participants from a plurality of sound collection devices, and an analysis that identifies utterances of each of the plurality of participants in the voice. and an output unit that causes a display unit to display information indicating the status of the utterance in association with each of the plurality of sound collecting devices.

The output unit may cause the display unit to display information indicating the amount of speech for each sound collection device or for each participant as information indicating the status of the speech.

The output unit may display information indicating the state of the utterance on the display unit provided on a communication terminal that communicates with the speech analysis device.

The output unit may display information indicating the status of the utterance at a position on the display unit provided in the communication terminal that corresponds to a position of each of the plurality of sound collecting devices.

When a predetermined person approaches any of the plurality of sound collecting devices, the output unit displays the person at a position on the display unit provided on the communication terminal corresponding to the position of the sound collecting device. Information indicating the approach of the vehicle may be displayed.

The voice analysis device may further include a setting unit that sets the position of each of the plurality of sound collecting devices based on signals exchanged between the plurality of sound collecting devices.

The output unit may cause the display unit provided in each of the plurality of sound collecting devices to display information indicating the status of the utterance.

The output unit may cause the display unit provided in each of the plurality of sound collection devices to display information indicating the status of the utterance for each of the sound collection devices.

The output unit may cause the display unit provided for each of the plurality of participants in each of the plurality of sound collection devices to display information indicating the state of the utterance of each of the participants.

A voice analysis method according to a second aspect of the present invention includes a step in which a processor acquires voices uttered by a plurality of participants from a plurality of sound collection devices, and identifies utterances of each of the plurality of participants in the voices. and displaying information indicating the status of the utterance on a display unit in association with each of the plurality of sound collecting devices.

A voice analysis program according to a third aspect of the present invention includes the steps of: acquiring voices uttered by a plurality of participants from a plurality of sound collecting devices to a computer; and identifying utterances of each of the plurality of participants in the voices. and displaying information indicating the status of the utterance on a display unit in association with each of the plurality of sound collecting devices.

A voice analysis system according to a fourth aspect of the present invention includes a voice analysis device, a communication terminal capable of communicating with the voice analysis device, and a plurality of sound collection devices, At least one of the devices includes a display unit that displays information, and the voice analysis device includes an acquisition unit that acquires voices uttered by the plurality of participants from the plurality of sound collection devices; and an output unit that causes the display unit to display information indicating the status of the utterance in association with each of the plurality of sound collecting devices.

According to the present invention, it is possible to output information indicating the status of comments in a plurality of groups.

FIG. 1 is a schematic diagram of a speech analysis system according to the present embodiment. FIG. 1 is a block diagram of a speech analysis system according to the present embodiment. It is a schematic diagram of the speech analysis method performed by the speech analysis system concerning this embodiment. FIG. 3 is a schematic diagram of a method for setting group position information by a setting section. It is a front view of the display part of the communication terminal which is displaying a participant setting screen. FIG. 2 is a side view of a sound collection device displaying the status of speech by a group and participants. FIG. 3 is a front view of a display unit of a communication terminal displaying the status of group comments. FIG. 3 is a front view of a display unit of a communication terminal displaying the status of participants' statements. FIG. 2 is a sequence diagram of a speech analysis method performed by the speech analysis system according to the present embodiment.

[Overview of speech analysis system S]
FIG. 1 is a schematic diagram of a speech analysis system S according to this embodiment. The speech analysis system S includes a speech analysis device 100, a sound collection device 10, and a communication terminal 20. The number of sound collection devices 10 and communication terminals 20 included in the voice analysis system S is not limited. The speech analysis system S may also include other devices such as servers and terminals.

The voice analysis device 100, the sound collection device 10, and the communication terminal 20 are connected via a network N such as a local area network or the Internet. At least some of the voice analysis device 100, the sound collection device 10, and the communication terminal 20 may be directly connected without going through the network N.

The sound collecting device 10 includes a microphone array including a plurality of sound collecting sections (microphones) arranged in different directions. For example, a microphone array includes eight microphones arranged at equal intervals on the same circumference in a horizontal plane relative to the ground. The sound collection device 10 transmits the voice acquired using the microphone array to the voice analysis device 100 as data.

The communication terminal 20 is a communication device that can perform wired or wireless communication. The communication terminal 20 is, for example, a mobile terminal such as a smartphone terminal, or a computer terminal such as a personal computer. The communication terminal 20 receives the setting of analysis conditions from the analyst and displays the analysis results by the speech analysis device 100. The communication terminal 20 may be held by the discussion analyst or assistant, or may be placed near each sound collection device 10.

The voice analysis device 100 is a computer that analyzes the voice acquired by the sound collection device 10 using a voice analysis method described below. Additionally, the voice analysis device 100 transmits the voice analysis results to the communication terminal 20.

[Configuration of speech analysis system S]
FIG. 2 is a block diagram of the speech analysis system S according to this embodiment. In FIG. 2, arrows indicate main data flows, and there may be data flows that are not shown in FIG. In FIG. 2, each block shows the configuration of a functional unit rather than a hardware (device) unit. As such, the blocks shown in FIG. 2 may be implemented within a single device or may be implemented separately within multiple devices. Data may be exchanged between blocks via any means such as a data bus, a network, or a portable storage medium.

In addition to the above-mentioned sound collecting section, the sound collecting device 10 has an overall lamp 11 that displays the status of the entire group, and an individual lamp 12 that displays the status of each participant. The total number of lamps 11 is at least one per sound collector 10. The number of individual lamps 12 is equal to or greater than the number of participants that can be handled by at least one sound collection device 10 (that is, the number of participants that can belong to one discussion group). Each individual lamp 12 is provided to correspond to each participant surrounding the sound collection device 10. For example, in front of each participant on the sound collection device 10, at least one individual lamp 12 is provided corresponding to that participant.

The overall lamp 11 and the individual lamps 12 are display units that emit light under the control of the audio analysis device 100, respectively. For example, the overall lamp 11 and the individual lamps 12 can blink the light, change the color of the light, or change the intensity of the light depending on the signal received from the speech analysis device 100. Other display devices such as a liquid crystal display may be used as the overall lamp 11 and the individual lamps 12 as long as they can show the speaking status to the participants or assistants.

The communication terminal 20 includes a display section 21 for displaying various information and an operation section 22 for accepting operations by an analyst. The display unit 21 includes a display device such as a liquid crystal display or an organic light emitting diode (OLED) display. The operation unit 22 includes operation members such as buttons, switches, and dials. The display section 21 and the operation section 22 may be integrally configured by using a touch screen capable of detecting the position of touch by the analyst as the display section 21.

The speech analysis device 100 includes a control section 110, a communication section 120, and a storage section 130. The control section 110 includes a setting section 111 , an audio acquisition section 112 , a sound source localization section 113 , an analysis section 114 , and an output section 115 . The storage unit 130 includes a setting information storage unit 131 , a voice storage unit 132 , and an analysis result storage unit 133 .

The communication unit 120 is a communication interface for communicating with the sound collection device 10 and the communication terminal 20 via the network N. The communication unit 120 includes a processor, a connector, an electric circuit, etc. for performing communication. The communication unit 120 acquires data by performing predetermined processing on communication signals received from the outside, and inputs the acquired data to the control unit 110. Furthermore, the communication unit 120 performs predetermined processing on the data input from the control unit 110 to generate a communication signal, and transmits the generated communication signal to the outside.

The storage unit 130 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, and the like. The storage unit 130 stores in advance a program to be executed by the control unit 110. The storage unit 130 may be provided outside the speech analysis device 100, and in that case, data may be exchanged with the control unit 110 via the communication unit 120.

The setting information storage unit 131 stores setting information indicating analysis conditions including the positions of groups and participants. The audio storage unit 132 stores the audio acquired by the sound collection device 10. The analysis result storage unit 133 stores analysis results indicating the results of analyzing speech. The setting information storage section 131, the voice storage section 132, and the analysis result storage section 133 may each be a storage area on the storage section 130, or may be a database configured on the storage section 130.

The control unit 110 is, for example, a processor such as a CPU (Central Processing Unit), and executes a program stored in the storage unit 130 to control the setting unit 111, the audio acquisition unit 112, the sound source localization unit 113, the analysis unit 114, and the like. It functions as an output section 115. The functions of the setting section 111, audio acquisition section 112, sound source localization section 113, analysis section 114, and output section 115 will be described later using FIGS. 3 to 8. At least some of the functions of the control unit 110 may be performed by an electrical circuit. Further, at least part of the functions of the control unit 110 may be executed by a program executed via a network.

The speech analysis system S according to this embodiment is not limited to the specific configuration shown in FIG. 2. For example, the speech analysis device 100 is not limited to one device, and may be configured by two or more physically separate devices connected by wire or wirelessly.

[Explanation of voice analysis method]
FIG. 3 is a schematic diagram of a speech analysis method performed by the speech analysis system S according to this embodiment. First, the analyst sets analysis conditions by operating the operation unit 22 of the communication terminal 20. The communication terminal 20 receives settings for analysis conditions from the analyst, and transmits them as setting information to the speech analysis device 100 (a). The setting unit 111 of the speech analysis device 100 causes the setting information storage unit 131 to store the setting information received from the communication terminal 20 or the setting information specified by the setting unit 111 itself.

The setting information includes participant position information indicating the position of each participant (i.e., each of a plurality of participants) in a group regarding one sound collection device 10, and participant position information indicating the position of each participant (i.e., each of a plurality of participants) in a group related to one sound collection device 10, and participant position information indicating the position of each participant in a group (i.e., each of a plurality of groups) that discuss simultaneously. and group position information indicating the position. The position of each group corresponds to the position of each sound collection device 10 (that is, each of the plurality of sound collection devices 10). For example, the participant position information is information indicating the number of participants in the discussion to be analyzed and the direction in which each participant is located with respect to the sound collection device 10. For example, group position information is information indicating the number of groups to be analyzed and the relative or absolute position of each group.

FIGS. 4A and 4B are schematic diagrams of a method for setting group position information by the setting unit 111. FIG. 4A shows the display unit 21 of the communication terminal 20 displaying the group setting screen A for setting group position information. When setting the group position information, the communication terminal 20 displays the group setting screen A on the display unit 21 and accepts the setting of the group position information by the analyst. Group setting screen A includes a position setting area A1, a reference position A2, a completion button A3, and an automatic setting button A4.

The position setting area A1 is an area for setting the position of each group G based on a predetermined reference position A2 for discussions that are held simultaneously (for example, discussions that are held in the same room). For example, the position setting area A1 is a rectangular area including the reference position A2 (for example, the position of the teacher's desk) as shown in FIG. 4(a). The reference position A2 may be specified by the analyst on the position setting area A1, or may be registered in the speech analysis device 100 in advance.

The analyst sets the position of each group G in the position setting area A1 by operating the operation unit 22 of the communication terminal 20. For example, when the analyst presses somewhere within the position setting area A1, the setting unit 111 of the speech analysis device 100 sets the position pressed by the analyst as the position of one group G. Identification information for identifying each group G (here, G1 to G4) is assigned and displayed at the position set for each group G. The identification information for each group G may be input by the analyst, or may be automatically determined by the setting unit 111. In the example of FIG. 4(a), four groups G1 to G4 are set.

The completion button A3 and the automatic setting button A4 are virtual buttons displayed on the display unit 21, respectively. When the finish button A3 is pressed by the analyst, the setting unit 111 stores the position of each group G set on the group setting screen A in the setting information storage unit 131 as group position information. Set the position of Then, the setting unit 111 causes the communication terminal 20 to display a participant setting screen C shown in FIG. 5 for each group G set on the group setting screen A.

When the automatic setting button A4 is pressed by the analyst, the setting unit 111 automatically sets the position of each group G based on the information acquired from the sound collection device 10. FIG. 4B is a schematic diagram of a method in which the setting unit 111 automatically sets the position of each group G. For example, the setting unit 111 automatically sets the position of each sound collection device 10, that is, the position of each group G, based on sound or radio wave signals exchanged between the sound collection devices 10.

For automatic setting, each sound collection device 10 is equipped with a transmitter that generates a signal of a predetermined sound (sound wave or ultrasonic wave) or a predetermined radio wave (for example, a radio wave of short-range wireless communication such as Bluetooth (registered trademark)). The sound collecting device 10 includes a receiver that receives the signal. The setting unit 111 causes the transmitters of each sound collection device 10 to sequentially generate signals. The setting unit 111 acquires the time and intensity at which the signal was detected by the receiver of each sound collection device 10.

Next, for each sound collecting device 10, the setting unit 111 identifies the sound collecting device 10 that is the source of the signal using the time at which the signal was detected, and also specifies the sound collecting device 10 that is the source of the signal using the intensity at which the signal is detected. Calculate the distance B to 10. Then, the setting unit 111 determines the position of each sound collecting device 10, that is, the position of each group G, based on the calculated distance B between the sound collecting devices 10, for example, by triangulation. The setting unit 111 sets the position of each group G by storing the determined position of each group G in the setting information storage unit 131 as group position information. Then, the setting unit 111 causes the communication terminal 20 to display a participant setting screen C shown in FIG. 5 for each group G.

Here, a method of determining the position of the sound collecting device 10 by exchanging sound or radio wave signals between the sound collecting devices 10 has been described as an example, but the setting unit 111 determines the position of the sound collecting device 10 using other methods. You may. For example, a sound or radio wave signal may be generated from a device provided outside the sound collection device 10.

The setting unit 111 may perform a combination of automatic setting and manual setting of the position of group G. In this case, for example, the setting unit 111 displays the position of each group G set by the automatic setting in FIG. 4(b) in the position setting area A1 in FIG. 4(a), and further allows the analyst to manually set the position. accept. Thereby, the position of each group G set by automatic setting can be corrected by manual setting, and the position of each group G can be set more reliably.

FIG. 5 is a front view of the display unit 21 of the communication terminal 20 displaying a participant setting screen C for setting participant location information. The communication terminal 20 displays a participant setting screen C on the display unit 21 when setting participant location information, and allows the analyst to set participant location information for each group set on the group setting screen A. accept. The participant setting screen C includes a position setting area C1, a start button C2, and an end button C3. The position setting area C1 is an area for setting the direction in which each participant U is actually positioned with respect to the sound collection device 10 in a discussion to be analyzed. For example, the position setting area C1 represents a circle centered on the position of the sound collector 10 as shown in FIG. 5, and further represents an angle with the sound collector 10 as a reference along the circle.

The analyst inputs the position of each participant U in the position setting area C1 by operating the operation unit 22 of the communication terminal 20. The setting unit 111 sets the position of each participant U by storing the position of each participant U input by the analyst in the setting information storage unit 131 as participant position information. Identification information (here, U1 to U4) for identifying each participant U is assigned and displayed near the position set for each participant U. In the example of FIG. 5, four participants U1 to U4 are set. The portion corresponding to each participant U within the position setting area C1 is displayed in a different color for each participant. This allows the analyst to easily recognize the orientation in which each participant U is set.

The start button C2 and the end button C3 are virtual buttons displayed on the display unit 21, respectively. When the analyst presses the start button C2, the communication terminal 20 transmits a start instruction signal to the speech analysis device 100. When the analyst presses the end button C3, the communication terminal 20 transmits a signal instructing the end to the voice analysis device 100. In the present embodiment, one discussion includes from the start instruction to the end instruction by the analyst.

When the voice acquisition unit 112 of the voice analysis device 100 receives the start instruction signal from the communication terminal 20, it transmits a signal instructing to acquire the voice to the sound collection device 10 (b). The sound collection device 10 starts acquiring audio when receiving a signal instructing audio acquisition from the audio analysis device 100 . Furthermore, when the audio acquisition unit 112 of the audio analysis device 100 receives a termination instruction signal from the communication terminal 20, it transmits a signal instructing the termination of audio acquisition to the sound collection device 10. When the sound collection device 10 receives a signal from the speech analysis device 100 instructing the end of the audio acquisition, the sound collection device 10 ends the audio acquisition.

The sound collection device 10 acquires sounds from each of the plurality of sound collection sections and internally records them as sounds of each channel corresponding to each sound collection section. Then, the sound collection device 10 transmits the acquired voices of the plurality of channels to the voice analysis device 100 (c). The sound collection device 10 may transmit the acquired sounds one after another, or may transmit a predetermined amount or a predetermined amount of time. The voice acquisition unit 112 of the voice analysis device 100 receives voice from the sound collection device 10 and stores it in the voice storage unit 132.

The voice analysis device 100 analyzes the voices of each group acquired from the sound collection device 10 sequentially, that is, in real-time processing. For example, the voice analysis device 100 sequentially analyzes voices for a predetermined period of time (for example, 30 seconds) in the past going back from the current time.

When analyzing audio, the sound source localization section 113 first localizes the sound source based on the multiple channels of audio acquired by the audio acquisition section 112 (d). Sound source localization is a process of estimating the direction of a sound source included in the audio acquired by the audio acquisition unit 112 at each time (for example, every 10 to 100 milliseconds). The sound source localization unit 113 associates the direction of the sound source estimated for each time with the direction of the participant indicated by the setting information stored in the setting information storage unit 131.

The sound source localization unit 113 can use a known sound source localization method such as the MUSIC (Multiple Signal Classification) method or the beamforming method, as long as the direction of the sound source can be specified based on the sound acquired from the sound collection device 10. .

Next, the analysis unit 114 analyzes the audio based on the audio acquired by the audio acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113 (e). Specifically, first, the analysis unit 114 analyzes the analysis target every time (for example, from 10 milliseconds to 100 milliseconds) in the discussion of the analysis target based on the sound acquired by the voice acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113. (every second), it is determined which participant spoke (uttered). The analysis unit 114 specifies a continuous period from when one participant starts speaking until it ends as a speaking period, and stores it in the analysis result storage unit 133. If multiple participants speak at the same time, the analysis unit 114 identifies the speaking period for each participant.

Furthermore, the analysis unit 114 calculates the amount of speech by each participant for each time period, and stores it in the analysis result storage unit 133. Specifically, the analysis unit 114 calculates the amount of speech per hour by dividing the length of time a participant speaks by the length of the time window in a certain time window (for example, 5 seconds). do. The analysis unit 114 then shifts the time window by a predetermined time (for example, 1 second) from the start time of the discussion to the end time (in the case of real-time processing, the current time), and calculates the amount of speech for each participant per hour. repeat. In addition, the analysis unit 114 calculates the total value of the amount of speech of participants belonging to the group for each time as the amount of speech (activity level) of the group for each time.

Then, the analysis unit 114 uses the amount of comments made by the participants and groups to identify the state of comments made by the participants and groups. The analysis unit 114 determines, as the state of the participants' comments, the total or integral value of the amount of comments made by the participants over a predetermined period (for example, 20 seconds) going back from the present, or the total value or the integral value of the amount of comments made by the participants. Calculate the ratio (i.e. relative value) between the two. The status of participants' statements calculated in this manner can be used as an index representing the degree of contribution of each participant to the discussion.

The analysis unit 114 also determines the status of the group's comments, such as the total or integrated value of the group's amount of comments over a predetermined period (for example, 20 seconds) retroactively, or the group's total or integrated value of the amount of comments. Calculate the ratio (i.e. relative value) between The status of the group's statements calculated in this way can be used as an index representing the level of excitement of the discussion in each group. The analysis unit 114 causes the analysis result storage unit 133 to store situation information indicating the state of statements of each participant and each group as an analysis result.

The atmosphere of the group may be used as the status of the group's comments. The atmosphere of the group is information indicating whether there are many or few changes of speakers in the discussion. Specifically, the analysis unit 114 calculates the average number of speech transitions between the same participants U (that is, switching from one speech period to another speech period) and the number of speech transitions between different participants. The average number of transitions is calculated, and the ratio between them is specified as the atmosphere of the group. For example, when the ratio of speech transitions between the same participants U is large, one participant in that group tends to speak for a long time, and when the ratio of speech transitions between different participants is large, the group tends to have multiple participants. Participants tend to take turns speaking.

The output unit 115 associates the speech status of each participant and each group identified by the analysis unit 114 with the sound collection device 10 corresponding to the participant and the group, and displays it on the sound collection device 10 and the communication terminal 20. Display it in the section. Specifically, the output unit 115 reads the situation information stored in the analysis result storage unit 133 and transmits it to the sound collection device 10, thereby transmitting information indicating the situation of the statement identified by the analysis unit 114 to the overall lamp. 11 and the individual lamps 12 (f).

The output unit 115 also causes the display unit 21 to display information indicating the status of the statement identified by the analysis unit 114 by reading out the status information stored in the analysis result storage unit 133 and transmitting it to the communication terminal 20. Perform control (g). A method of outputting the status of a statement by the output unit 115 will be explained below using FIGS. 6 to 8.

[Explanation of how to display the speaking status]
FIG. 6 is a side view of the sound collection device 10 displaying the status of the group and participants' statements. The output unit 115 uses the overall lamp 11 and individual lamps provided in the sound collection device 10 to display information indicating the status of the group and participants' statements. Specifically, the output unit 115 causes the sound collection device 10 to display the status of the group's utterances by causing the overall lamp 11 to emit a predetermined light corresponding to the status of the group's utterances. For example, the output unit 115 changes the blinking speed of the light, the color of the light, or the intensity of the light depending on the total value or integral value of the amount of speech of the group. Thereby, discussion participants and assistants can easily grasp the status of statements made by the entire group.

Furthermore, the output unit 115 displays the speaking status of each participant on the sound collection device 10 by generating a predetermined light corresponding to the speaking status of the participant from the individual lamp 12 corresponding to the participant. let That is, the output unit 115 causes the individual lamps 12 provided at positions corresponding to each participant (for example, in front of each participant) to emit light in accordance with the state of speech of each participant. For example, the output unit 115 changes the blinking speed of the light, the color of the light, or the intensity of the light according to the total value or integral value of the amount of speech by the participants. This allows discussion participants to objectively recognize the status of their own statements, and discussion assistants to understand the contribution of each participant to the discussion.

FIG. 7 is a front view of the display unit 21 of the communication terminal 20 displaying the status of group comments. The output unit 115 causes the display unit 21 of the communication terminal 20 to display information indicating the status of the group's comments as the overall status screen D. The overall situation screen D includes a circle D1 indicating the status of statements for each group, and an icon D2 indicating the position of the assistant.

The output unit 115 displays a circle D1 in a display mode corresponding to the group's speaking status at a position on the display unit 21 corresponding to the position of each group indicated by the group position information stored in the setting information storage unit 131. . That is, the output unit 115 displays information indicating the status of the group's statements on a map indicating the position of each group (each sound collection device 10). For example, the output unit 115 changes the color, shape, pattern, etc. of the circle D1 according to the total value or integral value of the amount of speech of the group. This allows the discussion analyst or assistant to easily grasp the status of each group's statements.

Further, the output unit 115 displays an icon D2 indicating the position of the assistant at a position on the display unit 21 corresponding to the position of the assistant in the discussion. In order to detect the position of the assistant, the output unit 115 uses, for example, a signal exchanged between the sound collection device 10 and the assistant. In this case, the assistant holds a transmitter that emits a predetermined signal using radio waves or ultrasonic waves of wireless communication such as Bluetooth, and the sound collection device 10 includes a receiver that receives the signal.

Then, when the receiver of the sound collection device 10 is able to receive a signal from the transmitter of the assistant, or when the intensity of the received signal is equal to or higher than a predetermined threshold, the output unit 115 sends the sound collection device 10 to the sound collection device 10 to provide assistance. Detects when someone approaches. Further, the output unit 115 outputs a signal to the sound collecting device 10 when the receiver of the sound collecting device 10 is unable to receive a signal from the transmitter of the assistant or when the intensity of the received signal becomes less than a predetermined threshold. Detects that the assistant has left the station.

When the output unit 115 detects that an assistant approaches one of the sound collection devices 10, it displays an icon D2 near the sound collection device 10 (group) on the display unit 21. This allows the discussion analyst to analyze changes in the situation of each group's statements between when the assistant is close to the group and when the assistant is not, and also to evaluate the assistant. It can be carried out.

FIG. 8 is a front view of the display unit 21 of the communication terminal 20 displaying the status of participants' statements. When any group is specified by the analyst or assistant on the overall situation screen D, the output unit 115 communicates information indicating the state of statements of each participant belonging to the group as an individual situation screen E. It is displayed on the display unit 21 of the terminal 20. The individual status screen E includes an area E1 showing the status of each participant's statements. Area E1 consists of multiple areas corresponding to multiple participants.

The output unit 115 displays an area E1 in a display mode corresponding to the speaking situation of each participant. For example, the output unit 115 changes the color, pattern, etc. of the area corresponding to each participant in the area E1 according to the total value or integral value of the amount of speech by each participant. Furthermore, the output unit 115 displays identification information (here, U1 to U4) for identifying each participant near the area corresponding to each participant. This allows the discussion analyst or assistant to easily grasp the status of statements made by each participant belonging to one group.

The method of outputting information indicating the status of statements of groups and participants shown in FIGS. 6 to 8 is an example, and the information can be displayed in association with the sound collection device 10 corresponding to the group and participant. Other output methods may also be used. The output unit 115 does not necessarily need to display information indicating the speaking status by both the sound collection device 10 and the communication terminal 20, and may display information indicating the speaking status by at least one of them. The output unit 115 may output information indicating the speech status by other means, such as printing with a printer or recording data in a storage device.

[Speech analysis method sequence]
FIG. 9 is a sequence diagram of a speech analysis method performed by the speech analysis system S according to this embodiment. First, the communication terminal 20 receives settings for analysis conditions from the analyst, and transmits them as setting information to the speech analysis device 100 (S11). The setting unit 111 of the speech analysis device 100 causes the setting information storage unit 131 to store the setting information acquired from the communication terminal 20 or the setting information specified by the setting unit 111 itself. The setting information includes participant position information indicating the position of each participant in the group regarding one sound collection device 10, and group position information indicating the position of each group (i.e., each sound collection device 10) having a discussion at the same time. .

Next, the voice acquisition unit 112 of the voice analysis device 100 transmits a signal instructing the acquisition of voice to the sound collection device 10 (S12). When the sound collection device 10 receives a signal instructing to acquire audio from the audio analysis device 100, it starts recording audio using the plurality of sound collection units, and transfers the recorded audio of the plurality of channels to the audio analysis device 100. (S13). The voice acquisition unit 112 of the voice analysis device 100 receives voice from the sound collection device 10 and stores it in the voice storage unit 132.

The speech analysis device 100 analyzes the acquired speech sequentially, that is, in real-time processing. When analyzing the sound, the sound source localization unit 113 first performs sound source localization based on the sound acquired by the sound acquisition unit 112 (S14).

Next, the analysis unit 114 determines which participant spoke at each time based on the audio acquired by the audio acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113. Calculate the amount of speech per hour. Then, the analysis unit 114 identifies the state of the participant's speech using the calculated amount of speech (S15). The status of the participants' comments is, for example, the total value or integral value of the amount of comments made by the participants, or the ratio (ie, relative value) between the participants of the total value or integral value of the amount of comments made by the participants.

In addition, the analysis unit 114 calculates the amount of comments made by each group by summing up the amount of comments made by each group for each group, and uses the calculated amount of comments to identify the state of the group's comments ( S16). The status of comments of a group is, for example, the total value or integral value of the amount of comments made by the group, or the ratio (ie, relative value) between groups of the total value or integral value of the amount of comments made by the group. The analysis unit 114 causes the analysis result storage unit 133 to store situation information indicating the state of statements of each participant and each group as an analysis result.

The output unit 115 displays information indicating the speaking status of each participant and each group by transmitting status information indicating the speaking status of each participant and each group to the sound collection device 10 and the communication terminal 20. Control is performed (S17).

The sound collection device 10 displays information indicating the speech status of each participant and each group by lighting the overall lamp 11 and the individual lamps 12 according to the status information received from the voice analysis device 100 (S18). According to the display information received from the speech analysis device 100, the communication terminal 20 causes the display unit 21 to display the overall situation screen D and the individual situation screen E, which represent information indicating the state of speech of each participant and each group (S19 ). The speech analysis device 100 analyzes speech in real-time processing by repeating steps S12 to S19 at predetermined time intervals.

[Effects of this embodiment]
The speech analysis device 100 according to the present embodiment identifies the state of speech in the group related to the sound collecting device 10 based on the sound acquired using the sound collecting device 10 having a plurality of sound collecting sections, and Or display it on the communication terminal 20. Therefore, even when multiple groups are having a discussion at the same time, the discussion analyst or assistant can check the status of comments in the multiple groups by referring to the display on the sound collection device 10 or the communication terminal 20. can be easily understood.

When the speech analysis device 100 displays the speech status on the sound collection device 10, in addition to the discussion analyst or assistant, the participants can also objectively know the speech status of the group to which they belong. . In addition, by displaying the speaking status on the individual lamp 12 provided for each participant on the sound collection device 10, the participant can easily distinguish his own speaking status from the speaking status of other participants. be able to.

When the speech analysis device 100 displays the speech status on the communication terminal 20, the discussion analyst or assistant can see the speech status of all groups from a bird's-eye view. Further, since there is no need to provide a lamp for each sound collection device 10, the cost is low.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. be. For example, the specific embodiment of dispersion/integration of devices is not limited to the above embodiments, and all or part of the devices may be configured by functionally or physically distributing/integrating them in arbitrary units. Can be done. In addition, new embodiments created by arbitrary combinations of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiment resulting from the combination have the effects of the original embodiment.

In the above description, the voice analysis device 100 is used to analyze voices in a discussion conducted with participants surrounding the sound collection device 10, but it can also be applied to other uses. For example, the speech analysis device 100 can be applied to a situation where one commentator gives an explanation to a plurality of audiences at a poster session or the like.

The processors of the voice analysis device 100, the sound collection device 10, and the communication terminal 20 are the main bodies of each step (process) included in the voice analysis method shown in FIG. That is, the processors of the speech analysis device 100, the sound collection device 10, and the communication terminal 20 read a program for executing the speech analysis method shown in FIG. The voice analysis method shown in FIG. 9 is executed by controlling each part of the sound device 10 and the communication terminal 20. Some of the steps included in the speech analysis method shown in FIG. 9 may be omitted, the order of the steps may be changed, or a plurality of steps may be performed in parallel.

S speech analysis system 100 speech analysis device 110 control section 111 setting section 112 speech acquisition section 114 analysis section 115 output section 10 sound collection device 20 communication terminal 21 display section

Claims (7)

a storage unit that stores information on a plurality of participants surrounding the sound collection device in association with each of the plurality of sound collection devices;
an acquisition unit that acquires, from each of the plurality of sound collection devices , voices uttered by the plurality of participants associated with the sound collection device in the storage unit ;
an analysis unit that identifies utterances of each of the plurality of participants associated with the plurality of sound collection devices in the storage unit in the audio acquired by the acquisition unit from each of the plurality of sound collection devices ;
a setting unit that sets the position of each of the plurality of sound collecting devices based on signals exchanged between the plurality of sound collecting devices;
Information indicating the status of the utterances of the plurality of participants associated with the sound collection devices in the storage section at a position on the display section corresponding to the position of each of the plurality of sound collection devices set by the setting section. an output section that displays
A voice analysis device with The output unit displays information indicating the status of the utterance at a position corresponding to the position of each of the plurality of sound collecting devices on a map showing the positions of the plurality of sound collecting devices displayed on the display unit. The speech analysis device according to claim 1 . The speech analysis device according to claim 1 or 2 , wherein the output section causes the display section provided in a communication terminal that communicates with the speech analysis device to display information indicating the status of the utterance. The voice analysis device according to claim 1 or 2 , wherein the output unit causes the display unit provided in each of the plurality of sound collection devices to display information indicating the state of the utterance. The processor
In a storage unit that stores information of a plurality of participants surrounding the sound collection device in association with each of the plurality of sound collection devices from each of the plurality of sound collection devices, the plurality of participants associated with the sound collection device; a step of obtaining the voice emitted by the
a step of identifying the utterances of each of the plurality of participants associated with the plurality of sound collection devices in the storage unit in the audio obtained in the acquiring step from each of the plurality of sound collection devices ;
setting the position of each of the plurality of sound collecting devices based on signals exchanged between the plurality of sound collecting devices;
Indicating the status of the utterances of the plurality of participants associated with the plurality of sound collecting devices in the storage section at a position on the display section corresponding to the position of each of the plurality of sound collecting devices set in the setting step. a step of displaying information;
A speech analysis method that performs. to the computer,
In a storage unit that stores information of a plurality of participants surrounding the sound collection device in association with each of the plurality of sound collection devices from each of the plurality of sound collection devices, the plurality of participants associated with the sound collection device; a step of obtaining the voice emitted by the
a step of identifying the utterances of each of the plurality of participants associated with the plurality of sound collection devices in the storage unit in the audio obtained in the acquiring step from each of the plurality of sound collection devices ;
setting the position of each of the plurality of sound collecting devices based on signals exchanged between the plurality of sound collecting devices;
Indicating the status of the utterances of the plurality of participants associated with the plurality of sound collecting devices in the storage section at a position on the display section corresponding to the position of each of the plurality of sound collecting devices set in the setting step. a step of displaying information;
A speech analysis program that runs comprising a voice analysis device and a communication terminal capable of communicating with the voice analysis device,
The communication terminal has a display unit that displays information,
The voice analysis device includes:
a storage unit that stores information on a plurality of participants surrounding the sound collection device in association with each of the plurality of sound collection devices;
an acquisition unit that acquires, from each of the plurality of sound collection devices , voices uttered by the plurality of participants associated with the sound collection device in the storage unit ;
an analysis unit that identifies utterances of each of the plurality of participants associated with the plurality of sound collection devices in the storage unit in the audio acquired by the acquisition unit from each of the plurality of sound collection devices ;
a setting unit that sets the position of each of the plurality of sound collecting devices based on signals exchanged between the plurality of sound collecting devices;
Indicating the status of the utterances of the plurality of participants associated with the plurality of sound collecting devices in the storage section at a position on the display section corresponding to the position of each of the plurality of sound collecting devices set by the setting section. an output section that displays information;
A voice analysis system with

Images