JP2020016893A - Voice analysis device, voice analysis method, voice analysis program, and voice analysis system - Google Patents

Abstract

To provide a voice analysis device, a voice analysis method, a voice analysis program, and a voice analysis system, which can output information indicating a state of speech spoken in a plurality of groups.SOLUTION: In a voice analysis system in which a sound collecting device, a communication terminal, and a voice analysis device are connected through a network, a voice analysis device 100 includes: a voice acquisition unit 112 that acquires, from a plurality of sound collecting devices, voice uttered by a plurality of participants; an analysis unit 114 that specifies, in the voice, speech spoken by the plurality of respective participants; and an output unit 115 that displays, on a display unit, information indicating a state of the speech in association with the plurality of respective sound collecting devices.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a voice analysis device for analyzing voice, a voice analysis method, a voice analysis program, and a voice analysis system.

  As a method for analyzing a discussion in a group learning or a meeting, a Harkness method (also called a Harkness method) is known (for example, see Non-Patent Document 1). In the Harkness method, the transition of the speech of each participant is recorded by a line. Thus, it is possible to analyze each participant's contribution to the discussion and the relationship with others. The Harkness method can also be effectively applied to active learning in which students study independently.

Paul Sevigny, "Extreme Discussion Circles: Preparing ESL Students for" The Harkness Method "", Polyglossia, Language Education Center, Ritsumeikan Asia Pacific University, October 2012, No. 23, p. 181-191

  When performing an analysis such as the Harkness Law in a school or an organization, a plurality of groups often discuss at the same time. In this case, since the assistants (teachers, facilitators, etc.) of the discussion are assigned to a plurality of groups, there is a problem that it is difficult to grasp the state of remarks in the discussion of each group.

  The present invention has been made in view of these points, and an object of the present invention is to provide a voice analysis device, a voice analysis method, a voice analysis program, and a voice analysis system that can output information indicating the state of speech in a plurality of groups. I do.

  A voice analysis device according to a first aspect of the present invention is an acquisition unit that obtains voices emitted by a plurality of participants from a plurality of sound collection devices, and an analysis that specifies a statement of each of the plurality of participants in the voice. And an output unit that causes the display unit to display information indicating the state of the utterance in association with each of the plurality of sound collection devices.

  The output unit may cause the display unit to display information indicating a speech amount for each sound collecting device or each participant as information indicating a situation of the speech.

  The output unit may cause the display unit provided in a communication terminal that communicates with the voice analysis device to display information indicating the status of the utterance.

  The output unit may display information indicating the state of the utterance at a position on the display unit provided in the communication terminal corresponding to a position of each of the plurality of sound collection devices.

  The output unit, when a predetermined person approaches any of the plurality of sound collection devices, the person on a position on the display unit provided in the communication terminal corresponding to the position of the sound collection device, May be displayed.

  The sound analysis device may further include a setting unit that sets a position of each of the plurality of sound collection devices based on a signal transmitted and received between the plurality of sound collection devices.

  The output unit may display information indicating the state of the utterance on the display unit provided in each of the plurality of sound collection devices.

  The output unit may cause the display unit provided in each of the plurality of sound collection devices to display information indicating the state of the utterance for each of the sound collection devices.

  The output unit may cause the display unit provided for each of the plurality of participants in each of the plurality of sound collection devices to display information indicating the state of the utterance for each of the participants.

  In the voice analysis method according to a second aspect of the present invention, the processor obtains voices uttered by a plurality of participants from a plurality of sound collectors, and specifies a speech of each of the plurality of participants in the voice. And displaying the information indicating the state of the utterance on a display unit in association with each of the plurality of sound collection devices.

  A voice analysis program according to a third aspect of the present invention includes the steps of: causing a computer to obtain voices uttered by a plurality of participants from a plurality of sound collecting devices; and specifying the utterances of the plurality of participants in the voice. And displaying the information indicating the state of the utterance on a display unit in association with each of the plurality of sound collection devices.

  A voice analysis system according to a fourth aspect of the present invention includes a voice analysis device, a communication terminal capable of communicating with the voice analysis device, and a plurality of sound collection devices, and includes a communication terminal and a plurality of sound collection devices. At least one has a display unit that displays information, the voice analysis device is an acquisition unit that obtains voices emitted by a plurality of participants from the plurality of sound collection devices, and the plurality of participations in the voice. An analysis unit that specifies the utterance of each person; and an output unit that causes the display unit to display information indicating the status of the utterance in association with each of the plurality of sound collection devices.

  Advantageous Effects of Invention According to the present invention, there is an effect that information indicating the status of utterances in a plurality of groups can be output.

It is a schematic diagram of a speech analysis system according to the present embodiment. It is a block diagram of a speech analysis system according to the present embodiment. It is a schematic diagram of the voice analysis method performed by the voice analysis system according to the present embodiment. FIG. 6 is a schematic diagram of a setting method of group position information by a setting unit. It is a front view of the display part of the communication terminal which is displaying the participant setting screen. It is a side view of a sound collection device which displays a situation of a statement of a group and a participant. It is a front view of the display part of the communication terminal which displays the situation of the group's remark. It is a front view of the display part of the communication terminal which displays the situation of the speech of the participant. FIG. 4 is a sequence diagram of a voice analysis method performed by the voice analysis system according to the embodiment.

[Overview of Speech Analysis System S]
FIG. 1 is a schematic diagram of a voice analysis system S according to the present embodiment. The voice analysis system S includes a voice analysis device 100, a sound collection device 10, and a communication terminal 20. The numbers of the sound collection devices 10 and the communication terminals 20 included in the voice analysis system S are not limited. The voice analysis system S may include other devices such as a server and a terminal.

  The voice analyzer 100, the sound collector 10, and the communication terminal 20 are connected via a network N such as a local area network or the Internet. At least a part of the voice analysis device 100, the sound collection device 10, and the communication terminal 20 may be directly connected without using the network N.

  The sound collection device 10 includes a microphone array including a plurality of sound collection units (microphones) arranged in different directions. For example, a microphone array includes eight microphones equally spaced on the same circumference in a horizontal plane with respect to the ground. The sound collection device 10 transmits the sound acquired using the microphone array to the sound analysis device 100 as data.

  The communication terminal 20 is a communication device capable of performing wired or wireless communication. The communication terminal 20 is, for example, a mobile terminal such as a smartphone terminal or a computer terminal such as a personal computer. The communication terminal 20 receives the setting of the analysis condition from the analyst, and displays the analysis result by the voice analysis device 100. The communication terminal 20 may be held by an analyst or assistant in the discussion, or may be located near each sound collector 10.

  The voice analysis device 100 is a computer that analyzes the voice acquired by the sound collection device 10 using a voice analysis method described below. Further, the voice analysis device 100 transmits the result of the voice analysis to the communication terminal 20.

[Configuration of Speech Analysis System S]
FIG. 2 is a block diagram of the speech analysis system S according to the present embodiment. In FIG. 2, arrows indicate main data flows, and there may be data flows not shown in FIG. In FIG. 2, each block shows not a configuration of a hardware (device) unit but a configuration of a function unit. Thus, the blocks shown in FIG. 2 may be implemented in a single device, or may be separately implemented in multiple devices. The transfer of data between blocks may be performed via any means such as a data bus, a network, a portable storage medium, and the like.

  The sound collecting device 10 has, in addition to the sound collecting unit described above, an overall lamp 11 for displaying the status of the entire group and an individual lamp 12 for displaying the status of each participant. The number of the entire lamps 11 is at least one per one sound collecting device 10. The individual lamps 12 are the number of participants that can be processed by at least one sound collection device 10 (ie, the number of participants that can belong to one discussion group). Each individual lamp 12 is provided so as to correspond to each participant surrounding the sound collecting device 10. For example, at least one individual lamp 12 corresponding to the participant is provided in front of each participant on the sound collecting device 10.

  The overall lamp 11 and the individual lamps 12 are display units that emit light under the control of the voice analyzer 100, respectively. For example, the overall lamp 11 and the individual lamp 12 can blink the light, change the color of the light, or change the intensity of the light according to the signal received from the audio analysis device 100. Other display devices, such as a liquid crystal display, may be used as the overall lamp 11 and the individual lamp 12 as long as it is possible to show the state of a statement to a participant or an assistant.

  The communication terminal 20 has a display unit 21 for displaying various information and an operation unit 22 for receiving an operation by an analyst. The display unit 21 includes a display device such as a liquid crystal display and an organic electroluminescent (OLED) display. The operation unit 22 includes operation members such as buttons, switches, and dials. The display unit 21 and the operation unit 22 may be integrally configured by using a touch screen capable of detecting the position of the contact by the analyst as the display unit 21.

  The voice analysis device 100 has a control unit 110, a communication unit 120, and a storage unit 130. The control unit 110 includes a setting unit 111, a voice acquisition unit 112, a sound source localization unit 113, an analysis unit 114, and an output unit 115. The storage unit 130 includes a setting information storage unit 131, a voice storage unit 132, and an analysis result storage unit 133.

  The communication unit 120 is a communication interface for communicating with the sound collection device 10 and the communication terminal 20 via the network N. The communication unit 120 includes a processor, a connector, an electric circuit, and the like for executing communication. The communication unit 120 performs predetermined processing on a communication signal received from the outside to acquire data, and inputs the acquired data to the control unit 110. The communication unit 120 performs a predetermined process on the data input from the control unit 110 to generate a communication signal, and transmits the generated communication signal to the outside.

  The storage unit 130 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, and the like. The storage unit 130 stores a program to be executed by the control unit 110 in advance. The storage unit 130 may be provided outside the voice analysis device 100, and in that case, data may be exchanged with the control unit 110 via the communication unit 120.

  The setting information storage unit 131 stores setting information indicating analysis conditions including positions of groups and participants. The sound storage unit 132 stores the sound obtained by the sound collection device 10. The analysis result storage unit 133 stores an analysis result indicating a result of analyzing the voice. Each of the setting information storage unit 131, the voice storage unit 132, and the analysis result storage unit 133 may be a storage area on the storage unit 130, or may be a database configured on the storage unit 130.

  The control unit 110 is, for example, a processor such as a CPU (Central Processing Unit), and executes a program stored in the storage unit 130, so that the setting unit 111, the voice acquisition unit 112, the sound source localization unit 113, the analysis unit 114, It functions as the output unit 115. The functions of the setting unit 111, the voice acquisition unit 112, the sound source localization unit 113, the analysis unit 114, and the output unit 115 will be described later with reference to FIGS. At least a part of the function of the control unit 110 may be executed by an electric circuit. Further, at least a part of the function of the control unit 110 may be executed by a program executed via a network.

  The voice analysis system S according to the present embodiment is not limited to the specific configuration shown in FIG. For example, the voice analysis device 100 is not limited to one device, and may be configured by connecting two or more physically separated devices by wire or wirelessly.

[Description of voice analysis method]
FIG. 3 is a schematic diagram of a voice analysis method performed by the voice analysis system S according to the present embodiment. First, the analyst operates the operation unit 22 of the communication terminal 20 to set analysis conditions. The communication terminal 20 receives the setting of the analysis condition from the analyst, and transmits the setting to the voice analysis device 100 as setting information (a). The setting unit 111 of the voice analysis device 100 causes the setting information storage unit 131 to store the setting information received from the communication terminal 20 or the setting information specified by the setting unit 111 itself.

  The setting information includes participant position information indicating the position of each participant (that is, each of a plurality of participants) in a group related to one sound collection device 10 and participant position information indicating the position of each group (that is, each of a plurality of groups) that simultaneously discusses. And group position information indicating the position. The position of each group corresponds to the position of each sound collecting device 10 (that is, each of the plurality of sound collecting devices 10). For example, the participant position information is information indicating the number of participants in the discussion to be analyzed and the direction in which each participant is located based on the sound collection device 10. For example, the group position information is information indicating the number of groups to be analyzed and the relative or absolute position of each group.

  FIGS. 4A and 4B are schematic diagrams of a setting method of the group position information by the setting unit 111. FIG. FIG. 4A shows the display unit 21 of the communication terminal 20 displaying the group setting screen A for setting the group position information. The communication terminal 20 displays the group setting screen A on the display unit 21 when setting the group position information, and accepts the setting of the group position information by the analyst. The group setting screen A includes a position setting area A1, a reference position A2, a completion button A3, and an automatic setting button A4.

  The position setting area A1 is an area for setting the position of each group G based on a predetermined reference position A2 for discussions performed simultaneously (for example, discussions performed in the same room). For example, the position setting area A1 is a rectangular area including the reference position A2 (for example, the position of the lectern) as shown in FIG. The reference position A2 may be specified by the analyst on the position setting area A1, or may be registered in the voice analyzer 100 in advance.

  The analyst sets the position of each group G in the position setting area A1 by operating the operation unit 22 of the communication terminal 20. For example, when the analyst presses somewhere in the position setting area A1, the setting unit 111 of the voice analysis device 100 sets the position pressed by the analyst as the position of one group G. At the position set for each group G, identification information (here, G1 to G4) for identifying each group G is assigned and displayed. The identification information of each group G may be input by the analyst, or may be automatically determined by the setting unit 111. In the example of FIG. 4A, four groups G1 to G4 are set.

  The completion button A3 and the automatic setting button A4 are virtual buttons displayed on the display unit 21, respectively. When the analyst presses the complete button A3, the setting unit 111 stores the position of each group G set on the group setting screen A in the setting information storage unit 131 as group position information, so that each group G Set the position of. Then, the setting unit 111 causes the communication terminal 20 to display the participant setting screen C shown in FIG. 5 for each group G set on the group setting screen A.

  When the analyst presses the automatic setting button A4, the setting unit 111 automatically sets the position of each group G based on the information acquired from the sound collection device 10. FIG. 4B is a schematic diagram of a method in which the setting unit 111 automatically sets the position of each group G. For example, the setting unit 111 automatically sets the position of each sound collecting device 10, that is, the position of each group G, based on a sound or radio wave signal transmitted and received between the sound collecting devices 10.

  For automatic setting, each sound collection device 10 includes a transmitter that generates a signal of a predetermined sound (sound wave or ultrasonic wave) or a predetermined radio wave (for example, a short-range wireless communication radio wave such as Bluetooth (registered trademark)). The sound collecting apparatus 10 includes a receiver that receives the signal. The setting unit 111 sequentially generates a signal from the transmitter of each sound collection device 10. The setting unit 111 acquires a time and an intensity at which a signal is detected in the receiver of each sound collection device 10.

  Next, the setting unit 111 specifies, for each sound collection device 10, the sound collection device 10 that is the signal generation source using the time at which the signal is detected, and uses the intensity at which the signal is detected to determine the sound collection device of the generation source. The distance B to 10 is calculated. Then, based on the calculated distance B between the sound collection devices 10, the setting unit 111 determines the position of each sound collection device 10, that is, the position of each group G by, for example, triangulation. The setting unit 111 sets the position of each group G by storing the determined position of each group G in the setting information storage unit 131 as group position information. Then, the setting unit 111 causes the communication terminal 20 to display the participant setting screen C shown in FIG.

  Here, the method of determining the position of the sound collection device 10 by exchanging sound or radio signals between the sound collection devices 10 has been described as an example, but the setting unit 111 determines the sound collection device 10 by another method. May be. For example, a sound or radio signal may be generated from a device provided outside the sound collecting device 10.

  The setting unit 111 may perform the automatic setting and the manual setting of the position of the group G in combination. In this case, for example, the setting unit 111 displays the position of each group G set by the automatic setting of FIG. 4B in the position setting area A1 of FIG. 4A, and further sets the manual setting by the analyst. Accept. Thereby, the position of each group G set by the automatic setting can be corrected by the manual setting, and the position of each group G can be set more reliably.

  FIG. 5 is a front view of the display unit 21 of the communication terminal 20 displaying the participant setting screen C for setting the participant position information. The communication terminal 20 displays the participant setting screen C on the display unit 21 when setting the participant position information, and allows the analyst to set the participant position information for each group set on the group setting screen A. Accept. The participant setting screen C includes a position setting area C1, a start button C2, and an end button C3. The position setting area C1 is an area for setting the direction in which each participant U is actually located based on the sound collector 10 in the discussion of the analysis target. For example, the position setting area C1 represents a circle centered on the position of the sound collecting device 10 as shown in FIG. 5, and further represents an angle along the circle with respect to the sound collecting device 10.

  The analyst operates the operation unit 22 of the communication terminal 20 to input the position of each participant U in the position setting area C1. The setting unit 111 sets the position of each participant U by storing the position of each participant U input by the analyst in the setting information storage unit 131 as participant position information. In the vicinity of the position set for each participant U, identification information (here, U1 to U4) for identifying each participant U is assigned and displayed. In the example of FIG. 5, four participants U1 to U4 are set. The part corresponding to each participant U in the position setting area C1 is displayed in a different color for each participant. Thereby, the analyst can easily recognize the direction in which each participant U is set.

  The start button C2 and the end button C3 are virtual buttons displayed on the display unit 21, respectively. When the analyst presses the start button C2, the communication terminal 20 transmits a signal of a start instruction to the voice analysis device 100. When the end button C3 is pressed by the analyst, the communication terminal 20 transmits a signal of an end instruction to the voice analyzer 100. In this embodiment, the process from the start instruction to the end instruction by the analyst is one discussion.

  When receiving the signal of the start instruction from the communication terminal 20, the audio acquisition unit 112 of the audio analysis device 100 transmits a signal to instruct the acquisition of the audio to the sound collection device 10 (b). When the sound collection device 10 receives a signal instructing to acquire sound from the sound analysis device 100, the sound collection device 10 starts acquiring sound. Further, when receiving the signal of the end instruction from the communication terminal 20, the audio acquisition unit 112 of the audio analysis device 100 transmits a signal instructing the end of the audio acquisition to the sound collection device 10. The sound collection device 10 ends the acquisition of the sound when receiving a signal instructing to end the acquisition of the sound from the sound analysis device 100.

  The sound collection device 10 acquires sound from each of the plurality of sound collection units, and records the sound as sound of each channel corresponding to each sound collection unit. Then, the sound collection device 10 transmits the acquired sounds of the plurality of channels to the sound analysis device 100 (c). The sound collection device 10 may sequentially transmit the acquired voice, or may transmit a predetermined amount of voice or a predetermined time of voice. The sound acquisition unit 112 of the sound analysis device 100 receives a sound from the sound collection device 10 and stores the sound in the sound storage unit 132.

  The voice analysis device 100 analyzes the voice of each group acquired from the sound collection device 10 sequentially, that is, by real-time processing. For example, the voice analyzer 100 sequentially analyzes voices for a predetermined time (for example, 30 seconds) in the past retroactively from the current time.

  When analyzing the sound, the sound source localization unit 113 first performs the sound source localization based on the sound of the plurality of channels acquired by the sound acquisition unit 112 (d). The sound source localization is a process of estimating the direction of the sound source included in the sound acquired by the sound acquiring unit 112 at every time (for example, every 10 ms to 100 ms). The sound source localization unit 113 associates the direction of the sound source estimated for each time with the direction of the participant indicated by the setting information stored in the setting information storage unit 131.

  The sound source localization unit 113 can use a known sound source localization method such as a MUSIC (Multiple Signal Classification) method or a beam forming method as long as the direction of the sound source can be specified based on the sound acquired from the sound collection device 10. .

  Next, the analysis unit 114 analyzes the sound based on the sound acquired by the sound acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113 (e). Specifically, first, the analysis unit 114 performs time-based (for example, 10 milliseconds to 100 milliseconds) in the discussion of the analysis target based on the sound acquired by the sound acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113. (Every second), it is determined which participant uttered (uttered). The analysis unit 114 specifies a continuous period from the start to the end of one participant until the end of the participant, and causes the analysis result storage unit 133 to store the continuous period. When a plurality of participants speak at the same time, the analysis unit 114 specifies a speech period for each participant.

  In addition, the analysis unit 114 calculates the utterance amount of each participant for each time, and causes the analysis result storage unit 133 to store the utterance amount. Specifically, in a certain time window (for example, 5 seconds), the analysis unit 114 calculates a value obtained by dividing the length of time during which the participant made a utterance by the length of the time window as the amount of utterance for each time. I do. Then, the analysis unit 114 calculates the amount of speech per time for each participant while shifting the time window by a predetermined time (for example, 1 second) from the start time of the discussion to the end time (current in the case of real-time processing). repeat. In addition, the analysis unit 114 calculates the total value of the utterance amounts of the participants belonging to the group for each time as the utterance amount (activity) of the group for each time.

  Then, the analysis unit 114 specifies the state of the utterance of the participant and the group using the utterance amount of the participant and the group. The analysis unit 114 may include, as a participant's utterance status, for example, a total value or an integral value of the utterance amount of the participant for a predetermined period (for example, 20 seconds) retroactively from the present, or a participation of the total value or the integral value of the utterance amount Calculate the ratio (ie, relative value) between the participants. The situation of the participant's statement calculated in this way can be used as an index indicating the degree of contribution of each participant to the discussion.

  In addition, the analysis unit 114 may determine, for example, the total value or the integral value of the utterance amount of the group for a predetermined period (for example, 20 seconds) retroactively from the present, or the group of the total value or the integral value of the utterance amount The ratio between them (ie, the relative value) is calculated. The state of remarks of the group calculated in this way can be used as an index indicating the degree of the excitement of the discussion in each group. The analysis unit 114 causes the analysis result storage unit 133 to store, as an analysis result, status information indicating the status of the speech of each participant and each group.

  The atmosphere of the group may be used as the state of the group's remark. The atmosphere of the group is information indicating the atmosphere of whether the number of speaker changes is large or small in the discussion. Specifically, the analysis unit 114 calculates the average value of the number of times of the speech transition between the same participants U (that is, switching to another speech period after a certain speech period), and the An average value of the number of transitions is calculated, and the ratio between them is specified as the atmosphere of the group. For example, when the ratio of utterance transitions between the same participants U is large, the group tends to have one participant uttering a long time. Participants tend to speak in turn.

  The output unit 115 associates the speech situation of each participant and each group identified by the analysis unit 114 with the sound collection device 10 corresponding to the participant and the group, and displays the sound collection device 10 and the communication terminal 20. Display on the section. Specifically, the output unit 115 reads out the status information stored in the analysis result storage unit 133 and transmits the read status information to the sound collection device 10 so that the information indicating the status of the utterance identified by the analysis unit 114 is output to the entire lamp. The control for displaying on the individual lamps 11 and the individual lamps 12 is performed (f).

  In addition, the output unit 115 reads out the status information stored in the analysis result storage unit 133 and transmits the status information to the communication terminal 20, thereby causing the display unit 21 to display information indicating the status of the utterance specified by the analysis unit 114. Control is performed (g). A method of outputting the state of the utterance by the output unit 115 will be described below with reference to FIGS.

[Explanation of how to display the status of remark]
FIG. 6 is a side view of the sound collecting device 10 displaying the states of the statements of the group and the participants. The output unit 115 uses the overall lamp 11 and the individual lamps provided in the sound collection device 10 to display information indicating the state of the group and the participants' speech. Specifically, the output unit 115 causes the sound collecting device 10 to display the state of the group utterance by causing the entire lamp 11 to emit predetermined light corresponding to the state of the group utterance. For example, the output unit 115 changes the blinking speed of light, the color of light, or the intensity of light according to the total value or the integrated value of the utterance amounts of the group. Thus, the participants and assistants of the discussion can easily grasp the state of the remarks of the entire group.

  The output unit 115 also displays the state of each participant's statement on the sound collection device 10 by generating predetermined light corresponding to the participant's statement from the individual lamp 12 corresponding to the participant. Let it. That is, the output unit 115 causes the individual lamp 12 provided at a position (for example, in front of each participant) corresponding to each participant to emit light in accordance with the state of speech for each participant. For example, the output unit 115 changes the flashing speed of light, the color of light, or the intensity of light according to the total value or the integrated value of the speech amounts of the participants. Thereby, the participants of the discussion can objectively recognize the state of their own remarks, and the assistants of the discussion can grasp the contribution to the discussion for each participant.

  FIG. 7 is a front view of the display unit 21 of the communication terminal 20 displaying the status of the group utterance. The output unit 115 causes the display unit 21 of the communication terminal 20 to display information indicating the status of the group utterance as the overall status screen D. The overall status screen D includes a circle D1 indicating the status of a statement for each group and an icon D2 indicating the position of the assistant.

  The output unit 115 displays a circle D1 in a display mode corresponding to the state of the utterance of the group at a position on the display unit 21 corresponding to the position of each group indicated by the group position information stored in the setting information storage unit 131. . That is, the output unit 115 displays information indicating the state of the utterance of the group on a map indicating the position of each group (each sound collecting device 10). For example, the output unit 115 changes the color, shape, pattern, and the like of the circle D1 according to the total value or the integrated value of the utterance amounts of the group. Thereby, the analyst or assistant of the discussion can easily grasp the state of the utterance of each group.

  The output unit 115 also displays an icon D2 indicating the position of the assistant at a position on the display unit 21 corresponding to the position of the assistant of the discussion. In order to detect the position of the assistant, the output unit 115 uses, for example, a signal transmitted and received between the sound collector 10 and the assistant. In this case, the assistant holds a transmitter that emits a predetermined signal by radio waves or ultrasonic waves of wireless communication such as Bluetooth, and the sound collection device 10 includes a receiver that receives the signal.

  The output unit 115 assists the sound collection device 10 when a signal from the transmitter of the assistant can be received by the receiver of the sound collection device 10 or when the intensity of the received signal is equal to or higher than a predetermined threshold. Detects that the person has approached. In addition, the output unit 115 outputs the signal to the sound collection device 10 when the receiver of the sound collection device 10 cannot receive a signal from the transmitter of the assistant or when the intensity of receiving the signal becomes less than a predetermined threshold. It detects that the assistant has left from.

  The output unit 115 displays an icon D2 on the display unit 21 in the vicinity of the sound collection device 10 (group) when detecting that the assistant approaches one of the sound collection devices 10. This allows the analyst of the discussion to analyze changes in the state of speech of each group between when the assistant is close to the group and when it is not, and to evaluate the evaluation of the assistant. It can be carried out.

  FIG. 8 is a front view of the display unit 21 of the communication terminal 20 displaying the status of the participant's remark. When any group is designated by the analyst or the assistant on the overall situation screen D, the output unit 115 communicates information indicating the speech situation of each participant belonging to the group as an individual situation screen E. It is displayed on the display unit 21 of the terminal 20. The individual situation screen E includes an area E1 indicating the speech situation of each participant. The area E1 includes a plurality of areas corresponding to a plurality of participants.

  The output unit 115 displays an area E1 in a display mode corresponding to the state of a statement for each participant. For example, the output unit 115 changes a color, a pattern, and the like of an area corresponding to each participant in the area E1 according to a total value or an integral value of the speech amounts of each participant. Further, the output unit 115 displays identification information (here, U1 to U4) for identifying each participant in the vicinity of an area corresponding to each participant. Thereby, the analyst or assistant of the discussion can easily grasp the state of the speech of each participant belonging to one group.

  The output method of the information indicating the state of the speech of the group and the participant shown in FIGS. 6 to 8 is an example, and the information can be displayed in association with the sound collecting device 10 corresponding to the group and the participant. Other output methods may be used. The output unit 115 does not necessarily need to display the information indicating the state of the utterance by both the sound collection device 10 and the communication terminal 20, and may display the information indicating the state of the utterance by at least one of them. The output unit 115 may output information indicating the state of the utterance by other means, such as printing by a printer or recording data in a storage device.

[Speech analysis method sequence]
FIG. 9 is a sequence diagram of a voice analysis method performed by the voice analysis system S according to the present embodiment. First, the communication terminal 20 receives the setting of the analysis condition from the analyst, and transmits the setting to the voice analysis device 100 as setting information (S11). The setting unit 111 of the voice analysis device 100 causes the setting information storage unit 131 to store the setting information acquired from the communication terminal 20 or the setting information specified by the setting unit 111 itself. The setting information includes participant position information indicating the position of each participant in a group related to one sound collection device 10 and group position information indicating the position of each group (that is, each sound collection device 10) that is simultaneously discussing. .

  Next, the sound acquisition unit 112 of the sound analysis device 100 transmits a signal for instructing acquisition of sound to the sound collection device 10 (S12). When the sound collection device 10 receives a signal instructing acquisition of a sound from the sound analysis device 100, the sound collection device 10 starts recording the sound using the plurality of sound collection units, and the recorded sound of the plurality of channels is output to the sound analysis device 100. (S13). The sound acquisition unit 112 of the sound analysis device 100 receives a sound from the sound collection device 10 and stores the sound in the sound storage unit 132.

  The voice analyzer 100 analyzes the obtained voice sequentially, that is, by real-time processing. When analyzing the sound, the sound source localization unit 113 first performs sound source localization based on the sound acquired by the sound acquisition unit 112 (S14).

  Next, the analysis unit 114 determines which participant has made a speech at each time based on the sound acquired by the sound acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113. Calculate the amount of speech for each time. Then, the analysis unit 114 specifies the utterance status of the participant using the calculated utterance amount (S15). The state of the participant's statement is, for example, the total value or integral value of the participant's statement amount, or the ratio (ie, relative value) between the participants of the total value or the integral value of the participant's statement amount.

  In addition, the analysis unit 114 calculates the utterance amount of the group for each time by summing the utterance amount of the participant for each time for each group, and specifies the utterance state of the group using the calculated utterance amount ( S16). The state of the utterance of the group is, for example, a total value or an integral value of the utterance amount of the group, or a ratio (that is, a relative value) between the total value or the integral value of the utterance amount of the group. The analysis unit 114 causes the analysis result storage unit 133 to store, as an analysis result, status information indicating the status of the speech of each participant and each group.

  The output unit 115 displays information indicating the state of the utterance of each participant and each group by transmitting the state information indicating the state of the utterance of each participant and each group to the sound collection device 10 and the communication terminal 20. Control is performed (S17).

  The sound collection device 10 displays information indicating the state of speech of each participant and each group by causing the entire lamp 11 and the individual lamp 12 to emit light in accordance with the status information received from the voice analysis device 100 (S18). The communication terminal 20 causes the display unit 21 to display an overall status screen D and an individual status screen E indicating information indicating the status of the utterance of each participant and each group according to the display information received from the voice analysis device 100 (S19). ). The voice analyzer 100 analyzes the voice by real-time processing by repeating steps S12 to S19 at predetermined time intervals.

[Effects of the present embodiment]
The voice analysis device 100 according to the present embodiment specifies the state of utterance in a group related to the sound collection device 10 based on sounds acquired using the sound collection device 10 having a plurality of sound collection units, and Alternatively, it is displayed on the communication terminal 20. Therefore, even when a plurality of groups are discussing at the same time, the analyst or assistant of the discussion refers to the display on the sound collection device 10 or the communication terminal 20 to determine the state of the utterance in the plurality of groups. Can be easily grasped.

  When the voice analysis device 100 displays the state of speech on the sound collection device 10, in addition to the analyst or assistant of the discussion, the participant can also objectively know the status of speech of the group to which the participant belongs. . In addition, by displaying the state of the utterance on the individual lamp 12 provided for each participant on the sound collecting device 10, the participant can easily distinguish the state of his / her own utterance from the state of the utterance of other participants. be able to.

  When the voice analysis device 100 displays the state of the statement on the communication terminal 20, the analyst or assistant of the discussion can view the state of the statement of all groups from a bird's-eye view. Further, since there is no need to provide a lamp for each sound collecting device 10, the cost is low.

  As described above, the present invention has been described using the embodiment, but the technical scope of the present invention is not limited to the scope described in the above embodiment, and various modifications and changes are possible within the scope of the gist. is there. For example, the specific embodiment of the dispersion / integration of the apparatus is not limited to the above-described embodiment, and all or a part of the apparatus may be functionally or physically dispersed / integrated in an arbitrary unit. Can be. Further, new embodiments that are generated by arbitrary combinations of the plurality of embodiments are also included in the embodiments of the present invention. The effect of the new embodiment caused by the combination has the effect of the original embodiment.

  In the above description, the voice analysis device 100 is used for analyzing voice in a discussion held by participants around the sound collection device 10, but can be applied to other uses. For example, the voice analysis device 100 can be applied to a situation where one commentator explains to a plurality of spectators in a poster session or the like.

  The processor of the voice analysis device 100, the sound collection device 10, and the communication terminal 20 is a main component of each step (process) included in the voice analysis method illustrated in FIG. That is, the processor of the voice analysis device 100, the sound collection device 10, and the communication terminal 20 reads a program for executing the voice analysis method illustrated in FIG. 9 from the storage unit, and executes the program to execute the voice analysis device 100, The sound analysis method shown in FIG. 9 is executed by controlling the respective units of the sound device 10 and the communication terminal 20. Some of the steps included in the voice analysis method shown in FIG. 9 may be omitted, the order between the steps may be changed, and a plurality of steps may be performed in parallel.

S voice analysis system 100 voice analysis device 110 control unit 111 setting unit 112 voice acquisition unit 114 analysis unit 115 output unit 10 sound collection device 20 communication terminal 21 display unit

Claims (12)

An acquisition unit that acquires sounds emitted by a plurality of participants from a plurality of sound collection devices;
An analysis unit that specifies the remarks of each of the plurality of participants in the voice,
An output unit that displays information indicating the state of the utterance on a display unit in association with each of the plurality of sound collection devices;
A voice analysis device having:   The voice analysis device according to claim 1, wherein the output unit causes the display unit to display information indicating a speech amount for each of the sound collection devices or the participants as information indicating a situation of the speech.   The voice analysis device according to claim 1, wherein the output unit causes the display unit provided in a communication terminal that communicates with the voice analysis device to display information indicating a status of the utterance.   The voice according to claim 3, wherein the output unit displays information indicating the state of the utterance at a position on the display unit provided in the communication terminal corresponding to a position of each of the plurality of sound collection devices. Analysis equipment.   The output unit, when a predetermined person approaches any of the plurality of sound collection devices, the person on a position on the display unit provided in the communication terminal corresponding to the position of the sound collection device, The voice analysis device according to claim 3, wherein information indicating the approach of the object is displayed.   The voice analysis device according to claim 4, further comprising: a setting unit configured to set a position of each of the plurality of sound collection devices based on a signal transmitted and received between the plurality of sound collection devices.   The voice analysis device according to any one of claims 1 to 6, wherein the output unit causes the display unit provided in each of the plurality of sound collection devices to display information indicating the state of the utterance.   The voice analysis device according to claim 7, wherein the output unit causes the display unit provided in each of the plurality of sound collection devices to display information indicating a status of the utterance for each of the sound collection devices.   The output unit causes the display unit provided for each of the plurality of participants in each of the plurality of sound collection devices to display information indicating a state of the utterance of each of the participants. 9. The voice analysis device according to 8. The processor
Obtaining sounds emitted by a plurality of participants from a plurality of sound collectors;
Identifying the remarks of each of the plurality of participants in the voice;
Causing the display unit to display information indicating the state of the utterance in association with each of the plurality of sound collection devices;
Perform a voice analysis method. On the computer,
Obtaining sounds emitted by a plurality of participants from a plurality of sound collectors;
Identifying the remarks of each of the plurality of participants in the voice;
Causing the display unit to display information indicating the state of the utterance in association with each of the plurality of sound collection devices;
A voice analysis program to run. A voice analysis device, comprising a communication terminal and a plurality of sound collection devices capable of communicating with the voice analysis device,
At least one of the communication terminal and the plurality of sound collection devices has a display unit for displaying information,
The voice analyzer,
An acquisition unit that acquires sounds emitted by a plurality of participants from the plurality of sound collection devices,
An analysis unit that specifies the remarks of each of the plurality of participants in the voice,
An output unit that causes the display unit to display information indicating the state of the utterance in association with each of the plurality of sound collection devices;
A speech analysis system comprising:

Images