JP6589042B1 - Speech analysis apparatus, speech analysis method, speech analysis program, and speech analysis system - Google Patents

Abstract

An object of the present invention is to provide a voice analysis device, a voice analysis method, a voice analysis program, and a voice analysis system that can output information indicating the state of speech in a plurality of groups. The voice analysis device 100 according to an embodiment of the present invention specifies a voice acquisition unit 112 that acquires voices uttered by a plurality of participants from a plurality of sound collectors, and specifies a speech of each of the plurality of participants in the voice. It has an analysis unit 114 and an output unit 115 that displays information indicating the state of speech in association with each of the plurality of sound collectors on the display unit.

Description

  The present invention relates to a speech analysis apparatus, a speech analysis method, a speech analysis program, and a speech analysis system for analyzing speech.

  A Harkness method (also referred to as a Harkness method) is known as a method for analyzing discussions in group learning and meetings (see, for example, Non-Patent Document 1). In the Harkness method, the transition of each participant's speech is recorded as a line. As a result, each participant's contribution to the discussion and the relationship with others can be analyzed. The Harkness method can be effectively applied to active learning where students learn independently.

Paul Sevigny, “Extreme Discussion Circles: Preparing ESL Students for“ The Harkness Method ””, Polyglossia, Center for Language Education, Ritsumeikan Asia Pacific University, October 2012, No. 23, p. 181-191

  When an analysis such as the Harkness method is performed at a school or organization, a plurality of groups often discuss at the same time. In this case, since discussion assistants (teachers, facilitators, etc.) have multiple groups, there is a problem that it is difficult to grasp the state of speech in each group discussion.

  The present invention has been made in view of these points, and an object of the present invention is to provide a voice analysis device, a voice analysis method, a voice analysis program, and a voice analysis system that can output information indicating the state of speech in a plurality of groups. To do.

  The voice analysis device according to the first aspect of the present invention includes an acquisition unit that acquires voices uttered by a plurality of participants from a plurality of sound collection devices, and an analysis that identifies each of the plurality of participants in the voice. And an output unit that causes the display unit to display information indicating the state of the speech in association with each of the plurality of sound collecting devices.

  The output unit may cause the display unit to display information indicating the amount of speech for each sound collecting device or for each participant as information indicating the state of the speech.

  The output unit may display information indicating the state of the utterance on the display unit provided in a communication terminal that communicates with the voice analysis device.

  The output unit may display information indicating the state of the statement at a position on the display unit provided in the communication terminal corresponding to the position of each of the plurality of sound collecting devices.

  When the predetermined person approaches one of the plurality of sound collectors, the output unit is located at a position on the display unit provided in the communication terminal corresponding to the position of the sound collector. Information indicating the approach may be displayed.

  The voice analysis device may further include a setting unit that sets a position of each of the plurality of sound collection devices based on a signal exchanged between the plurality of sound collection devices.

  The output unit may display information indicating the state of the utterance on the display unit provided in each of the plurality of sound collecting devices.

  The output unit may cause the display unit provided in each of the plurality of sound collecting devices to display information indicating the state of the utterance for each sound collecting device.

  The output unit may cause the display unit provided for each of the plurality of participants in each of the plurality of sound collecting devices to display information indicating the state of the speech for each participant.

  In the speech analysis method according to the second aspect of the present invention, the processor obtains speech uttered by a plurality of participants from a plurality of sound collectors, and specifies the speech of each of the plurality of participants in the speech. And a step of displaying information indicating the state of the speech on the display unit in association with each of the plurality of sound collecting devices.

  The speech analysis program according to the third aspect of the present invention specifies, in a computer, a step of acquiring speech uttered by a plurality of participants from a plurality of sound collecting devices, and specifying a speech of each of the plurality of participants in the speech. And a step of displaying information indicating the state of the speech on the display unit in association with each of the plurality of sound collecting devices.

  A speech analysis system according to a fourth aspect of the present invention includes a speech analysis device, a communication terminal capable of communicating with the speech analysis device, and a plurality of sound collection devices, and the communication terminal and the plurality of sound collection devices. At least one has a display unit for displaying information, and the voice analysis device acquires an audio generated by a plurality of participants from the plurality of sound collection devices, and the plurality of participations in the voice And an output unit that displays information indicating the status of the speech on the display unit in association with each of the plurality of sound collecting devices.

  According to the present invention, it is possible to output information indicating the state of speech in a plurality of groups.

It is a mimetic diagram of a voice analysis system concerning this embodiment. It is a block diagram of the speech analysis system concerning this embodiment. It is a schematic diagram of the audio | voice analysis method which the audio | voice analysis system which concerns on this embodiment performs. It is a schematic diagram of the setting method of the group position information by a setting part. It is a front view of the display part of the communication terminal which is displaying the participant setting screen. It is a side view of the sound collector which is displaying the situation of a group and a participant's utterance. It is a front view of the display part of the communication terminal which is displaying the status of a group utterance. It is a front view of the display part of the communication terminal which is displaying the situation of a participant's statement. It is a sequence diagram of the audio | voice analysis method which the audio | voice analysis system which concerns on this embodiment performs.

[Outline of Speech Analysis System S]
FIG. 1 is a schematic diagram of a speech analysis system S according to the present embodiment. The voice analysis system S includes a voice analysis device 100, a sound collection device 10, and a communication terminal 20. The number of the sound collecting devices 10 and the communication terminals 20 included in the voice analysis system S is not limited. The voice analysis system S may include other servers and terminals.

  The voice analysis device 100, the sound collection device 10, and the communication terminal 20 are connected via a network N such as a local area network or the Internet. At least some of the voice analysis device 100, the sound collection device 10, and the communication terminal 20 may be directly connected without going through the network N.

  The sound collection device 10 includes a microphone array including a plurality of sound collection units (microphones) arranged in different directions. For example, the microphone array includes eight microphones arranged at equal intervals on the same circumference in a horizontal plane with respect to the ground. The sound collection device 10 transmits the sound acquired using the microphone array to the sound analysis device 100 as data.

  The communication terminal 20 is a communication device capable of performing wired or wireless communication. The communication terminal 20 is a portable terminal such as a smartphone terminal or a computer terminal such as a personal computer. The communication terminal 20 receives the analysis condition setting from the analyst and displays the analysis result by the voice analysis device 100. The communication terminal 20 may be held by an analyst or an assistant of discussion, or may be disposed in the vicinity of each sound collecting device 10.

  The voice analysis device 100 is a computer that analyzes the voice acquired by the sound collection device 10 by a voice analysis method described later. In addition, the voice analysis device 100 transmits the result of the voice analysis to the communication terminal 20.

[Configuration of speech analysis system S]
FIG. 2 is a block diagram of the speech analysis system S according to the present embodiment. In FIG. 2, arrows indicate main data flows, and there may be data flows not shown in FIG. In FIG. 2, each block represents a functional unit configuration, not a hardware (device) unit configuration. Therefore, the blocks shown in FIG. 2 may be implemented in a single device, or may be separately implemented in a plurality of devices. Data exchange between the blocks may be performed via any means such as a data bus, a network, a portable storage medium, or the like.

  In addition to the above-described sound collection unit, the sound collection device 10 includes an overall lamp 11 that displays the status of the entire group, and an individual lamp 12 that displays the status of each participant. The number of total lamps 11 is at least one for one sound collecting device 10. The individual lamp 12 is the number of participants that can be processed by at least one sound collector 10 (that is, the number of participants that can belong to one discussion group or more). Each individual lamp 12 is provided so as to correspond to each participant surrounding the sound collector 10. For example, at least one individual lamp 12 corresponding to the participant is provided in front of each participant on the sound collecting device 10.

  Each of the general lamp 11 and the individual lamp 12 is a display unit that emits light according to control by the voice analysis device 100. For example, the general lamp 11 and the individual lamp 12 can blink light, change the color of light, or change the intensity of light according to a signal received from the voice analysis device 100. As the general lamp 11 and the individual lamp 12, other display devices such as a liquid crystal display may be used as long as it is possible to indicate the state of speech to the participants or assistants.

  The communication terminal 20 includes a display unit 21 for displaying various information and an operation unit 22 for receiving an operation by an analyst. The display unit 21 includes a display device such as a liquid crystal display and an organic light emitting diode (OLED) display. The operation unit 22 includes operation members such as buttons, switches, and dials. By using a touch screen capable of detecting the position of contact by the analyst as the display unit 21, the display unit 21 and the operation unit 22 may be configured integrally.

  The voice analysis device 100 includes a control unit 110, a communication unit 120, and a storage unit 130. The control unit 110 includes a setting unit 111, a sound acquisition unit 112, a sound source localization unit 113, an analysis unit 114, and an output unit 115. The storage unit 130 includes a setting information storage unit 131, a voice storage unit 132, and an analysis result storage unit 133.

  The communication unit 120 is a communication interface for communicating between the sound collection device 10 and the communication terminal 20 via the network N. The communication unit 120 includes a processor, a connector, an electric circuit, and the like for executing communication. The communication unit 120 performs predetermined processing on a communication signal received from the outside to acquire data, and inputs the acquired data to the control unit 110. In addition, the communication unit 120 performs predetermined processing on the data input from the control unit 110 to generate a communication signal, and transmits the generated communication signal to the outside.

  The storage unit 130 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, and the like. The storage unit 130 stores a program executed by the control unit 110 in advance. The storage unit 130 may be provided outside the voice analysis device 100, and in that case, data may be exchanged with the control unit 110 via the communication unit 120.

  The setting information storage unit 131 stores setting information indicating analysis conditions including the positions of groups and participants. The sound storage unit 132 stores the sound acquired by the sound collection device 10. The analysis result storage unit 133 stores an analysis result indicating a result of analyzing the voice. Each of the setting information storage unit 131, the voice storage unit 132, and the analysis result storage unit 133 may be a storage area on the storage unit 130, or may be a database configured on the storage unit 130.

  The control unit 110 is a processor such as a CPU (Central Processing Unit), for example, and by executing a program stored in the storage unit 130, a setting unit 111, a sound acquisition unit 112, a sound source localization unit 113, an analysis unit 114, and It functions as the output unit 115. The functions of the setting unit 111, the sound acquisition unit 112, the sound source localization unit 113, the analysis unit 114, and the output unit 115 will be described later with reference to FIGS. At least a part of the function of the control unit 110 may be executed by an electric circuit. Further, at least part of the functions of the control unit 110 may be executed by a program executed via a network.

  The voice analysis system S according to the present embodiment is not limited to the specific configuration shown in FIG. For example, the voice analysis device 100 is not limited to one device, and may be configured by connecting two or more physically separated devices in a wired or wireless manner.

[Description of voice analysis method]
FIG. 3 is a schematic diagram of a speech analysis method performed by the speech analysis system S according to the present embodiment. First, an analyst sets analysis conditions by operating the operation unit 22 of the communication terminal 20. The communication terminal 20 receives the analysis condition setting from the analyst and transmits it as setting information to the speech analysis apparatus 100 (a). The setting unit 111 of the voice analysis device 100 causes the setting information storage unit 131 to store the setting information received from the communication terminal 20 or the setting information specified by the setting unit 111 itself.

  The setting information includes participant position information indicating the position of each participant (that is, each of a plurality of participants) in a group related to one sound collecting device 10, and each group (that is, each of a plurality of groups) that performs discussion at the same time. Group position information indicating the position. The position of each group corresponds to the position of each sound collection device 10 (that is, each of the plurality of sound collection devices 10). For example, the participant position information is information indicating the number of participants in the discussion to be analyzed and the direction in which each participant is located with reference to the sound collection device 10. For example, the group position information is information indicating the number of groups to be analyzed and the relative or absolute position of each group.

  FIG. 4A and FIG. 4B are schematic diagrams of a group position information setting method by the setting unit 111. FIG. 4A shows the display unit 21 of the communication terminal 20 displaying the group setting screen A for setting group position information. The communication terminal 20 displays the group setting screen A on the display unit 21 when setting the group position information, and accepts the setting of the group position information by the analyst. The group setting screen A includes a position setting area A1, a reference position A2, a completion button A3, and an automatic setting button A4.

  The position setting area A1 is an area for setting the position of each group G with respect to a predetermined reference position A2 for a discussion that is performed simultaneously (for example, a discussion performed in the same room). For example, the position setting area A1 is a rectangular area including a reference position A2 (for example, the position of a teacher) as shown in FIG. The reference position A2 may be designated by the analyst on the position setting area A1, or may be registered in advance in the speech analysis apparatus 100.

  The analyst operates the operation unit 22 of the communication terminal 20 to set the position of each group G in the position setting area A1. For example, when the analyst presses somewhere in the position setting area A1, the setting unit 111 of the voice analysis device 100 sets the position pressed by the analyst as the position of one group G. Identification information (here, G1 to G4) for identifying each group G is assigned to the position set for each group G and displayed. The identification information of each group G may be input by an analyst or may be automatically determined by the setting unit 111. In the example of FIG. 4A, four groups G1 to G4 are set.

  The completion button A3 and the automatic setting button A4 are virtual buttons displayed on the display unit 21, respectively. When the completion button A3 is pressed by the analyst, the setting unit 111 stores the position of each group G set on the group setting screen A in the setting information storage unit 131 as group position information. Set the position of. Then, the setting unit 111 displays the participant setting screen C shown in FIG. 5 on the communication terminal 20 for each group G set on the group setting screen A.

  When the automatic setting button A4 is pressed by the analyst, the setting unit 111 automatically sets the position of each group G based on the information acquired from the sound collection device 10. FIG. 4B is a schematic diagram of a method in which the setting unit 111 automatically sets the position of each group G. For example, the setting unit 111 automatically sets the position of each sound collection device 10, that is, the position of each group G, based on sound or radio wave signals exchanged between the sound collection devices 10.

  For automatic setting, each sound collection device 10 has a transmitter that generates a signal of a predetermined sound (sound wave or ultrasonic wave) or a predetermined radio wave (for example, a short-range wireless communication radio wave such as Bluetooth (registered trademark)). The sound collecting apparatus 10 includes a receiver that receives the signal. The setting unit 111 sequentially generates signals from the transmitters of the sound collection devices 10. The setting unit 111 acquires the time and intensity at which the signal is detected in the receiver of each sound collecting device 10.

  Next, the setting unit 111 specifies, for each sound collecting device 10, the sound collecting device 10 that is the signal generation source using the time when the signal is detected, and also uses the intensity that detected the signal to collect the sound collecting device that is the generation source. A distance B up to 10 is calculated. And the setting part 111 determines the position of each sound collector 10, ie, the position of each group G, for example by triangulation based on the calculated distance B between the sound collectors. The setting unit 111 sets the position of each group G by causing the setting information storage unit 131 to store the determined position of each group G as group position information. And the setting part 111 displays the participant setting screen C shown in FIG.

  Here, the method of determining the position of the sound collecting device 10 by exchanging sound or radio wave signals between the sound collecting devices 10 is described as an example, but the setting unit 111 determines the sound collecting device 10 by other methods. May be. For example, a sound or radio wave signal may be generated from a device provided outside the sound collecting device 10.

  The setting unit 111 may perform a combination of automatic setting and manual setting of the position of the group G. In this case, for example, the setting unit 111 displays the position of each group G set by the automatic setting of FIG. 4B in the position setting area A1 of FIG. 4A, and further the manual setting by the analyst. Accept. Thereby, the position of each group G set by automatic setting can be corrected by manual setting, and the position of each group G can be set more reliably.

  FIG. 5 is a front view of the display unit 21 of the communication terminal 20 displaying the participant setting screen C for setting the participant position information. The communication terminal 20 displays the participant setting screen C on the display unit 21 when setting the participant position information, and the participant position information is set by the analyst for each group set on the group setting screen A. Accept. The participant setting screen C includes a position setting area C1, a start button C2, and an end button C3. The position setting area C1 is an area for setting the direction in which each participant U is actually located with reference to the sound collection device 10 in the discussion of the analysis target. For example, the position setting area C1 represents a circle centered on the position of the sound collector 10 as shown in FIG. 5, and further represents an angle along the circle with reference to the sound collector 10.

  The analyst inputs the position of each participant U in the position setting area C1 by operating the operation unit 22 of the communication terminal 20. The setting unit 111 sets the position of each participant U by causing the setting information storage unit 131 to store the position of each participant U input by the analyst as participant position information. In the vicinity of the position set for each participant U, identification information (here, U1 to U4) for identifying each participant U is assigned and displayed. In the example of FIG. 5, four participants U1 to U4 are set. The part corresponding to each participant U in the position setting area C1 is displayed in a different color for each participant. Thereby, the analyst can easily recognize the direction in which each participant U is set.

  The start button C2 and the end button C3 are virtual buttons displayed on the display unit 21, respectively. When the start button C2 is pressed by the analyst, the communication terminal 20 transmits a start instruction signal to the voice analysis device 100. When the end button C3 is pressed by the analyst, the communication terminal 20 transmits an end instruction signal to the voice analysis device 100. In the present embodiment, one discussion is from the start instruction to the end instruction by the analyst.

  When the voice acquisition unit 112 of the voice analysis device 100 receives a start instruction signal from the communication terminal 20, the voice acquisition unit 112 transmits a signal instructing voice acquisition to the sound collector 10 (b). When the sound collection device 10 receives a signal instructing acquisition of sound from the sound analysis device 100, the sound collection device 10 starts acquiring sound. In addition, when the voice acquisition unit 112 of the voice analysis device 100 receives an end instruction signal from the communication terminal 20, the voice acquisition unit 112 transmits a signal instructing the end of voice acquisition to the sound collector 10. When the sound collection device 10 receives a signal instructing the end of voice acquisition from the voice analysis device 100, the sound collection device 10 ends the voice acquisition.

  The sound collection device 10 acquires sound in each of the plurality of sound collection units, and records the sound therein as sound of each channel corresponding to each sound collection unit. Then, the sound collecting device 10 transmits the acquired sounds of the plurality of channels to the sound analyzing device 100 (c). The sound collecting device 10 may sequentially transmit the acquired sound, or may transmit sound of a predetermined amount or a predetermined time. The voice acquisition unit 112 of the voice analysis device 100 receives the voice from the sound collection device 10 and stores it in the voice storage unit 132.

  The voice analyzer 100 analyzes the voice for each group acquired from the sound collector 10 sequentially, that is, by real-time processing. For example, the voice analysis apparatus 100 sequentially analyzes voices for a predetermined time (for example, 30 seconds) in the past from the current time.

  When analyzing the sound, the sound source localization unit 113 first performs sound source localization based on a plurality of channels of sound acquired by the sound acquisition unit 112 (d). The sound source localization is a process of estimating the direction of the sound source included in the sound acquired by the sound acquisition unit 112 every time (for example, every 10 milliseconds to 100 milliseconds). The sound source localization unit 113 associates the sound source direction estimated for each time with the participant direction indicated by the setting information stored in the setting information storage unit 131.

  The sound source localization unit 113 can use a known sound source localization method such as a MUSIC (Multiple Signal Classification) method or a beam forming method as long as the direction of the sound source can be specified based on the sound acquired from the sound collector 10. .

  Next, the analysis unit 114 analyzes the sound based on the sound acquired by the sound acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113 (e). Specifically, first, the analysis unit 114 performs analysis (for example, 10 milliseconds to 100 millimeters) in the analysis target based on the sound acquired by the sound acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113. Every second), it is determined which participant speaks (speaks). The analysis unit 114 specifies a continuous period from when one participant starts speaking until it ends as a speech period, and causes the analysis result storage unit 133 to store it. When a plurality of participants make a statement at the same time, the analysis unit 114 specifies a statement period for each participant.

  In addition, the analysis unit 114 calculates the amount of speech of each participant for each hour and stores it in the analysis result storage unit 133. Specifically, the analysis unit 114 calculates a value obtained by dividing the length of the time when the participant made a speech by the length of the time window in a certain time window (for example, 5 seconds) as the amount of speech for each time. To do. Then, the analysis unit 114 calculates the amount of speech for each participant for each participant while shifting the time window by a predetermined time (for example, 1 second) from the start time of the discussion to the end time (current in the case of real-time processing). repeat. Moreover, the analysis part 114 calculates the total value for every hour of the amount of speech of the participant who belongs to a group as the amount of speech (activity) for every time of a group.

  And the analysis part 114 specifies the situation of the speech of a participant and a group using the amount of speech of a participant and a group. The analysis unit 114, for example, as the participant's speech status, participates in the total value or integral value of the participant's speech amount for a predetermined period (for example, 20 seconds) from the present, or the total value or integral value of the speech amount. The ratio between persons (that is, the relative value) is calculated. The state of the participant's remarks calculated in this way can be used as an index that represents the degree of contribution of each participant to the discussion.

  In addition, the analysis unit 114 may, for example, as a group's speech status, for example, a total value or integral value of a group's speech amount for a predetermined period (for example, 20 seconds) retroactive from the present, The ratio between them (ie relative value) is calculated. The status of group remarks calculated in this way can be used as an index representing the degree of excitement of discussion in each group. The analysis unit 114 causes the analysis result storage unit 133 to store status information indicating the status of the speech of each participant and each group as an analysis result.

  The group atmosphere may be used as the situation of the group remarks. The atmosphere of the group is information indicating the atmosphere of whether there are many or few speakers in the discussion. Specifically, the analysis unit 114 determines the average value of the number of transitions of speech between the same participants U (that is, switching from one speech period to another speech period) and the speech between different participants. The average value of the number of transitions is calculated, and the ratio between them is specified as the group atmosphere. For example, when the ratio of the speech transition between the same participants U is large, the group tends to speak for a long time, and when the ratio of the speech transition between different participants is large, the group is plural. Participants tend to speak in turns.

  The output unit 115 associates the speech status of each participant and each group specified by the analysis unit 114 with the sound collection device 10 corresponding to the participant and the group, and displays the sound collection device 10 and the communication terminal 20. Display on the screen. Specifically, the output unit 115 reads out the status information stored in the analysis result storage unit 133 and transmits the status information to the sound collecting device 10, so that information indicating the status of the utterance specified by the analysis unit 114 is displayed on the entire lamp. 11 and the individual lamp 12 are displayed (f).

  Further, the output unit 115 reads out the status information stored in the analysis result storage unit 133 and transmits the status information to the communication terminal 20, thereby causing the display unit 21 to display information indicating the status of the statement specified by the analysis unit 114. Control is performed (g). A method for outputting the state of speech by the output unit 115 will be described below with reference to FIGS.

[Explanation of how to display the status of remarks]
FIG. 6 is a side view of the sound collecting device 10 displaying the state of the speech of the group and the participant. The output unit 115 uses the general lamp 11 and the individual lamps provided in the sound collection device 10 to display information indicating the status of the group and participants. Specifically, the output unit 115 causes the sound collecting device 10 to display the state of the group remarks by causing the overall lamp 11 to generate predetermined light corresponding to the state of the group remarks. For example, the output unit 115 changes the blinking speed of light, the color of light, or the intensity of light according to the total value or integral value of the amount of speech of the group. Thereby, the participant and assistant of discussion can grasp | ascertain easily the situation of the speech of the whole group.

  Further, the output unit 115 displays the state of each participant's speech on the sound collecting device 10 by generating predetermined light corresponding to the state of the participant's speech from the individual lamp 12 corresponding to the participant. Let That is, the output unit 115 causes the individual lamp 12 provided at a position corresponding to each participant (for example, the front of each participant) to emit light according to the state of speech for each participant. For example, the output unit 115 changes the blinking speed of light, the color of light, or the intensity of light according to the total value or integral value of the participant's speech amount. As a result, the participants of the discussion can objectively recognize the situation of their remarks, and the assistant of the discussion can grasp the contribution degree to the discussion for each participant.

  FIG. 7 is a front view of the display unit 21 of the communication terminal 20 displaying the state of the group remarks. The output unit 115 causes the display unit 21 of the communication terminal 20 to display information indicating the status of the group remarks as the overall status screen D. The overall status screen D includes a circle D1 indicating the status of a statement for each group, and an icon D2 indicating the position of the assistant.

  The output unit 115 displays a circle D <b> 1 having a display mode corresponding to the utterance status of the group at a position on the display unit 21 corresponding to the position of each group indicated by the group position information stored in the setting information storage unit 131. . That is, the output unit 115 displays information indicating the state of speech of the group on a map indicating the position of each group (each sound collecting device 10). For example, the output unit 115 changes the color, shape, pattern, or the like of the circle D1 according to the total value or integral value of the amount of speech of the group. Thereby, the analyst or assistant of the discussion can easily grasp the state of the remarks of each group.

  The output unit 115 displays an icon D2 indicating the position of the assistant at a position on the display unit 21 corresponding to the position of the assistant for discussion. In order to detect the position of the assistant, the output unit 115 uses, for example, a signal exchanged between the sound collecting device 10 and the assistant. In this case, for example, the assistant holds a transmitter that emits a predetermined signal by radio waves or ultrasonic waves of wireless communication such as Bluetooth, and the sound collection device 10 includes a receiver that receives the signal.

  The output unit 115 assists the sound collecting device 10 when the receiver of the sound collecting device 10 can receive a signal from the assistant's transmitter or when the intensity of receiving the signal exceeds a predetermined threshold value. Detect that the person approached. The output unit 115 also receives the sound collecting device 10 when the receiver of the sound collecting device 10 cannot receive the signal from the auxiliary transmitter or when the intensity of receiving the signal becomes less than a predetermined threshold. It is detected that the assistant has left from.

  When the output unit 115 detects that an assistant has approached any of the sound collecting devices 10, the output unit 115 displays an icon D <b> 2 in the vicinity of the sound collecting device 10 (group) on the display unit 21. This allows the analyst of the discussion to analyze changes in the situation of each group's speech between when the assistant is approaching the group and when it is not approaching the group. It can be carried out.

  FIG. 8 is a front view of the display unit 21 of the communication terminal 20 displaying the participant's speech status. When any group is designated by an analyst or an assistant on the overall status screen D, the output unit 115 communicates information indicating the speech status of each participant belonging to the group as an individual status screen E. It is displayed on the display unit 21 of the terminal 20. The individual status screen E includes an area E1 indicating the status of the speech for each participant. The region E1 includes a plurality of regions corresponding to a plurality of participants.

  The output unit 115 displays an area E1 having a display mode corresponding to the state of speech for each participant. For example, the output unit 115 changes the color, pattern, and the like of the region corresponding to each participant in the region E1 according to the total value or integral value of the speech amount of each participant. The output unit 115 displays identification information (here, U1 to U4) for identifying each participant in the vicinity of the area corresponding to each participant. Thereby, the analyst or assistant of the discussion can easily grasp the speech status of each participant belonging to one group.

  The method for outputting information indicating the state of the speech of the group and the participant shown in FIGS. 6 to 8 is an example, and the information can be displayed in association with the sound collecting device 10 corresponding to the group and the participant. Other output methods may be used. The output unit 115 does not necessarily need to display information indicating the state of speech by both the sound collection device 10 and the communication terminal 20, and may display information indicating the state of speech by at least one of them. The output unit 115 may output information indicating the state of the utterance by other means such as printing by a printer or recording data in a storage device.

[Speech analysis method sequence]
FIG. 9 is a sequence diagram of a speech analysis method performed by the speech analysis system S according to the present embodiment. First, the communication terminal 20 receives setting of analysis conditions from an analyst, and transmits it as setting information to the voice analysis device 100 (S11). The setting unit 111 of the voice analysis device 100 causes the setting information storage unit 131 to store the setting information acquired from the communication terminal 20 or the setting information specified by the setting unit 111 itself. The setting information includes participant position information that indicates the position of each participant in the group related to one sound collector 10 and group position information that indicates the position of each group (that is, each sound collector 10) to be discussed at the same time. .

  Next, the voice acquisition unit 112 of the voice analyzer 100 transmits a signal instructing acquisition of voice to the sound collector 10 (S12). When the sound collection device 10 receives a signal instructing the acquisition of sound from the sound analysis device 100, the sound collection device 10 starts recording the sound using a plurality of sound collection units, and the sound analysis device 100 converts the recorded sound of the plurality of channels. (S13). The voice acquisition unit 112 of the voice analysis device 100 receives the voice from the sound collection device 10 and stores it in the voice storage unit 132.

  The voice analysis device 100 analyzes the acquired voice sequentially, that is, by real-time processing. When analyzing sound, the sound source localization unit 113 first performs sound source localization based on the sound acquired by the sound acquisition unit 112 (S14).

  Next, the analysis unit 114 determines which participant has spoken at each time based on the sound acquired by the sound acquisition unit 112 and the direction of the sound source estimated by the sound source localization unit 113. The amount of speech for each hour is calculated. And the analysis part 114 specifies the situation of a participant's utterance using the calculated amount of utterances (S15). The state of the participant's speech is, for example, a total value or an integral value of the participant's speech amount, or a ratio (that is, a relative value) between the participants of the total value or the integral value of the participant's speech amount.

  Further, the analysis unit 114 calculates the amount of speech for each group time by adding the amount of speech for each participant for each group, and specifies the state of the group speech using the calculated amount of speech ( S16). The state of the group speech is, for example, the total value or integral value of the group speech amount, or the ratio (ie, relative value) of the total value or integral value of the group speech amount between the groups. The analysis unit 114 causes the analysis result storage unit 133 to store status information indicating the status of the speech of each participant and each group as an analysis result.

  The output unit 115 displays information indicating the state of speech of each participant and each group by transmitting the state information indicating the state of speech of each participant and each group to the sound collection device 10 and the communication terminal 20. Control is performed (S17).

  The sound collection device 10 displays information indicating the speech status of each participant and each group by causing the overall lamp 11 and the individual lamp 12 to emit light according to the status information received from the voice analysis device 100 (S18). In accordance with the display information received from the voice analysis device 100, the communication terminal 20 causes the display unit 21 to display an overall status screen D and an individual status screen E representing information indicating the status of each participant and each group. ). The voice analysis device 100 analyzes the voice by real-time processing by repeating steps S12 to S19 at predetermined time intervals.

[Effect of this embodiment]
The speech analysis apparatus 100 according to the present embodiment identifies the state of speech in the group related to the sound collection device 10 based on the sound acquired using the sound collection device 10 having a plurality of sound collection units, and the sound collection device 10. Alternatively, it is displayed on the communication terminal 20. Therefore, even if a plurality of groups are discussing at the same time, the analyst or assistant of the discussion can refer to the display on the sound collection device 10 or the communication terminal 20 to determine the state of remarks in the plurality of groups. Can be easily grasped.

  When the voice analysis device 100 displays the speech status on the sound collection device 10, in addition to the analyst or assistant of the discussion, the participant can objectively know the status of the speech of the group to which the speech analysis device 100 belongs. . Further, by displaying the state of speech on the individual lamp 12 provided for each participant on the sound collection device 10, the participant can easily distinguish his speech status from the speech status of other participants. be able to.

  When the speech analysis apparatus 100 displays the speech status on the communication terminal 20, the discussion analyst or assistant can view the speech status of all groups from a bird's-eye view. Further, since it is not necessary to provide a lamp for each sound collecting device 10, the cost is low.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment, A various deformation | transformation and change are possible within the range of the summary. is there. For example, the specific embodiments of device distribution / integration are not limited to the above-described embodiments, and all or a part of them may be configured to be functionally or physically distributed / integrated in arbitrary units. Can do. In addition, new embodiments generated by any combination of a plurality of embodiments are also included in the embodiments of the present invention. The effect of the new embodiment produced by the combination has the effect of the original embodiment.

  In the above description, the voice analysis device 100 is used for voice analysis in the discussion that is performed by the participants surrounding the sound collection device 10, but it can also be applied to other uses. For example, the speech analysis apparatus 100 can be applied to a situation where one commentator explains to a plurality of audiences in a poster session or the like.

  The processors of the speech analysis device 100, the sound collection device 10, and the communication terminal 20 are the main components of each step (process) included in the speech analysis method shown in FIG. That is, the speech analysis apparatus 100, the sound collection apparatus 10, and the processor of the communication terminal 20 read out a program for executing the speech analysis method shown in FIG. The voice analysis method shown in FIG. 9 is executed by controlling each unit of the sound device 10 and the communication terminal 20. Some steps included in the speech analysis method shown in FIG. 9 may be omitted, the order between steps may be changed, and a plurality of steps may be performed in parallel.

S voice analysis system 100 voice analysis device 110 control unit 111 setting unit 112 voice acquisition unit 114 analysis unit 115 output unit 10 sound collector 20 communication terminal 21 display unit

Claims (11)

A storage unit that stores information on a plurality of participants surrounding the sound collection device in association with each of the plurality of sound collection devices;
From each of the plurality of sound collection devices , an acquisition unit that obtains sounds emitted by the plurality of participants associated with the sound collection device in the storage unit ;
An analysis unit that identifies a speech of each of the plurality of participants associated with the sound collection device in the storage unit in the sound acquired by the acquisition unit from each of the plurality of sound collection devices ;
In association with each of the plurality of sound collection devices , information indicating the amount of speech for each sound collection device of the plurality of participants associated with the sound collection device in the storage unit, and the sound collection device in the storage unit An output unit that displays information indicating the amount of each of the plurality of participants associated with the display unit as information indicating a state of the statement;
A voice analysis apparatus having The speech analysis apparatus according to claim 1 , wherein the output unit displays information indicating the state of the utterance on the display unit provided in a communication terminal that communicates with the speech analysis apparatus. The output unit corresponds to the position of each of the plurality of sound collecting devices, the position on the display unit provided in the communication terminal to display the information indicating the status of the talk, according to claim 2 Voice analysis device. When the predetermined person approaches one of the plurality of sound collectors, the output unit is configured to place the person at a position on the display unit provided in the communication terminal corresponding to the position of the sound collector. The voice analysis device according to claim 2 or 3 , wherein information indicating the approach of the is displayed. 5. The voice analysis according to claim 1, further comprising: a setting unit configured to set a position of each of the plurality of sound collecting devices based on a signal exchanged between the plurality of sound collecting devices. apparatus. And the output unit, the multiple sound collector the display unit provided to each, to display the information indicating the status of the talk, voice analysis device according to any one of claims 1 to 5. The voice analysis device according to claim 6 , wherein the output unit displays information indicating a status of the utterance for each of the sound collecting devices on the display unit provided in each of the plurality of sound collecting devices. And the output unit, the at each of the plurality of sound collection devices on the display unit provided towards each of the plurality of participants, and displays the information indicating the status of the speech of each of the participants, according to claim 6 or 8. The voice analysis device according to 7 . Processor
Storing information on a plurality of participants surrounding the sound collection device in a storage unit in association with each of the plurality of sound collection devices;
From each of the plurality of sound collecting devices, a step of acquiring speech said plurality of participants uttered associated with said population sound device in the storage unit,
Identifying the utterances of each of the plurality of participants associated with the sound collection device in the storage unit in the sound acquired from each of the plurality of sound collection devices ;
In association with each of the plurality of sound collection devices , information indicating the amount of speech for each sound collection device of the plurality of participants associated with the sound collection device in the storage unit, and the sound collection device in the storage unit Displaying information indicating the amount of speech of each of the plurality of participants associated with a display unit as information indicating the status of the speech;
Perform voice analysis method. On the computer,
Storing information on a plurality of participants surrounding the sound collection device in a storage unit in association with each of the plurality of sound collection devices;
From each of the plurality of sound collecting devices, a step of acquiring speech said plurality of participants uttered associated with said population sound device in the storage unit,
Identifying the utterances of each of the plurality of participants associated with the sound collection device in the storage unit in the sound acquired from each of the plurality of sound collection devices ;
In association with each of the plurality of sound collection devices , information indicating the amount of speech for each sound collection device of the plurality of participants associated with the sound collection device in the storage unit, and the sound collection device in the storage unit Displaying information indicating the amount of speech of each of the plurality of participants associated with a display unit as information indicating the status of the speech;
Voice analysis program that runs A speech analyzer, a communication terminal capable of communicating with the speech analyzer and a plurality of sound collectors,
At least one of the communication terminal and the plurality of sound collecting devices has a display unit for displaying information,
The speech analyzer is
A storage unit that stores information on a plurality of participants surrounding the sound collection device in association with each of the plurality of sound collection devices;
From each of the plurality of sound collection devices , an acquisition unit that obtains sounds emitted by the plurality of participants associated with the sound collection device in the storage unit ;
An analysis unit that identifies a speech of each of the plurality of participants associated with the sound collection device in the storage unit in the sound acquired by the acquisition unit from each of the plurality of sound collection devices ;
In association with each of the plurality of sound collection devices , information indicating the amount of speech for each sound collection device of the plurality of participants associated with the sound collection device in the storage unit, and the sound collection device in the storage unit an output unit for displaying the information indicating the plurality of participants each utterance amount associated, on the display unit as the information indicating the status of the remarks,
A speech analysis system.

Images