JP2017161787A - Speech response detection method, speech response detection program, and speech response detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a speech response detection method capable of detecting a response part in a speech.SOLUTION: The speech response detection method is executed by a computer, including: a conversion step S302 to convert a piece of voice information including a speech into a piece of first character information; an extraction step S304 to extract a part of the character information from a piece of second character information with respect to the speech based on the frequency of occurrence of the part of the character information; and a detection step S306 to detect a part which includes the part of the character information in the first character information as a response part in the speech.SELECTED DRAWING: Figure 8

Description

  The present invention relates to a speech echo detection method, a speech echo detection program, and a speech echo detection device.

  A technique is known in which candidates for election and voters can exchange opinions via the Internet (see, for example, Patent Document 1). In addition, there is also known a technique for taking in the voice of an audience attending a lecture as voice information from a sound collecting microphone and evaluating the contents of the lecture by comparing with voice patterns set in advance (for example, Patent Document 2). reference). Furthermore, a technique for extracting a keyword from previously recorded audio data is also known (see, for example, Patent Document 3).

JP 2002-041718 A JP 2001-266148 A JP 2006-178648 A

  Here, for example, when evaluating the contents of a speech in an election using the various technologies described above, if the audience's voice is taken as voice information and a keyword is extracted from the voice information, the audience's response to the entire speech is detected via the keyword. I may be able to do it.

  However, it is very difficult to accurately detect the audience's partial response to the speech. For example, if a speech contains multiple policy issues, it is extremely difficult to accurately detect which policy issue the audience is interested in. .

  It is an object of the present invention to provide a speech echo detection method, a speech echo detection program, and a speech echo detection device that can detect an echo portion in a speech.

  The speech echo detection method disclosed in the present specification includes a conversion step of converting speech information including a speech into first character information, and a part of character information from the second character information corresponding to the speech. The computer executes an extraction step of extracting based on the appearance frequency of the character information and a detection step of detecting a portion including the partial character information in the first character information as an echo portion in the speech This is a speech echo detection method.

  The speech echo detection method disclosed in the present specification calculates the degree of support for each speech based on the voice and / or sound volume directed to each speech included in each of a plurality of speech information. A first calculation step, a characterizing step for classifying character information for each speech into character information opposite to the character information for approval, using character information indicating the opposite to character information indicating approval, and the character for approval A second calculating step for calculating the degree of reverberation for each speech based on the information or the opposite character information; and a speaker for each speech based on the degree of support and the degree of reverberation Is a speech echo detection method executed by a computer.

  The speech echo detection program disclosed in this specification includes a conversion step of converting speech information including a speech into first character information, and part of character information from the second character information corresponding to the speech. An extraction step of extracting based on the appearance frequency of the character information and a detection step of detecting a portion including the partial character information in the first character information as a reverberation portion in the speech This is a speech echo detection program.

  The speech echo detection program disclosed in the present specification calculates the degree of support for each speech based on the voice and / or the sound volume directed to each speech included in each of a plurality of speech information. A first calculation step, a characterizing step for classifying character information for each speech into character information opposite to the character information for approval, using character information indicating the opposite to character information indicating approval, and the character for approval A second calculating step for calculating the degree of reverberation for each speech based on the information or the opposite character information; and a speaker for each speech based on the degree of support and the degree of reverberation Is a speech echo detection program for causing a computer to execute an estimation step for estimating the support status of each speech.

  The speech echo detection apparatus disclosed in this specification includes a conversion unit that converts speech information including a speech into first character information, and a part of character information from the second character information corresponding to the speech. The speech echo having extraction means for extracting based on the appearance frequency of the character information, and detection means for detecting a part including the partial character information in the first character information as an echo part in the speech It is a detection device.

  The speech echo detection device disclosed in the present specification calculates the degree of support for each speech based on the voice and / or sound volume directed to each speech included in each of a plurality of speech information. Classifying means for classifying the character information for each speech into character information opposite to the character information for approval, using character information indicating the opposite to the character information indicating approval for the first calculation means, and the character for approval A second calculating means for calculating the degree of reverberation for each speech based on the information or the opposite character information, and a speaker performing each speech based on the degree of support and the degree of reverberation, respectively. And an estimation means for estimating the support status for each speaker.

  As one aspect, a reverberation part in a speech can be detected.

FIG. 1 is an example of a speech echo detection system according to the first embodiment. FIG. 2 is an example of a hardware configuration of the server apparatus. FIG. 3 is an example of a functional block diagram of the server apparatus according to the first embodiment. FIG. 4A is an example of a speech voice storage unit. FIG. 4B is an example of an audio file. FIG. 4C shows another example of an audio file. FIG. 5A is an example of an SNS information storage unit. FIG. 5B is an example of the approval / disapproval information storage unit. FIG. 6 is a flowchart illustrating an example of processing executed by the server device according to the first embodiment. FIG. 7 is a flowchart illustrating another example of the process executed by the server device according to the first embodiment. FIG. 8 is a flowchart showing another example of processing executed by the server device according to the first embodiment. FIG. 9 is a diagram for explaining an example of the processing result. FIG. 10 is an example of a pros and cons evaluation screen displayed on the terminal device. FIG. 11 is an example of various screens displayed on the terminal device. FIG. 12 is an example of a speech echo detection system according to the second embodiment. FIG. 13 is an example of a functional block diagram of the server apparatus according to the second embodiment. FIG. 14 shows another example of the speech voice storage unit. FIG. 15 shows another example of the SNS information storage unit. FIG. 16 is a flowchart illustrating an example of processing executed by the server device according to the second embodiment.

  Hereinafter, an embodiment for carrying out this case will be described with reference to the drawings.

(First embodiment)
FIG. 1 is an example of a speech echo detection system S1 according to the first embodiment. The speech echo detection system S1 includes at least a terminal device 100 and a server device 200 as a speech echo detection device. In FIG. 1, a stationary (desktop type) personal computer (PC) including an input device, a display device, and a control device is shown as an example of the terminal device 100, but the terminal device 100 is not particularly limited. For example, the terminal device 100 may be a portable (notebook type) PC, a smartphone, a tablet terminal, or the like. The terminal device 100 is installed in the election countermeasure office 10, for example. The election office 10 is used by the speaker RA who has run for the election. On the other hand, the server device 200 is deployed in the data center DC on the cloud CL.

  The terminal device 100 and the server device 200 are connected via a communication network NW. An example of the communication network NW is the Internet. Therefore, the terminal device 100 can communicate with the server device 200. That is, the terminal device 100 receives various information transmitted from the server device 200 or transmits various information to the server device 200.

  For example, when the terminal device 100 transmits a browsing request for browsing the information indicating the response to the speech to the server device 200, the server device 200 receives the browsing request. More specifically, the browsing request transmitted by the terminal device 100 is received by the server device 200 via the communication network NW. Upon receiving the browsing request, the server device 200 permits the terminal device 100 to browse information. Specifically, when the server device 200 receives a browsing request, the server device 200 transmits information corresponding to the browsing request to the terminal device 100.

  A microphone (specifically, a sound collecting microphone) 300 is connected to the communication network NW. In FIG. 1, the microphone 300 is directly connected to the communication network NW by wire. However, a portable PC (not shown) is interposed between the microphone 300 and the communication network NW, the microphone 300 and the portable PC are connected by wire or wirelessly, and the portable PC and the communication network NW are wirelessly connected. You may connect. In this case, the microphone 300 and the communication network NW are indirectly connected.

  The microphone 300 acquires the voice including the speech of the speaker RA and converts the acquired voice into voice information. In FIG. 1, the speaker RA is giving a speech before the audience AD. If the transmission destination of the audio information is set to the microphone 300 in the server device 200, the microphone 300 transmits the audio information to the server device 200. Note that when the microphone 300 and the portable PC are connected by wire or wirelessly, the setting of the audio information transmission destination is performed on the portable PC. Thereby, the server apparatus 200 collects audio information including the speech of the speaker RA.

  A social networking service (hereinafter referred to as SNS) 400 is connected to the communication network NW. A server device (not shown) that realizes the SNS 400 may be connected to the communication network NW. The SNS 400 may be included in the communication network NW. Examples of the SNS 400 include TWITTER (registered trademark) (Twitter (registered trademark)), FACEBOOK (registered trademark) (Facebook (registered trademark)), and GOOGLE + (registered trademark) (Google Plus (registered trademark)).

  Character information (for example, tweets, comments, messages, etc.) from an unspecified number of users who use the service is posted on the SNS 400. Users include the audience AD described above. Therefore, if the audience AD posts character information on the SNS 400 before the speaker RA is speaking, the character information is spread on the communication network NW. The server device 200 collects character information spread on the communication network NW as SNS information. Note that the server device 200 collects SNS information using an Application Programming Interface (API) such as Twitter API or FB API.

  The server device 200 analyzes or analyzes the collected voice information and SNS information. For example, the server apparatus 200 analyzes the SNS information and evaluates the reverberation part in the speech. When the server apparatus 200 receives the analysis result or the browsing request for browsing the analysis result from the terminal apparatus 100, the server apparatus 200 permits the browsing. Thereby, the terminal device 100 displays an analysis result or an analysis result. The detailed operation of the server device 200 will be described later.

  Next, the hardware configuration of the server apparatus 200 will be described with reference to FIG. Note that the terminal device 100 described above has basically the same hardware configuration as that of the server device 200, and a description thereof will be omitted.

  FIG. 2 is an example of a hardware configuration of the server device 200. As shown in FIG. 2, the server device 200 includes at least a central processing unit (CPU) 200A, a random access memory (RAM) 200B, a read only memory (ROM) 200C, and a network I / F (interface) 200D. The server device 200 may include at least one of a hard disk drive (HDD) 200E, an input I / F 200F, an output I / F 200G, an input / output I / F 200H, and a drive device 200I as necessary. The CPU 200A,..., The drive device 200I are connected to each other by an internal bus 200J. At least the CPU 200A and the RAM 200B cooperate to realize a computer.

An input device 710 is connected to the input I / F 200F. Examples of the input device 710 include a keyboard and a mouse.
A display device 720 is connected to the output I / F 200G. An example of the display device 720 is a liquid crystal display.
A semiconductor memory 730 is connected to the input / output I / F 200H. Examples of the semiconductor memory 730 include a universal serial bus (USB) memory and a flash memory. The input / output I / F 200 </ b> H reads programs and data stored in the semiconductor memory 730.
The input I / F 200F and the input / output I / F 200H include, for example, a USB port. The output I / F 200G includes a display port, for example.

A portable recording medium 740 is inserted into the drive device 200I. Examples of the portable recording medium 740 include a removable disk such as a Compact Disc (CD) -ROM and a Digital Versatile Disc (DVD). The drive device 200I reads a program and data recorded on the portable recording medium 740.
The network I / F 200D includes, for example, a LAN port. Network I / F 200D is connected to communication network NW.

  In the above-described RAM 200B, a program stored in the ROM 200C or the HDD 200E is stored by the CPU 200A. In the RAM 200B, the program recorded on the portable recording medium 740 is stored by the CPU 200A. When the stored program is executed by the CPU 200A, various functions to be described later are realized, and various processes to be described later are executed. In addition, what is necessary is just to make a program according to the flowchart mentioned later.

  Next, functions of the server device 200 according to the first embodiment will be described with reference to FIGS. 3 to 5.

  FIG. 3 is an example of a functional block diagram of the server apparatus 200 according to the first embodiment. FIG. 4A is an example of the speech voice storage unit 202. FIG. 4B is an example of an audio file. FIG. 4C shows another example of an audio file. FIG. 5A is an example of the SNS information storage unit 204. FIG. 5B is an example of the approval / disapproval information storage unit 205.

  As shown in FIG. 3, the server device 200 includes a voice collection unit 201 and a speech voice storage unit 202 as first collection means. The server device 200 includes an SNS information collection unit 203 and an SNS information storage unit 204 as second collection means. Further, the server device 200 includes a pros and cons information storage unit 205, a character string conversion unit 206 as a conversion unit, a keyword extraction unit 207 as an extraction unit, an SNS information classification unit 208 as a classification unit, an echo detection unit 209 as a detection unit, And the evaluation result presentation part 210 is included.

  The voice collection unit 201 collects voice information. More specifically, the voice collection unit 201 collects voice information transmitted directly or indirectly from the microphone 300. The voice collection unit 201 collects voice information from the first trigger sound to the second trigger sound. Examples of the first trigger sound include a voice that conveys the name of a political party, a voice that conveys the name of a speaker, and a voice that signals the start of a speech (for example, “Introduction”). In addition, examples of the second trigger sound include a sound that signals the end of the speech (for example, “listening ...”), and a sound of applause that records a predetermined volume or more. As a result, the voice collection unit 201 can avoid wasteful voice collection that does not include a speech. The voice collection unit 201 stores the collected voice information in the speech voice storage unit 202.

  The speech voice storage unit 202 stores voice information including a speech. More specifically, as shown in FIG. 4A, the speech voice storage unit 202 manages voice information using a voice table T1. The voice information includes a speech ID, a speaker, a speech date, a speech time, a place, a voice file, a speech volume, a cheering volume, and a clapping volume as constituent elements.

  The speech ID represents identification information for identifying voice information. The speaker represents the name of the speaker. The speech date represents the date on which the speech was made. The speech time represents from the speech start time to the speech end time. The place represents the place where the speech was given. An audio file represents an electronic file name of an audio including a speech. For example, the voice file “EZ1.mp3” includes the first speech voice 21 as shown in FIG. For example, the audio file “EZ2.mp3” includes the second speech audio 22 as shown in FIG. Both the first speech voice 21 and the second speech voice 22 represent a part of the whole speech.

  The speech volume, the cheering volume, and the applause volume represent the speech, cheering, and applause volume, respectively. These volumes may be any of the highest volume, the lowest volume, or the average volume. Further, speech, cheering, and applause are classified by the voice collection unit 201 based on, for example, a difference in voice frequency (for example, a voiceprint difference between a speaker and a person other than the speaker). In addition, cheering is an example, and if it is a voice, it will not be limited to cheering, For example, a shout, a cheer, a father, etc. may be sufficient. Applause is also an example, and it is not limited to applause as long as it is a sound, and may be a whistle or a finger whistle.

  Returning to FIG. 3, the SNS information collection unit 203 collects SNS information. More specifically, the SNS information collection unit 203 collects SNS information from the SNS 400 using an API. In particular, it is desirable that the SNS information collection unit 203 collects SNS information posted to the SNS 400 within a predetermined time (for example, within several tens of minutes) after the voice information is collected by the voice collection unit 201. This predetermined time may be the time from the first trigger sound to the second trigger sound described above. As a result, real-time responses to the speech can be collected. The SNS information collection unit 203 stores the collected SNS information in the SNS information storage unit 204.

  The SNS information storage unit 204 stores SNS information. More specifically, as shown in FIG. 5A, the SNS information storage unit 204 manages the SNS information using the SNS table T2. The SNS information includes an SNS ID, an SNS type, a posting date, a posting time, and a posting comment as constituent elements. SNS ID represents identification information for identifying SNS information. The SNS type represents the type of SNS. The posting date represents the date when the posting was made. The posting time represents the time when the posting was made. The posted comment represents the posted opinion. As shown in FIG. 5 (a), if the audience AD posts character information on the SNS 400 before the speaker RA is speaking, the SNS information storage unit 204 basically displays character information according to the contents of the speech. Remember.

  Returning to FIG. 3, the approval / disapproval information storage unit 205 stores approval / disapproval information regarding approval / disapproval for a specific character string. More specifically, as shown in FIG. 5B, the approval / disapproval information storage unit 205 manages the approval / disapproval information by using the approval / disapproval table T3. The approval / disapproval information includes approval / disapproval ID, comment, and approval / disapproval type as components. The approval / disapproval ID represents identification information for identifying approval / disapproval information. The comment represents a specific character string used when judging the approval or disapproval of the posted comment. The comment may include a so-called emoticon. The type of approval or disapproval expresses the intention of the comment by agreeing or disagreeing. For example, the comments “good” and “best” indicate approval. On the other hand, comments “worry” and “(;;)” indicate the opposite. Based on the approval / disapproval information, the voter's approval / disapproval can be determined.

  Returning to FIG. 3, the character string conversion unit 206 converts voice information into speech character information. More specifically, when the character string conversion unit 206 detects the second trigger sound, the character string conversion unit 206 acquires voice information from the speech voice storage unit 202 and converts the acquired voice information into speech character information. That is, the character string conversion unit 206 starts conversion of speech information into speech character information with the end of the speech as a trigger. Thereby, useless waiting time until the conversion process is started can be reduced. Conversion from voice information to speech character information is performed using, for example, voice recognition software.

  In addition, the character string conversion unit 206 extracts an important keyword (hereinafter referred to as an important keyword) from the speech character information. For example, the character string conversion unit 206 recognizes a part of voice information having a volume equal to or higher than a predetermined threshold in the whole voice information as an important keyword, and extracts the important keyword from the speech character information. This is because important policy issues are likely to be spoken loudly in the speech. When the conversion of voice information into speech character information and extraction of important keywords are completed, the character string conversion unit 206 outputs speech character information to the echo detection unit 209.

  The keyword extraction unit 207 acquires SNS information from the SNS information storage unit 204, and extracts part of the character information from the acquired SNS information as a frequent keyword based on the appearance frequency of the part of character information. For example, if character information such as “groceries”, “consumption tax”, and “reduced tax rate” appears more frequently than a predetermined value in common with each posted comment in a plurality of SNS information, the keyword extracting unit 207 These character information is extracted as frequent keywords.

  The keyword extraction unit 207 outputs the extracted frequent keywords to the SNS information classification unit 208 and the echo detection unit 209. When the keyword extraction unit 207 also detects the second trigger sound, the SNS information is acquired from the SNS information storage unit 204, and frequent keywords are extracted from the acquired SNS information. The waiting time can be reduced.

  The SNS information classification unit 208 acquires SNS information from the SNS information storage unit 204 and acquires approval / disapproval information from the approval / disapproval information storage unit 205. More specifically, when the frequent keyword is output from the keyword extraction unit 207, the SNS information classification unit 208 acquires SNS information and approval / disapproval information. The SNS information classification unit 208 uses the approval / disapproval information and the frequent keywords to classify some SNS information including the frequent keywords in the acquired SNS information into the SNS information representing the opposite and the SNS information representing the opposite. . On the other hand, the SNS information classifying unit 208 excludes a part of SNS information that does not include a frequent keyword from the acquired SNS information. When the SNS information classification unit 208 completes the classification of the SNS information, the SNS information classification unit 208 outputs the SNS information representing the opposite to the SNS information representing the approval to the echo detection unit 209.

  The reverberation detection unit 209 detects a portion including frequently-occurring keywords in the speech character information including important keywords as a reverberation portion in the speech. More specifically, the echo detection unit 209 detects a sentence including a frequent keyword among sentences including an important keyword as an echo part in the speech. Therefore, even if a sentence includes an important keyword, if the frequent keyword is not included, it is determined that the sentence is less interested for the audience AD. In other words, a sentence including both an important keyword and a frequent keyword is determined to be highly interested for the audience AD, and is detected as an echo part.

  Further, the echo detection unit 209 evaluates the approval / disapproval of the echo part based on the SNS information indicating approval and the SNS information indicating opposition. For example, the echo detection unit 209 calculates a rate opposite to the rate of approval of the echo part based on the number and rate of SNS information indicating the opposite and SNS information indicating the opposite. The echo detection unit 209 outputs the detected echo part and the evaluated pros and cons to the evaluation result presentation unit 210. In addition, the echo detection unit 209 outputs various analysis results.

  The evaluation result presentation unit 210 holds the detected echo part, evaluated pros and cons, and other analysis results. When receiving the browsing request from the terminal device 100, the evaluation result presenting unit 210 permits or rejects browsing based on the authentication information included in the browsing request. For example, if the authentication result is successful, the evaluation result presentation unit 210 permits browsing of the detected echo portion and the evaluated pros and cons. Conversely, if the authentication result fails, the evaluation result presentation unit 210 refuses to browse the detected echo part or the evaluated pros and cons. Thereby, the staff of the election office 10 can grasp the response part and the pros and cons in the speech of the speaker RA.

  Next, the operation of the server device 200 will be described with reference to FIGS.

  FIG. 6 is a flowchart illustrating an example of processing executed by the server device 200 according to the first embodiment. 7 and 8 are flowcharts illustrating other examples of processing executed by the server device 200 according to the first embodiment. FIG. 9 is a diagram for explaining an example of the processing result. FIG. 10 shows an example of the approval / disapproval evaluation screen displayed on the terminal device 100. FIG. 11 is an example of various screens displayed on the terminal device 100.

  First, as shown in FIG. 6, the voice collection unit 201 stands by until a first trigger sound is detected (step S101: NO). When the voice collection unit 201 detects the first trigger sound (step S101: YES), the voice collection unit 201 collects voice information (step S102) and stores the voice information in the speech voice storage unit 202 (step S103). The voice collection unit 201 collects and stores voice information until the second trigger sound is detected (step S104: NO). Then, when the voice collection unit 201 detects the second trigger sound (step S104: YES), the process ends. Thus, the speech voice storage unit 202 stores voice information including the speech.

  Next, as shown in FIG. 7, the SNS information collection unit 203 waits until the first trigger sound is detected (step S201: NO). When the SNS information collection unit 203 detects the first trigger sound (step S201: YES), the SNS information collection unit 203 collects SNS information (step S202) and stores the SNS information in the SNS information storage unit 204 (step S203). The SNS information collection unit 203 collects and stores the SNS information until the second trigger sound is detected (step S204: NO). And if the SNS information collection part 203 detects a 2nd trigger sound (step S204: YES), a process will be complete | finished. Thereby, the SNS information storage unit 204 stores the SNS information.

  Next, as shown in FIG. 8, the character string conversion unit 206 waits until it detects the second trigger sound (step S301: NO). When the character string conversion unit 206 detects the second trigger sound (step S301: YES), the character string conversion unit 206 converts voice information into speech character information (step S302). More specifically, the character string conversion unit 206 acquires voice information from the speech voice storage unit 202, and converts the acquired voice information into speech character information. For example, when the character string conversion unit 206 acquires the sound file shown in FIG. 4B as sound information, the character string conversion unit 206 converts the first speech sound 21 into speech character information based on a character string as shown in FIG. 9A. .

  When the process of step S302 is completed, the character string converter 206 then extracts important keywords based on the volume (step S303). More specifically, the character string conversion unit 206 recognizes voice information having a volume equal to or higher than a predetermined threshold in the entire voice information as an important keyword, and extracts the important keyword from the speech character information. Therefore, in the case where “single vote disparity”, “security”, and “consumption tax” in the entire speech information have a volume greater than or equal to a predetermined threshold, as shown in FIG. From the above, the text information “one vote disparity”, “security” and “consumption tax” are extracted as important keywords.

  When the process of step S303 is completed, the keyword extraction unit 207 then extracts frequently-used keywords from the SNS information (step S304). More specifically, the keyword extraction unit 207 acquires SNS information from the SNS information storage unit 204, and extracts frequent keywords from the acquired SNS information. For example, when the keyword extracting unit 207 acquires the SNS information shown in FIG. 5A, as shown in FIG. 9C, the character information “consumption tax” and “reduced tax rate” are frequently used as keywords from the SNS information. Extracted. The keyword extraction unit 207 also extracts character information “food”, but is omitted in FIG.

  When the process of step S304 is completed, the SNS information classification unit 208 then classifies the pros and cons of the part containing the frequently occurring keyword in the SNS information (step S305). More specifically, the SNS information classification unit 208 acquires SNS information from the SNS information storage unit 204, and classifies the pros and cons of some SNS information including frequent keywords from the entire acquired SNS information based on pros and cons information. For example, when the SNS information classifying unit 208 acquires the four SNS information shown in FIG. 5A, three SNS information including the frequent keywords “food”, “consumption tax”, and “reduced tax rate” from the four SNS information. Categorize pros and cons.

  Here, as shown in FIG. 5A, SNS IDs “S01” and “S03” are SNS information indicating approval, and SNS ID “S02” is SNS information indicating the opposite. For this reason, as shown in FIG.9 (c), the SNS information classification | category part 208 classify | categorizes 3 SNS information into 1 SNS information showing the opposite to 2 SNS information showing approval. After the classification, the SNS information classification unit 208 calculates the percentage of approval / disapproval of the SNS information. In the case of this embodiment, as shown in FIG. 9C, it is calculated that the approval = 67% and the opposite = 33%. Note that the order of the processes in steps S302 and S303 and the processes in steps S304 and S305 may be reversed, or the processes may proceed in parallel.

  When the process of step S305 is completed, the echo detection unit 209 then detects the echo part in the speech (step S306). More specifically, the echo detection unit 209 detects a part of character information including frequent keywords in the speech character information including important keywords as an echo part in the speech. For example, the echo detection unit 209 detects a sentence including a frequent keyword among sentences including an important keyword as an echo part in the speech. In addition, the echo detection unit 209 evaluates the percentage of approval and disapproval calculated by the SNS information classification unit 208 and writes it in the echo part. Therefore, as shown in FIG. 9 (d), the echo detection unit 209 detects one sentence including the important keyword / frequently used keyword “consumption tax” as an echo part in the speech, and also determines whether the echo part is in favor or not. = 67%, opposite = 33% ", and is written at the end of one detected sentence. The echo detection unit 209 determines the end of the sentence based on, for example, the presence or absence of a punctuation mark.

  When the process of step S306 is completed, the evaluation result presentation unit 210 then presents the evaluation result (step S307). For example, when receiving the browsing request from the terminal device 100, the evaluation result presentation unit 210 determines whether to permit or reject browsing based on the authentication information included in the browsing request. When the evaluation result presentation unit 210 permits browsing, the evaluation result presentation unit 210 causes the terminal device 100 to browse the approval / disapproval evaluation screen. On the approval / disapproval browsing screen, as shown in FIG. 10, first, the extracted important keywords are presented as the points of the entire speech, and then the calculated proportion of approval / disapproval and the detected reverberation part are presented. Thereby, the staff of the election office 10 can grasp which part of the speech of the speaker RA is reverberating to the voters.

  In addition, the evaluation result presentation unit 210 receives an instruction to switch the approval / disapproval browsing screen from the terminal device 100, and as shown in FIGS. 11 (a) to 11 (c), the review transition screen, the rapidly increasing keyword screen, and the opposite speaker identification Any of the screens may be displayed. The word-of-mouth transition screen shows the transition of the number of SNS information posted per unit time. Thereby, the time-sequential correlation between the speech and the SNS information can be grasped. The soaring keyword screen represents keywords that appear rapidly and frequently within a predetermined time. If the cause is clarified by comparing the keyword with the speech, it can be used for the next speech. The counter-speaker specifying screen is a screen for specifying a sender who transmits a counter-speak. For example, when a misunderstanding is included in the opposite statement by the caller, the specified caller can be protested via the SNS 400.

  As described above, according to the first embodiment, the server device 200 includes the character string conversion unit 206, the keyword extraction unit 207, and the echo detection unit 209. Here, the character string conversion unit 206 converts voice information including a speech into speech character information. The keyword extraction unit 207 extracts a part of predetermined character information as frequent keywords from the SNS information for the speech. The reverberation detection unit 209 detects a part of the text that is a part including the frequent keyword in the speech character information as a reverberation part in the speech. Thereby, the reverberation part in a speech can be detected.

(Second Embodiment)
Next, a second embodiment of the present case will be described with reference to FIGS.
FIG. 12 is an example of a speech echo detection system S2 according to the second embodiment. In addition, the same code | symbol is attached | subjected to the structure similar to each apparatus of the speech echo detection system S1 shown by FIG. 1, and the description is abbreviate | omitted. FIG. 12 shows a state where a speaker RB different from the speaker RA is giving a speech. Further, FIG. 12 shows an election countermeasure office 30 different from the election countermeasure office 10.

  The election office 30 is an office used by a speaker RB who has run for the election. Similarly to the election countermeasure office 10, a terminal device (not shown) is installed in the election countermeasure office 30. A terminal device installed in the election countermeasure office 30 is connected to the server device 200 via the communication network NW. Therefore, the terminal device can communicate with the server device 200.

  A microphone 310 is connected to the communication network NW related to the speech echo detection system S2. The microphone 310 acquires a voice including the speech of the speaker RB and converts the acquired voice into voice information. In FIG. 12, the speaker RB is giving a speech before the audience AE. If the transmission destination of the audio information is set to the microphone 310 in the server device 200, the microphone 310 transmits the audio information to the server device 200. A portable PC may be used as in the first embodiment. Thereby, the server apparatus 200 collects audio information including the speech of the speaker RB.

  Next, functions of the server device 200 according to the second embodiment will be described with reference to FIG.

  FIG. 13 is an example of a functional block diagram of the server apparatus 200 according to the second embodiment. In addition, the same code | symbol is attached | subjected to the structure similar to each part of the server apparatus 200 shown by FIG. 3, and the description is abbreviate | omitted.

  The server device 200 includes a voice collection unit 201 and a speech voice storage unit 202 as first collection means. The server device 200 includes an SNS information collection unit 203 and an SNS information storage unit 204 as second collection means. Further, the server device 200 includes a pros and cons information storage unit 205, an SNS information classification unit 208 as a classification unit, a support level calculation unit 211 as a first calculation unit, an echo level calculation unit 212 as a second calculation unit, and an estimation unit. The support status estimation unit 213 and the support status presentation unit 214 are included.

  The support level calculation unit 211 calculates the degree of support for each speech based on the voice and / or sound volume directed to each speech included in each of the plurality of audio information. For example, as shown in FIG. 14, when the speech information of each of a plurality of speakers is stored in the speech voice storage unit 202, when the support level calculation unit 211 detects the second trigger sound, the speech voice storage unit The voice information of each of a plurality of speakers is acquired from 202. The support level calculation unit 211 calculates the degree of support for each speech based on the voice and / or sound volume directed to each speech included in each of the acquired plurality of audio information.

  Specifically, the support level calculation unit 211 calculates the volume sum of voice and / or sound, and determines that the degree of support is high if the calculated volume sum is relatively large. The voice includes cheers, cheers, shouts, and the like. The sounds include clapping sounds, whistling sounds, finger whistling sounds, and the like. Therefore, the support level calculation unit 211 determines that the support level is high if the volume of encouragement and / or applause is relatively high, for example. The support level calculation unit 211 outputs the calculated support level for each speaker to the support status estimation unit 213.

  The reverberation degree calculation unit 212 calculates the degree of reverberation for each speech based on the SNS information for approval or the opposite SNS information classified by the SNS information classification unit 208. For example, as shown in FIG. 15, when SNS information for a plurality of speakers is stored in the SNS information storage unit 204, an SNS information opposite to the SNS information approved by the SNS information classifying unit 208 is stored in the echo calculation unit 212. Information is output.

  The reverberation calculating unit 212 further classifies the SNS information opposite to the SNS information in favor of each speaker. Specifically, the reverberation degree calculation unit 212 performs classification based on the name of the speaker who appears in the posted comment of the SNS information. The reverberation degree calculation unit 212 uses the opposite SNS information classified for each speaker, and calculates the ratio of the opposite SNS information for each speaker. For example, when there are 40 opposite SNS information for the speaker RA and 60 opposite SNS information for the speaker RB, the echo level calculation unit 212 sets the unacceptable response level for the speaker RA to 40% and the speaker RB. The unfavorable echo with respect to is calculated as 60%. The echo calculation unit 212 outputs the calculated unsupported echo for each speaker to the support situation estimation unit 213.

Based on the support level output from the support level calculation unit 211 and the echo level output from the echo level calculation unit 212, the support status estimation unit 213 determines the support status of each speaker who performs each speech for each speaker. presume. For example, the support situation estimation unit 213 uses the following calculation formula to estimate the support situation for each speaker. A and B represent the speakers RA and RB, respectively.
Support status A: B =
A's speech / support / (A's speech / support + A's response / non-support)
: B's speech / support / (B's speech / support + B's response / non-support)

Here, each speech / support of A and B, and each echo / non-support of A and B are calculated using the following calculation formulas. Note that dB represents decibels.
A's speech / support = A's speech / support (dB)
/ (A's speech / support (dB) + B's speech / support (dB))
A's response / non-support = A's response / non-support (%)
/ (A response / non-support (%) + B response / non-support (%))
B's speech / support = B's speech / support (dB)
/ (A's speech / support (dB) + B's speech / support (dB))
B's response / non-support = B's response / non-support (%)
/ (A response / non-support (%) + B response / non-support (%))

  In the present embodiment, the speech / support of the speaker A is calculated as 150 (dB) and the speech / support of the speaker B is calculated as 120 (dB) based on the sum of the volumes of the speakers A and B shown in FIG. Therefore, as described above, when the degree of unfavorable echo for the speaker RA is calculated as 40 (%) and the degree of unfavorable echo for the speaker RB is calculated as 60 (%), the support situation estimation unit 213 calculates the decimal point. Round down and estimate A: B = 57: 42. That is, the support status estimation unit 213 estimates that the support status of the speaker RA is 57% and the support status of the speaker RA is 42%. The support situation estimation unit 213 outputs the estimated support situation to the support situation presentation unit 214.

  The support status presentation unit 214 holds the estimated support status. When receiving a browsing request from the terminal device 100 of the election countermeasure office 10 or the terminal device (not shown) of the election countermeasure office 30, the support status presentation unit 214 permits browsing based on the authentication information included in the browsing request. Or refuse. For example, if the authentication is successful, the support status presentation unit 214 permits browsing of the estimated support status. Conversely, the support status presentation unit 214 rejects the estimated support status if the authentication fails. Thereby, each member of the election countermeasure office 10 and the election countermeasure office 30 can grasp the support status of the speeches of the speakers RA and RB.

  FIG. 16 is a flowchart illustrating an example of processing executed by the server device 200 according to the second embodiment.

  As shown in FIG. 16, the support degree calculation unit 211 waits until it detects the second trigger sound (step S401: NO). When the support level calculation unit 211 detects the second trigger sound (step S401: YES), the support level calculation unit 211 calculates the above-described support level (step S402). More specifically, the support level calculation unit 211 acquires audio information from the speech audio storage unit 202 and calculates the above-described support level.

  When the process of step S402 is completed, the SNS information classification unit 208 then classifies the pros and cons of the part containing the frequently occurring keyword in the SNS information (step S403). When the process of step S403 is completed, the reverberation degree calculation unit 212 calculates the reverberation degree described above (step S404). When the process of step S404 is completed, the support status estimation unit 213 then estimates the support status (step S405).

  When the process of step S405 is completed, the support status presentation unit 214 presents the support status (step S406). For example, the support status presentation unit 214 receives a browsing request from the terminal device 100 or the like. The support status presentation unit 214 determines whether or not to permit viewing of the support status based on the authentication information included in the browse request. If the authentication is successful, the support status presentation unit 214 permits browsing. As a result, the support status presentation unit 214 presents the support status to the terminal device 100. Therefore, each member of the election offices 10 and 30 can grasp the support status of the speakers RA and RB.

  As mentioned above, according to 2nd Embodiment, the estimated value of the support condition with respect to speaker RA and RB is grasped | ascertained using the audio | voice containing each speech of several speaker RA and RB, and SNS information with respect to each speech. be able to.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the specific embodiments according to the present invention, and various modifications are possible within the scope of the gist of the present invention described in the claims.・ Change is possible.

In addition, the following additional notes are disclosed regarding the above description.
(Supplementary note 1) A conversion step of converting speech information including a speech into first character information, and a part of the character information from the second character information corresponding to the speech based on the appearance frequency of the part of the character information A speech echo detection method in which a computer executes an extraction step of extracting the first character information, and a detection step of detecting a part including the partial character information in the speech as an echo part in the speech.
(Additional remark 2) The classification | category which classify | categorizes the part containing the said one part character information in the said 2nd character information into the part opposite to a part for favor using the character information which represents the character information showing favor and the opposite 3. The speech echo detection method according to claim 1, wherein the detection step includes evaluating the pros and cons of the echo part based on the pros and cons parts.
(Supplementary note 3) Supplementary note 1 or 2, characterized in that it includes a first collection step of collecting the voice information from a microphone and a second collection step of collecting the second character information from a social networking service. The speech echo detection method described in.
(Supplementary note 4) The supplementary note 3 is characterized in that the second collection step collects the second character information posted to the social networking service within a predetermined time after the voice information is collected. The speech echo detection method described in.
(Supplementary Note 5) A first calculation step for calculating a degree of support for each speech based on the volume of voice and / or sound directed to each speech included in each of a plurality of speech information; Classifying the character information for each speech into character information opposite to the character information in favor and the character information in favor of the character information representing Based on the second calculation step for calculating the degree of reverberation for each of the speeches, and the support status of the performers who perform each of the speeches based on the degree of support and the degree of the reverberations for each of the speakers. A speech echo detection method executed by a computer.
(Appendix 6) A conversion step of converting speech information including a speech into first character information, and a part of the character information from the second character information for the speech based on the appearance frequency of the part of the character information A speech echo detection program for causing a computer to execute an extraction step of extracting the first text information and a detection step of detecting a part including the partial character information in the speech as an echo part in the speech.
(Supplementary note 7) A first calculation step for calculating a degree of support for each speech based on a voice and / or a sound volume directed to each speech included in each of a plurality of speech information, and Classifying the character information for each speech into character information opposite to the character information in favor and the character information in favor of the character information representing Based on the second calculation step for calculating the degree of reverberation for each of the speeches, and the support status of the performers who perform each of the speeches based on the degree of support and the degree of the reverberations for each of the speakers. And a speech echo detection program for causing a computer to execute an estimation step.
(Supplementary Note 8) Conversion means for converting speech information including a speech into first character information, and part of the character information from the second character information for the speech based on the appearance frequency of the part of character information A speech echo detection apparatus comprising: extraction means for extracting the first character information; and detection means for detecting a part including the partial character information in the first character information as an echo part in the speech.
(Additional remark 9) The classification | category which classifies the part containing the said one part character information in the said 2nd character information into the part opposite to an approval part using the character information showing the opposite and the character information showing approval The speech echo detection apparatus according to appendix 8, wherein the detection means evaluates the pros and cons of the echo part based on the pros and cons parts.
(Additional remark 10) Additional remark 8 or 9 characterized by including the 1st collection means which collects the said voice information from a microphone, and the 2nd collection means which collects said 2nd character information from a social networking service. The speech echo detection device described in 1.
(Supplementary note 11) The supplementary note 10 is characterized in that the second collection means collects the second character information posted to the social networking service within a predetermined time after the voice information is collected. The speech echo detection device described in 1.
(Additional remark 12) 1st calculation means for calculating the degree of support for each speech based on the voice and / or sound volume directed to each speech included in each of a plurality of speech information; Classifying means for classifying the character information for each speech into the character information opposite to the character information in favor, using the character information representing the opposite to the character information representing, and the character information in favor or the character information opposite Based on the second calculating means for calculating the degree of reverberation for each speech, and the support status of each speaker who performs each speech based on the degree of support and the degree of reverberation for each speaker. A speech echo detection apparatus comprising:

S1, S2 Speech reflection detection system 100 terminal device 200 server device 206 character string conversion unit 207 keyword extraction unit 208 SNS information classification unit 209 echo detection unit 211 support level calculation unit 212 echo level calculation unit 213 support status estimation unit

Claims (9)

A conversion step of converting speech information including a speech into first character information;
Extracting a part of the character information from the second character information for the speech based on the appearance frequency of the part of the character information;
A detection step of detecting a portion including the partial character information in the first character information as a reverberation portion in the speech;
A speech echo detection method executed by a computer. Using the character information representing the opposite and the character information representing the opposite, including a classification step of classifying the portion including the part of the character information in the second character information into the portion opposite to the portion of favor,
2. The speech echo detection method according to claim 1, wherein the detection step evaluates the pros and cons of the echo part based on the pros and cons parts. A first collecting step of collecting the audio information from a microphone;
A second collecting step of collecting the second character information from a social networking service;
The speech echo detection method according to claim 1, wherein the speech echo detection method is included. The second collecting step collects the second character information posted to the social networking service within a predetermined time after the voice information is collected;
The speech echo detection method according to claim 3, wherein: A first calculation step of calculating a degree of support for each speech based on a voice and / or a sound volume directed to each speech included in each of a plurality of audio information;
A classification step for classifying the character information for each speech into character information opposite to the character information for approval, using character information indicating the opposite to the character information indicating approval,
A second calculation step of calculating a degree of reverberation for each speech based on the character information for and the opposite character information;
Based on the degree of support and the degree of reverberation, an estimation step for estimating the support status of each speaker who makes each speech, for each speaker,
A speech echo detection method executed by a computer. A conversion step of converting speech information including a speech into first character information;
Extracting a part of the character information from the second character information for the speech based on the appearance frequency of the part of the character information;
A detection step of detecting a portion including the partial character information in the first character information as a reverberation portion in the speech;
A speech echo detection program that causes a computer to execute. A first calculation step of calculating a degree of support for each speech based on a voice and / or a sound volume directed to each speech included in each of a plurality of audio information;
A classification step for classifying the character information for each speech into character information opposite to the character information for approval, using character information indicating the opposite to the character information indicating approval,
A second calculation step of calculating a degree of reverberation for each speech based on the character information for and the opposite character information;
Based on the degree of support and the degree of reverberation, an estimation step for estimating the support status of each speaker who makes each speech, for each speaker,
A speech echo detection program that causes a computer to execute. Conversion means for converting speech information including a speech into first character information;
Extraction means for extracting a part of character information from the second character information for the speech based on the appearance frequency of the part of character information;
Detecting means for detecting a part including the part of the character information in the first character information as an echo part in the speech;
Speech reverberation detection device. First calculation means for calculating the degree of support for each speech based on the voice and / or sound volume directed to each speech included in each of a plurality of audio information;
Classifying means for classifying the character information for each speech into the character information opposite to the character information for approval, using the character information indicating the opposite to the character information indicating approval,
Second calculating means for calculating a degree of reverberation for each speech based on the character information for and the opposite character information;
Based on the degree of support and the degree of reverberation, estimation means for estimating the support status of each speaker who performs each speech, for each speaker,
Speech reverberation detection device.

Images