JPWO2008126355A1 - Keyword extractor - Google Patents

Abstract

An object of the present invention is to extract keywords in a conversation without predicting and preparing the keywords in the conversation in advance. The keyword extraction device of the present invention includes a voice input unit 101 that inputs a utterance voice of a utterer, an utterance section determination unit 102 that determines a utterance section for each utterer with respect to the input utterance voice, and the above determination. Based on the speech recognition unit 103 that recognizes the utterance speech of the utterance section for each utterer, and the feature of the utterance response that suggests the presence of the keyword based on the response of the other utterers to the utterance speech of each utterer, An interrupt detection unit 104 that detects an interrupt that overlaps a subsequent utterance, a keyword extraction unit 105 that extracts a keyword from an utterance in an utterance section specified based on the interrupt, and a keyword search unit 106 that performs a keyword search using the keyword And a display unit 107 that displays the keyword search result.

Description

  The present invention relates to a keyword extraction device, and more particularly to a keyword extraction device that extracts keywords included in a conversation.

  A conventional keyword extracting device holds correspondence data indicating a correspondence relationship between a keyword such as a microwave oven and action information such as access to a URL in advance. Then, the keyword extraction device detects a keyword from a certain conversation based on the correspondence data, and executes processing based on action information corresponding to the keyword. In this way, information is presented by voice recognition (for example, Patent Document 1).

JP 2005-215726 A (see paragraphs 0021 to 0036 and FIGS. 2 to 3)

However, the apparatus described in Patent Document 1 has a problem that it is difficult to use the correspondence data because the corresponding data must be prepared for each possible scene.
The present invention has been made to cope with the above-described situation, and provides a keyword extraction device that can extract a keyword in a conversation without predicting and preparing the keyword in the conversation in advance. With the goal.

  In order to solve the above-described conventional problems, the present invention includes a voice input unit that inputs a utterance voice of a speaker, a utterance section determination unit that determines a utterance section for each utterer with respect to the input utterance voice, and A speech recognition unit that recognizes the speech of the determined speech section for each speaker, and a speech response that suggests the presence of a keyword based on the responses of other speakers to the speech of each speaker An utterance response feature extraction unit that extracts features, and a keyword extraction unit that extracts the keywords from the utterance speech of the utterance section specified based on the extracted utterance response features.

  According to the present invention, it is possible to extract a keyword in a conversation without predicting and preparing the keyword in the conversation in advance.

1 is a block diagram showing a configuration example of an entire system including a keyword extraction device in Embodiment 1 of the present invention. The figure which shows the example of the speech area in Embodiment 1 of this invention. The flowchart which shows operation | movement of the keyword extraction apparatus of FIG. The block diagram which shows the structural example of the keyword extraction apparatus in Embodiment 2 of this invention. The figure which shows the example of the pitch pattern in Embodiment 2 of this invention. 5 is a flowchart showing the operation of the keyword extraction device in FIG. 4. The block diagram which shows the structural example of the keyword extraction apparatus in Embodiment 3 of this invention. The flowchart which shows operation | movement of the keyword extraction apparatus of FIG. The block diagram which shows the structural example of the keyword extraction apparatus in Embodiment 4 of this invention. The figure which shows the example of the speech area in the Embodiment 4 of this invention, speech content, and a facial expression recognition result. 10 is a flowchart showing the operation of the keyword extraction device of FIG. The block diagram which shows the structural example of the keyword extraction apparatus in Embodiment 5 of this invention. The flowchart which shows operation | movement of the keyword extraction apparatus of FIG.

Explanation of symbols

100, 100A, 100B, 100C, 100D Keyword extraction device 101 Voice input unit 102 Speech segment determination unit 103 Speech recognition unit 104 Interrupt detection unit 105, 105A, 105B, 105C, 105D Keyword extraction unit 106 Keyword search unit 107 Display unit 201 Pitch determination unit 202 Pitch pattern determination unit 301 Function phrase extraction unit 302 Function phrase storage unit 401 Video input unit 402 Expression recognition unit 501 Swell reaction detection unit

Embodiments 1 to 5 of the present invention will be described below with reference to the drawings. In the first to fifth embodiments, for example, a case where two speakers A and B are talking using an information terminal such as a mobile phone will be described.
(Embodiment 1)
FIG. 1 is a block diagram showing a configuration example of the entire system including a keyword extracting device according to Embodiment 1 of the present invention.
In FIG. 1, a keyword extraction device 100 is an information terminal of a certain speaker A, and is configured to be connected to a network 400 such as the Internet. The network 400 is configured to be connected to the information terminal 200 and the search server 300 of another speaker B. The keyword extraction device 100 and the information terminal 200 are information terminals such as a mobile phone, a notebook personal computer, and a mobile information terminal. The search server 300 is a server equipped with a known search engine.

The keyword extraction device 100 includes a voice input unit 101, a speech segment determination unit 102, a voice recognition unit 103, an interrupt detection unit 104, a keyword extraction unit 105, a keyword search unit 106, and a display unit 107.
The voice input unit 101 is for inputting a voice of a speaker (hereinafter referred to as “speech voice”). The voice input unit 101 corresponds to, for example, a microphone, a communication interface with the network 400, or the like.

The utterance section determination unit 102 determines an utterance section for each speaker with respect to the input utterance voice. The utterance section refers to a section from the beginning to the end of the conversation by the speaker.
For example, when the conversation between the speaker A and the speaker B is as shown in FIG. 2 (a) or FIG. 2 (b), the speech segment determination unit 102 determines the end time te1 from the conversation start time ts1 of the speaker A. The section up to that time, that is, ts1-te1 is determined as the utterance section 1 of the speaker A. Furthermore, the utterance section determination unit 102 determines the section from the start time ts2 to the end time te2 of the conversation of the speaker B, that is, the section of ts2-te2 as the speech section 2 of the speaker B.

  Returning to FIG. 1, the voice recognition unit 103 recognizes the uttered voice of the determined utterance section for each speaker. More specifically, the voice recognition unit 103 converts the conversation voices of all the speakers into text characters using a known voice recognition technique. Furthermore, the voice recognition unit 103 associates the start time (start point) and the end time (end point) of the conversation voice of each speaker.

The interrupt detection unit 104 (speech response feature extraction unit) detects an interrupt in which the utterance feature, that is, the preceding utterance and the subsequent utterance overlap, based on the utterance voice of each utterer for the determined utterance period. For example, when the conversation between the speaker A and the speaker B is the conversation shown in FIG. 2B, the interrupt detection unit 104 follows the speaker B in the middle of the preceding speech of the speaker A, that is, at ts1. Since the utterance has been started, the interrupt is detected. This detection method is as follows.
That is, the interrupt detection unit 104 first measures a section (hereinafter referred to as an utterance interval) from the start time of the subsequent utterance to the end time of the immediately preceding utterance. For example, in the case of FIGS. 2A and 2B, the interrupt detection unit 104 calculates the speech interval using the calculation formula of utterance interval = ts2-te1 in FIGS. 2A and 2B. Next, the interrupt detection unit 104 determines whether the utterance interval becomes a negative value (see FIG. 2B) as a result of the above calculation. When the speech interval is a negative value (see FIG. 2B), the interrupt detection unit 104 detects that there is an interrupt.

  Based on the extracted utterance feature, that is, the interruption in which the preceding utterance and the succeeding utterance overlap, the keyword extraction unit 105 selects the utterance voice recognized by the voice recognition unit 102 within the conversation of the utterance voice. Extract a topic word (hereinafter referred to as a keyword). Specifically, the keyword extraction unit 105 acquires the conversation voice recognized by the voice recognition unit 102 from the voice recognition unit 102. This conversation voice is associated with the start time and end time of each speaker. In addition, the keyword extraction unit 105 is interrupted by the interrupt detection unit 104 and the utterance interval (for example, the utterance interval 2 of the speaker B in FIG. 2B) in which the interruption is detected by the interrupt detection unit 104. The utterance section (for example, the utterance section 1 of the speaker A in FIG. 2) is acquired. Each of these utterance sections is associated with a start time and an end time.

Furthermore, when extracting the said keyword, the keyword extraction part 105 extracts the last component (for example, noun) in the preceding utterance interrupted as a keyword, for example. Here, the end in the preceding utterance refers to the inside of the utterance section (for example, ts1-ts2 in FIG. 2B) before the interruption (for example, the time of ts2 in FIG. 2B).
Specifically, first, the keyword extracting unit 105 selects the utterance section (for example, the utterance section of the utterer with the earlier start time from the utterance sections of the respective utterers (for example, the utterance sections 1 and 2 in FIG. 2B). For example, the utterance section 1) in FIG. 2 (b) is selected. Next, in the selected utterance section (for example, utterance section 1 in FIG. 2B), the keyword extraction unit 105 starts the other acquired utterance section (that is, interrupt time, for example, FIG. 2B). The component (for example, noun) immediately before ts2) is detected. Next, the keyword extraction unit 105 extracts the detected constituents (for example, nouns) as keywords.

  The keyword search unit 106 performs a keyword search using the extracted keyword. Specifically, first, the keyword search unit 106 connects to the search server 300 via the network 400. Then, the search server 300 receives the keyword search request from the keyword search unit 106, and returns the search result of the keyword search to the keyword search unit 106 of the keyword extracting device 100 via the network 400. The keyword search unit 106 receives the search result of the keyword search from the search server 300 by the return.

  The display unit 107 displays the search result by the keyword search unit 106, that is, the search result of the search server 300. The display unit 107 is a display device such as a display or a display panel.

  In the present embodiment, the speech segment determination unit 102, the speech recognition unit 103, the interrupt detection unit 104, the keyword extraction unit 105, and the keyword search unit 106 correspond to a processing device such as a CPU. In addition, the keyword extraction device 100 is assumed to have a known configuration including a storage device (not shown) such as a memory.

Next, the operation of the keyword extracting device 100 will be described with reference to FIG. In FIG. 3, for example, it is assumed that two speakers A and B are having a conversation using the keyword extraction device 100 and the information terminal 200.
First, the keyword extraction device 100 (the utterance section determination unit 102) determines an utterance section for each speaker with respect to the utterance voices input from the voice input unit 100 and the information terminal 200 (step S101). At the time of this determination, the utterance section determination unit 102 determines whether or not the volume of the uttered voice of each speaker is equal to or greater than a threshold value, and determines a section that is equal to or greater than the threshold value as the utterance section.
For example, when the conversation between the speaker A and the speaker B is as shown in FIG. 2 (a) or FIG. 2 (b), the speech segment determination unit 102 determines the end time te1 from the conversation start time ts1 of the speaker A. The section up to that time, that is, ts1-te2 is determined as the utterance section 1 of the speaker A. Further, the utterance section determination unit 103 determines the section from the start time ts2 to the end time te2 of the conversation of the speaker B, and the section ts2-te2 as the speech section 2 of the speaker B.

  Next, the keyword extraction device 100 (voice recognition unit 103) recognizes the utterance voice of the determined utterance section for each speaker (step S102). This recognition shall be performed by the feature analysis by a frequency band, for example. Furthermore, when performing the above recognition, the speech recognition unit 103 converts the conversation speech of all the speakers into text characters using a known speech recognition technique.

  Next, the keyword extraction device 100 (interrupt detection unit 104) detects an interrupt based on the determined speech period (step S103). Specifically, the interrupt detection unit 104 subtracts the end time of the immediately preceding utterance from the start time of the subsequent utterance, that is, the utterance interval (for example, te1-ts2 in FIGS. 2A and 2B). ). As a result of this calculation, if the value of the speech interval (for example, te1-ts2 = speech interval in FIG. 2B) is negative, the interrupt detection unit 104 determines that there is an interruption of the subsequent speech. .

Next, the keyword extraction device 100 (keyword extraction unit 105) extracts and determines the keywords in the detected voice conversation (interactive voice recognition recognized in step S102) with the interruption (step S104). Specifically, the keyword extraction unit 105 extracts a noun in the preceding utterance immediately before the subsequent utterance, and determines this noun as a keyword in the utterance.
For example, when utterer A starts to speak “This time, New Tokyo Tower ...” at the time ts1 in FIG. 2B, the utterer B is “oh” at the time ts2 in FIG. When the conversation begins, “Where is it possible?”, The keyword extraction unit 105 determines the noun “Shin Tokyo Tower” of the speaker A immediately before ts2 as a keyword. As a result, the keyword extraction unit 105 determines “New Tokyo Tower” as a topic in the conversation without extracting the keyword of “New Tokyo Tower” from the database in which keywords predicted in advance are registered. be able to.

  Note that when the utterance interval shows a positive value (see FIG. 2A), the keyword extraction unit 105 determines that there is no keyword being uttered, and does not extract the keyword.

  Next, the keyword extraction device 100 (keyword search unit 106) performs keyword search for the determined keyword (step S105). Specifically, the keyword search unit 106 first requests the search server 300 for the keyword search via the network 400. Then, the search server 300 performs a keyword search in response to the request, and transmits the search result to the keyword search unit 106. Next, the keyword search unit 106 receives the search result transmitted from the search server 300.

  Next, the keyword search unit 106 displays the received search result on the display unit 107 (step S106). Thereby, the speaker can grasp the information (search result) related to the keyword (for example, New Tokyo Tower) in the conversation.

  Also, in place of the interrupt detection unit 104, operating a silence detection unit that detects silence whose utterance interval is greater than or equal to a predetermined threshold (for example, 3 seconds) can also be characterized by an utterance response that suggests the presence of a keyword. Useful for extraction.

  As described above, according to the present embodiment, the keyword extraction device 100 detects an interrupt as a feature of an utterance response that suggests the presence of a keyword, and extracts a keyword in the conversation. For this reason, the keyword extraction apparatus 100 can extract a keyword in a conversation based on the presence or absence of a speaker's interruption without predicting the keyword in the conversation in advance and registering it in a database or the like.

  In the first embodiment, the keyword extracting apparatus 100 has been described with respect to the case where the processes of steps S101 to S106 in FIG. 3 are sequentially performed, but the present invention is not limited to this. For example, the keyword extraction device 100 may execute the steps in FIG. 3 by changing the order of the steps, or may execute the steps in parallel.

(Embodiment 2)
The keyword extracting apparatus according to the second embodiment extracts keywords in a conversation based on a pitch (sound pitch) pattern that is a feature of an utterance response.
FIG. 4 is a block diagram illustrating a configuration example of the keyword extracting device according to the second embodiment of the present invention. In the second embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals and terms as those in the first embodiment, and redundant description is omitted.
In FIG. 4, the keyword extraction device 100 </ b> A includes a pitch determination unit 201 and a pitch pattern determination unit 202 instead of the interrupt detection unit 104 of the first embodiment in FIG. 1. Furthermore, the keyword extraction device 100A is different from the first embodiment in that it includes a keyword extraction unit 105A instead of the keyword extraction unit 105 of the first embodiment in FIG. The pitch determination unit 201, the pitch pattern determination unit 202, and the keyword extraction unit 105A are processing devices such as a CPU. In addition, the configuration of the entire system including the information terminal 200 is the same as that of FIG.

  The pitch determination unit 201 and the pitch pattern determination unit 202 (also collectively referred to as an utterance response feature extraction unit) perform the utterance of the utterance interval determined by the utterance interval determination unit 102 based on the utterance speech of each speaker. A pitch pattern that is a feature is extracted. Specifically, the pitch determination unit 201 determines the pitch of the speech voice. The pitch determination unit 201 of the present embodiment determines the pitch by dividing the uttered speech every 10 ms, for example.

Based on the determined pitch, the pitch pattern determination unit 202 has a descending pitch at the end of the preceding utterance (see tc1-te1 in FIG. 5), and a succeeding utterance immediately after the preceding utterance is an ascending pitch ( The pitch pattern (characteristic of the utterance) to be determined (see between tc2 and te2 in FIG. 5) is determined. An example of this determination is shown in FIG. The horizontal axis in FIG. 5 represents time, and the vertical axis represents frequency.
In the utterance section ts1-te1, there is a preceding utterance “New Tokyo Tower”, and in the utterance section ts2-te2, there is a subsequent utterance “Is that? A descending pitch is determined at the end of the preceding utterance of “New Tokyo Tower”, and an ascending pitch is determined for the subsequent utterance of “Is that ...?”. This determination is made because the pitch pattern determination unit 202 determines as follows.

  That is, the pitch pattern determination unit 202 has a higher frequency f at the end (at the end) of the utterance section than the frequency f at the midpoint tc1 in the utterance section ts1-te1 of “New Tokyo Tower is” in FIG. This is because it is determined that the pitch is rising. In addition, the pitch pattern determination unit 202 decreases in the utterance interval ts2-te2 of “What?” In FIG. 5 because the frequency f at the end (at the end) of the utterance interval is lower than the frequency f at the midpoint tc2. This is because the pitch is determined.

  In addition, although the pitch pattern determination part 202 of this Embodiment demonstrates the case where a raise pitch or a fall pitch is determined on the basis of the frequency of the middle point of an utterance area, it is not restricted to this. For example, the pitch determination unit 201 may perform determination based on a time point that is a predetermined period (for example, time T) from the end of the utterance period (for example, te1 and te2 in FIG. 5).

  The keyword extraction unit 105A extracts keywords from the preceding utterances shown in the determined pitch pattern. In this extraction, the keyword extraction unit 105A extracts, for example, the last constituent (for example, a noun) in the preceding utterance shown in the pitch pattern as a keyword.

  Next, the operation of the keyword extracting device 100A will be described with reference to FIG. In FIG. 6, for example, the speaker A uses the keyword extraction device 100A to say “Now, Tokyo Tower ...”, and then the speaker B uses the information terminal 200 to say “ Explain on the premise of saying "...?" 7 are the same as steps S101 to S102 and S105 to S106 in FIG. 3, and will be omitted as appropriate.

  First, the keyword extraction device 100A (the utterance section determination unit 102) uses the utterance section for each utterer (the utterance section 1 in FIG. 2A, FIG. 2) for the utterance speech input from the voice input section 100 and the information terminal 200. (See utterance section 2 in (b)) (step S101). Next, the keyword extraction device 100A (voice recognition unit 103) recognizes the utterance voice of the determined utterance section for each speaker (step S102).

  Next, the keyword extraction device 100A (pitch determination unit 201), for example, the utterance section 1 of the preceding utterance of the speaker A (see FIG. 2A) and the utterance section 2 of the subsequent utterance of the speaker B (FIG. 2 ( Based on the uttered voice of (b), the pitch of the uttered voice is determined (step S103A).

  Next, the keyword extraction device 100A (pitch pattern determination unit 202) determines whether there is a pitch pattern that changes from the descending pitch to the ascending pitch when the preceding utterance is shifted to the succeeding utterance based on the determined pitch. Determination is made (step S103B). Specifically, the pitch pattern determination unit 202 determines that the end of the preceding utterance is the descending pitch (see tc1-te1 in FIG. 5), and the succeeding utterance immediately after the preceding utterance is the ascending pitch (tc2 in FIG. 5). The pitch pattern is determined.

  Next, the keyword extraction device 100A (keyword extraction unit 105A) determines the preceding utterance of the utterance voice (recognized in step S102) indicated in the determined pitch pattern (for example, “New Tokyo Tower ]), Keywords are extracted (step S104A). In this extraction, the keyword extraction unit 105A extracts, for example, “Shin Tokyo Tower”, which is the last noun in the preceding utterance shown in the pitch pattern, as a keyword.

  Next, the keyword extraction device 100A (keyword search unit 106) performs keyword search for the determined keyword on the search server 300 via the network 400 (step S105). Next, the keyword search unit 106 displays the received search result on the display unit 107 (step S106). Thereby, the speaker can grasp information (search result) related to the topic word (for example, New Tokyo Tower).

  As described above, according to the present embodiment, keyword extracting apparatus 100A determines a pitch pattern that is a feature of an utterance response that suggests the presence of a keyword, and extracts a keyword in a conversation. For this reason, the keyword extraction device 100A can extract keywords in a conversation from the presence or absence of a pitch pattern without preparing in advance for registering the keywords used in the conversation in a database or the like. .

  In the second embodiment, the keyword extraction apparatus 100A has been described with respect to the case where the processes of steps S101 to S102, S103A to S103B, S104A, and S105 to S106 in FIG. 7 are sequentially performed, but the present invention is not limited thereto. For example, the keyword extracting device 100A may execute the steps in FIG. 7 by changing the order of the steps, or may execute the steps in parallel.

(Embodiment 3)
The keyword extraction device according to the third embodiment extracts keywords in a conversation based on a function phrase that is a feature of an utterance response.
FIG. 7 is a block diagram illustrating a configuration example of the keyword extraction device according to Embodiment 3 of the present invention. In the third embodiment, the same reference numerals and terms as those in the first embodiment are assigned to the same parts as those in the first embodiment, and the duplicate description is omitted.
In FIG. 7, the keyword extraction device 100B has a function phrase extraction unit 301 (utterance response feature extraction unit) instead of the interrupt detection unit 104 of the first embodiment in FIG. Furthermore, the keyword extraction device 100B includes a function phrase storage unit 302. Further, the keyword extraction device 100B is different from the first embodiment in that it includes a keyword extraction unit 105B instead of the keyword extraction unit 105 of the first embodiment in FIG. The function phrase extraction unit 301 is a processing device such as a CPU, and the function phrase storage unit 302 is a storage device such as a memory. In addition, the configuration of the entire system including the information terminal 200 is the same as that of FIG.

  The function phrase storage unit 302 stores a predetermined function phrase. This function phrase is a word representing the type of response, and is used in common with conversations regardless of various different conversation contents. For example, as a functional phrase, a question sentence such as “Is it?”, An agreement sentence such as “Like” or “I see” or “It is”, a negative sentence such as “No”, a request sentence such as “Please” Exclamation sentences such as “Oh” and indentation sentences such as “Nadeyanen” are applicable.

  The functional phrase extraction unit 301 extracts the functional phrase that is a feature of the uttered voice from the uttered voice. Specifically, the functional phrase extraction unit 301 compares a word included in the speech to be extracted with a functional phrase in the functional phrase storage unit 302, and extracts a functional phrase included in the speech.

  Next, the operation of the keyword extracting device 100B will be described with reference to FIG. In FIG. 8, for example, after the speaker A uses the keyword extraction device 100 </ b> B to say “This time, New Tokyo Tower can be created.”, The speaker B uses the information terminal 200 to say “Oh, it "Where can I do that?" Note that the processes in steps S101 to S102 and S105 to S106 in FIG. 8 are the same as the processes in steps S101 to S102 and S105 to S106 in FIG.

  First, the keyword extraction device 100B (the utterance section determination unit 102) uses the utterance section for each utterer (the utterance section 1 in FIG. 2A, FIG. 2) for the speech input from the voice input section 100 and the information terminal 200. (See utterance section 2 in (b)) (step S101). Next, the keyword extraction device 100B (voice recognition unit 103) recognizes the utterance voice in the determined utterance section for each speaker (step S102).

  Next, the keyword extraction device 100B (functional phrase extraction unit 301), for example, the utterance section 1 of the preceding utterance of the speaker A (see FIG. 2A) and the utterance section 2 of the subsequent utterance of the speaker B (FIG. 2). A functional phrase representing a question sentence or the like is extracted from the uttered voice of (b). Specifically, the functional phrase extraction unit 301 compares a sequence of words included in the uttered speech to be extracted with the functional phrase in the functional phrase storage unit 302, and extracts a functional phrase included in the uttered speech. To do. In the present embodiment, the functional phrase extraction unit 301 extracts the functional phrase of the question sentence “Is it?” From the utterance voice of “Oh, where can I do it?”. Here, the speech recognition result may be used for the word sequence included in the speech voice.

  Next, the keyword extraction device 100B (keyword extraction unit 105B) extracts a keyword from the utterance voice immediately before the utterance including the extracted functional phrase (recognized in step S102) (step S104B). When extracting this keyword, the keyword extraction unit 105B, for example, from the previous utterance "Now, you can make New Tokyo Tower." Extract as keywords.

  Next, the keyword extraction device 100B (keyword search unit 106) performs keyword search for the extracted keyword on the search server 300 via the network 400 (step S105). Next, the keyword search unit 106 displays the received search result on the display unit 107 (step S106). Thereby, the speaker can grasp information (search result) related to a keyword (for example, New Tokyo Tower) which is a topic in the conversation.

  In addition, according to the present embodiment, a question from a previous utterance, such as when utterer A asks "What is that?" And utterer B answers "What is New Tokyo Tower?" When the functional phrase of the sentence (“What was it?”) Is extracted, the keyword extraction unit 105B can be operated so as to extract the keyword (“New Tokyo Tower”) from the subsequent utterance. is there. At this time, whether the keyword is extracted from the immediately preceding utterance speech or the keyword is extracted from the immediately following utterance speech can be switched as follows. That is, it can be used by switching from the immediately preceding utterance when the pronoun pronoun “it” is included, from the immediately following utterance when including the indicating pronoun “that”, and from the immediately following utterance in other cases. At that time, the feature of the utterance response may be captured by using (using in combination) a pitch pattern in which the preceding utterance is the rising pitch and the subsequent utterance is the descending pitch in the same manner as in the second embodiment.

  As described above, according to the present embodiment, the keyword extraction device 100B extracts functional phrases (question sentences, etc.) that are commonly used regardless of the conversation content (genre), and extracts keywords in the conversation. To do. For this reason, in the keyword extracting device 100B, it is possible to extract a keyword by extracting a commonly used function phrase from the conversation sentence. Therefore, the keyword extraction device 100B is useful because it can extract keywords without preparing in advance a keyword corresponding to each genre conversation and registering it in a database or the like.

  In the third embodiment, the keyword extraction apparatus 100B has been described with respect to the case where the processes of steps S101 to S102, S103C, S104B, and S105 to S106 in FIG. 8 are sequentially performed. However, the present invention is not limited to this. For example, the keyword extracting device 100B may execute the steps in FIG. 9 by changing the order of the steps, or may execute the steps in parallel.

(Embodiment 4)
The keyword extraction device according to the fourth embodiment extracts keywords in a conversation based on changes in the facial expression of a person who has heard spoken speech.
FIG. 9 is a block diagram illustrating a configuration example of the keyword extracting device according to the fourth embodiment of the present invention. In the fourth embodiment, the same reference numerals and terms as those in the first embodiment are assigned to the same parts as those in the first embodiment, and the duplicate description is omitted.

  In FIG. 9, the keyword extraction device 100 </ b> C includes a video input unit 401 and a facial expression recognition unit 402 (also collectively referred to as an utterance response feature extraction unit) instead of the interrupt detection unit 104 of the first embodiment in FIG. 1. Have. Further, the keyword extraction apparatus 100C is different from the first embodiment in that it includes a keyword extraction unit 105C instead of the keyword extraction unit 105 of the first embodiment in FIG. The image input unit 401 is a camera, and the facial expression recognition unit 402 is a processing device such as a CPU. In addition, the configuration of the entire system including the information terminal 200 is the same as that of FIG.

The video input unit 401 is for inputting image data including a user's face portion. When the facial expression recognition unit 402 converts the image data into digital original image data that can be used to estimate the facial expression of the user, the facial expression recognition unit 402 extracts the face area of the user included in the original image data, and extracts the user's face area from the extracted face area. The contour position of at least one facial organ such as eyes and mouth constituting the face is extracted. Then, the facial expression recognition unit 402 extracts the contours of the upper and lower ends of the facial organ acquired over a plurality of video frames, and determines the facial expression of the user (for example, neutrality, surprise, etc.) based on how the facial organ contour is opened or bent. , Joy, anger, etc.).
At that time, the facial expression recognition unit 402 associates the time in the utterance section for each speaker obtained from the utterance section determination unit 102 with the recognition result of the facial expression of a person other than the speaker. Further, the facial expression recognition unit 402 extracts facial expression change points from the facial expression recognition result.
For example, in FIG. 10, t10 is the utterance start time of the utterance section 1 by the speaker A, t11 and t12 are equally spaced times following t10, t20 is the utterance start time of the utterance section 2 by the speaker B, t21, t22 is an equally spaced time following t20. Here, the facial expression recognition unit 402 recognizes the facial expression of the speaker B at each of the times t10, t11, and t12 and the facial expression of the speaker A at each of the times t20, t21, and t22. In this example, the expression of the speaker B at time t11 is a surprised expression, and at other times, the expression is neutral regardless of the speaker. That is, the facial expression recognition unit 402 extracts time t11 as a facial expression change point.

  When the facial expression recognition unit 402 recognizes that the recognized facial expression is a neutral facial expression at the start of the utterance and changes to another facial expression during the utterance, the keyword extraction unit 105C changes the facial expression. A word uttered at the time corresponding to the point is extracted as a keyword. At that time, the keyword extraction unit 105C may obtain the word at the time corresponding to the facial expression from the section information for each word in the speech recognition result, or may estimate it from the number of syllables included in the uttered speech. The corresponding time here refers to the time from when a word is perceived until the reaction appears in the facial expression (for example, 0.1 seconds), and the end of the word and the expression of the facial expression are associated with each other. It's time.

  Next, the operation of the keyword extracting device 100C will be described with reference to FIG. In FIG. 11, for example, the speaker A uses the keyword extraction device 100C to say “New Tokyo Tower can be done next”, and then the speaker B uses the information terminal 200 to say “What is that? ”On the assumption that“ Note that the processes in steps S101 to S102 and S105 to S106 in FIG. 11 are the same processes as steps S101 to S102 and S105 to S106 in FIG. The voice and video of the speaker B are input using the information terminal 200. For the sake of convenience, the input will be described as being input from the audio input unit 101 and the video input unit 401 in the same manner as the speaker A. .

  First, the keyword extraction device 100C (the utterance section determination unit 102) determines the utterance section for each speaker (see the utterance section 1 and the utterance section 2 in FIG. 10) for the uttered speech input from the speech input section 101 (see FIG. 10). Step S101). Next, the keyword extracting device 100C (voice recognition unit 103) recognizes the uttered voice of the determined utterance section for each speaker (step S102).

  On the other hand, the keyword extraction device 100C (the video input unit 401 and the facial expression recognition unit 402), for example, of the speaker B at the time corresponding to the utterance voice (see FIG. 10) in the utterance section 1 which is the preceding utterance uttered by the speaker A. The facial expression is recognized, and the facial expression of the speaker A at the time corresponding to the utterance voice (see FIG. 10) in the utterance section 2 which is the subsequent utterance uttered by the speaker B is recognized. That is, rather than recognizing the facial expression of the speaker, the facial expression of the person who is listening to the speech, that is, the facial expression of another speaker relative to the speech of the speaker is recognized (step S103D).

  Next, the keyword extracting device 100A (keyword extracting unit 105C) recognizes that the recognized facial expression is a neutral facial expression at the start of utterance and has changed to another facial expression during the utterance. A word uttered at the time corresponding to the facial expression change point is extracted as a keyword (step S104C). In the above example, “New Tokyo Tower” is extracted as a word corresponding to the time when the expression changes from neutral to a surprising expression.

  Next, the keyword extraction device 100C (keyword search unit 106) performs keyword search for the determined keyword to the search server 300 via the network 400 (step S105). Next, the keyword search unit 106 displays the received search result on the display unit 107 (step S106). Thereby, the speaker can grasp information (search result) related to the topic word (for example, New Tokyo Tower).

  As described above, according to the present embodiment, the keyword extraction device 100C extracts keywords in a conversation based on the recognition result of the facial expression of another person who is listening to the uttered voice. For this reason, in the keyword extraction device 100C, the keyword in the conversation is obtained from the feature of the utterance response that is captured as a change in facial expression without preparing the keyword used in the conversation in advance and registering it in the database or the like. Can be extracted.

  It should be noted that the same effect can be obtained by converting the expression of the eyes and the degree of opening of the mouth into numerical values instead of the facial expression recognition unit 402, and detecting changes in facial expressions based only on the magnitudes of those changes.

  In the fourth embodiment, the keyword extraction device 100C has been described with respect to the case where the processes of steps S101 to S102, S103D, S104C, and S105 to S106 in FIG. 11 are sequentially performed. However, the present invention is not limited to this. For example, the keyword extraction device 100C may execute the steps in FIG. 11 by changing the order of the steps, or may execute the steps in parallel.

(Embodiment 5)
The keyword extraction device according to the fifth embodiment extracts keywords in a conversation based on an excitement reaction of a person who has heard an uttered voice.
FIG. 12 is a block diagram illustrating a configuration example of the keyword extracting device according to the fifth embodiment of the present invention. In the fifth embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals and terms as those in the first embodiment, and redundant description is omitted.

  In FIG. 12, the keyword extraction device 100D includes a climax reaction detection unit 501 (also referred to as an utterance response feature extraction unit) instead of the interrupt detection unit 104 of the first embodiment in FIG. Further, the keyword extraction device 100D is different from the first embodiment in that it includes a keyword extraction unit 105D instead of the keyword extraction unit 105 of the first embodiment in FIG. The swell reaction detection unit 501 is a processing device such as a CPU. In addition, the configuration of the entire system including the information terminal 200 is the same as that of FIG.

The swell response detector 501 detects a sway response from voice and sound. Specifically, a swell response is detected by detecting a laughing voice, detecting a voice with a high degree of excitement, or detecting a sound of clapping or kneeling. For the laughing voice, applause, and kneeling sound, the climax reaction detection unit 501 facilitates a learning sample in advance, creates a GMM (gamma mixture model), obtains a likelihood for the input, and performs threshold processing. To detect. The excitement reaction detection unit 501 linearly combines amounts obtained by normalizing the loudness level, the pitch height, and the speaking speed with the average value of the speaker for a highly excited sound. It is detected by digitizing and threshold processing.
At this time, the climax reaction detection unit 501 regards the climax reaction detected near the end of the utterance interval determined by the utterance interval determination unit 102 as an excitement response corresponding to the utterance.

  The keyword detection unit 105D extracts a keyword from the utterance corresponding to the excitement reaction.

  Next, the operation of the keyword extracting device 100D will be described with reference to FIG. In FIG. 13, for example, the speaker A uses the keyword extraction device 100 </ b> C to say “Now, Tokyo Tower ...”, and then the speaker B uses the information terminal 200 to say “Ahaha”. I will explain on the assumption that I laughed. The processes in steps S101 to S102 and S105 to S106 in FIG. 13 are the same processes as steps S101 to S102 and S105 to S106 in FIG.

  First, the keyword extraction device 100D (the utterance section determination unit 102) determines an utterance section for each speaker from the speech input from the voice input unit 101 and the information terminal 200 (step S101). Next, the keyword extraction device 100D (voice recognition unit 103) recognizes the utterance voice in the determined utterance section for each speaker (step S102).

  Next, the keyword extraction device 100D (exciting reaction detection 501) detects the presence of an enlarging reaction, for example, in the vicinity of the utterance section uttered by the speaker A (step S103E). As a result, in the above-described utterance example, the laughter voice GMM is collated with high likelihood immediately after the utterance section of the speaker A, so that it is detected as an excitement reaction.

  Next, the keyword extraction device 100A (keyword extraction unit 105D) extracts, as a keyword, a word (for example, “New Tokyo Tower”) uttered in the utterance section corresponding to the excitement reaction (step S104D).

  Next, the keyword extraction device 100D (keyword search unit 106) performs keyword search of the determined keyword to the search server 300 via the network 400 (step S105). Next, the keyword search unit 106 displays the received search result on the display unit 107 (step S106). Thereby, the speaker can grasp information (search result) related to the topic word (for example, New Tokyo Tower).

  As described above, according to the present embodiment, the keyword extracting device 100D detects the excitement reaction of the person who has heard the uttered voice and extracts the keywords in the conversation. For this reason, in the keyword extraction apparatus 100D, it is possible to predict the keyword used in the conversation in advance from the characteristics of the utterance response that can be regarded as a swell and applause without preparing to register it in a database or the like. The keywords in can be extracted.

  In the fifth embodiment, the keyword extraction device 100D has been described with respect to the case where the processes of steps S101 to S102, S103E, S104D, and S105 to S106 in FIG. 13 are sequentially performed. However, the present invention is not limited to this. For example, the keyword extracting device 100D may execute the steps in FIG. 13 by changing the order of the steps, or may execute the steps in parallel.

  In Embodiments 1 to 3 and 5, the keyword extraction device (keyword extraction unit) has described the case where the noun at the end (immediately before interruption) in the utterance section is extracted as a keyword. However, the present invention is not limited to this. For example, the keyword extraction unit may search for a noun that is conceptually lowest among a plurality of nouns included in the preceding utterance to be searched as a keyword. In this case, the keyword extraction device further includes a dictionary information storage unit (not shown) such as a memory, and the dictionary information storage unit includes a conceptual upper level (for example, Italian cuisine) and a conceptual lower level (for example, pasta). ) Is used to store dictionary information organized and organized. Then, the keyword extraction unit extracts, as keywords, the conceptually lowest nouns included in the dictionary information of the dictionary information storage unit (not shown) from the nouns included in the utterance to be extracted. Thereby, the noun of a low-order concept is extracted as a keyword.

  In Embodiments 1 to 3 and 5, the keyword extraction unit may extract the noun with the highest pitch among the nouns included in the utterance to be extracted as the keyword, or the most frequently used. You may make it extract a noun as a keyword. Alternatively, the keyword extraction unit uses, as a keyword, a noun in which a combination of various parameters indicating the pitch and the number of times of use of each noun is optimal (a predetermined parameter pattern) from the nouns included in the utterance to be extracted. You may make it extract.

Although the present invention has been described in detail and with reference to specific embodiments, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the invention.
This application is based on a Japanese patent application filed on March 29, 2007 (Japanese Patent Application No. 2007-088321), the contents of which are incorporated herein by reference.

  The keyword extracting device of the present invention is useful for extracting important keywords included in a conversation. The keyword extraction device can be applied to uses such as a telephone, an in-vehicle terminal, a television, a conference system, a call center system, and a personal computer.

  The present invention relates to a keyword extraction device, and more particularly to a keyword extraction device that extracts keywords included in a conversation.

  A conventional keyword extracting device holds correspondence data indicating a correspondence relationship between a keyword such as a microwave oven and action information such as access to a URL in advance. Then, the keyword extraction device detects a keyword from a certain conversation based on the correspondence data, and executes processing based on action information corresponding to the keyword. In this way, information is presented by voice recognition (for example, Patent Document 1).

JP 2005-215726 A (see paragraphs 0021 to 0036 and FIGS. 2 to 3)

However, the apparatus described in Patent Document 1 has a problem that it is difficult to use the correspondence data because the corresponding data must be prepared for each possible scene.
An object of the present invention is to cope with the above situation, and provides a keyword extraction device that can extract keywords in a conversation without predicting and preparing the keywords in the conversation in advance. It is to be.

  In order to solve the above-described conventional problems, the present invention includes a voice input unit that inputs a utterance voice of a speaker, a utterance section determination unit that determines a utterance section for each utterer with respect to the input utterance voice, and A speech recognition unit that recognizes the speech of the determined speech section for each speaker, and a speech response that suggests the presence of a keyword based on the responses of other speakers to the speech of each speaker An utterance response feature extraction unit that extracts features, and a keyword extraction unit that extracts the keywords from the utterance speech of the utterance section specified based on the extracted utterance response features.

  According to the keyword extracting device of the present invention, it is possible to extract a keyword in a conversation without predicting and preparing the keyword in the conversation in advance.

Embodiments 1 to 5 of the present invention will be described below with reference to the drawings. In the first to fifth embodiments, for example, a case where two speakers A and B are talking using an information terminal such as a mobile phone will be described.
(Embodiment 1)
FIG. 1 is a block diagram showing a configuration example of the entire system including a keyword extracting device according to Embodiment 1 of the present invention.
In FIG. 1, a keyword extraction device 100 is an information terminal of a certain speaker A, and is configured to be connected to a network 400 such as the Internet. The network 400 is configured to be connected to the information terminal 200 and the search server 300 of another speaker B. The keyword extraction device 100 and the information terminal 200 are information terminals such as a mobile phone, a notebook personal computer, and a mobile information terminal. The search server 300 is a server equipped with a known search engine.

The keyword extraction device 100 includes a voice input unit 101, a speech segment determination unit 102, a voice recognition unit 103, an interrupt detection unit 104, a keyword extraction unit 105, a keyword search unit 106, and a display unit 107.
The voice input unit 101 is for inputting a voice of a speaker (hereinafter referred to as “speech voice”). The voice input unit 101 corresponds to, for example, a microphone, a communication interface with the network 400, or the like.

The utterance section determination unit 102 determines an utterance section for each speaker with respect to the input utterance voice. The utterance section refers to a section from the beginning to the end of the conversation by the speaker.
For example, when the conversation between the speaker A and the speaker B is as shown in FIG. 2 (a) or FIG. 2 (b), the speech segment determination unit 102 determines the end time te1 from the conversation start time ts1 of the speaker A. The section up to that time, that is, ts1-te1 is determined as the utterance section 1 of the speaker A. Furthermore, the utterance section determination unit 102 determines the section from the start time ts2 to the end time te2 of the conversation of the speaker B, that is, the section of ts2-te2 as the speech section 2 of the speaker B.

  Returning to FIG. 1, the voice recognition unit 103 recognizes the uttered voice of the determined utterance section for each speaker. More specifically, the voice recognition unit 103 converts the conversation voices of all the speakers into text characters using a known voice recognition technique. Furthermore, the voice recognition unit 103 associates the start time (start point) and the end time (end point) of the conversation voice of each speaker.

The interrupt detection unit 104 (speech response feature extraction unit) detects an interrupt in which the utterance feature, that is, the preceding utterance and the subsequent utterance overlap, based on the utterance voice of each utterer for the determined utterance period. For example, when the conversation between the speaker A and the speaker B is the conversation shown in FIG. 2B, the interrupt detection unit 104 follows the speaker B in the middle of the preceding speech of the speaker A, that is, at ts1. Since the utterance has been started, the interrupt is detected. This detection method is as follows.
That is, the interrupt detection unit 104 first measures a section (hereinafter referred to as an utterance interval) from the start time of the subsequent utterance to the end time of the immediately preceding utterance. For example, in the case of FIGS. 2A and 2B, the interrupt detection unit 104 calculates the speech interval using the calculation formula of utterance interval = ts2-te1 in FIGS. 2A and 2B. Next, the interrupt detection unit 104 determines whether the utterance interval becomes a negative value (see FIG. 2B) as a result of the above calculation. When the speech interval is a negative value (see FIG. 2B), the interrupt detection unit 104 detects that there is an interrupt.

  Based on the extracted utterance feature, that is, the interruption in which the preceding utterance and the succeeding utterance overlap, the keyword extraction unit 105 selects the utterance voice recognized by the voice recognition unit 102 within the conversation of the utterance voice. Extract a topic word (hereinafter referred to as a keyword). Specifically, the keyword extraction unit 105 acquires the conversation voice recognized by the voice recognition unit 102 from the voice recognition unit 102. This conversation voice is associated with the start time and end time of each speaker. In addition, the keyword extraction unit 105 is interrupted by the interrupt detection unit 104 and the utterance interval (for example, the utterance interval 2 of the speaker B in FIG. 2B) in which the interruption is detected by the interrupt detection unit 104. The utterance section (for example, the utterance section 1 of the speaker A in FIG. 2) is acquired. Each of these utterance sections is associated with a start time and an end time.

Furthermore, when extracting the said keyword, the keyword extraction part 105 extracts the last component (for example, noun) in the preceding utterance interrupted as a keyword, for example. Here, the end in the preceding utterance refers to the inside of the utterance section (for example, ts1-ts2 in FIG. 2B) before the interruption (for example, the time of ts2 in FIG. 2B).
Specifically, first, the keyword extracting unit 105 selects the utterance section (for example, the utterance section of the utterer with the earlier start time from the utterance sections of the respective utterers (for example, the utterance sections 1 and 2 in FIG. 2B). For example, the utterance section 1) in FIG. 2 (b) is selected. Next, in the selected utterance section (for example, utterance section 1 in FIG. 2B), the keyword extraction unit 105 starts the other acquired utterance section (that is, interrupt time, for example, FIG. 2B). The component (for example, noun) immediately before ts2) is detected. Next, the keyword extraction unit 105 extracts the detected constituents (for example, nouns) as keywords.

  The keyword search unit 106 performs a keyword search using the extracted keyword. Specifically, first, the keyword search unit 106 connects to the search server 300 via the network 400. Then, the search server 300 receives the keyword search request from the keyword search unit 106, and returns the search result of the keyword search to the keyword search unit 106 of the keyword extracting device 100 via the network 400. The keyword search unit 106 receives the search result of the keyword search from the search server 300 by the return.

  The display unit 107 displays the search result by the keyword search unit 106, that is, the search result of the search server 300. The display unit 107 is a display device such as a display or a display panel.

  In the present embodiment, the speech segment determination unit 102, the speech recognition unit 103, the interrupt detection unit 104, the keyword extraction unit 105, and the keyword search unit 106 correspond to a processing device such as a CPU. In addition, the keyword extraction device 100 is assumed to have a known configuration including a storage device (not shown) such as a memory.

Next, the operation of the keyword extracting device 100 will be described with reference to FIG. In FIG. 3, for example, it is assumed that two speakers A and B are having a conversation using the keyword extraction device 100 and the information terminal 200.
First, the keyword extraction device 100 (the utterance section determination unit 102) determines an utterance section for each speaker with respect to the utterance voices input from the voice input unit 100 and the information terminal 200 (step S101). At the time of this determination, the utterance section determination unit 102 determines whether or not the volume of the uttered voice of each speaker is equal to or greater than a threshold value, and determines a section that is equal to or greater than the threshold value as the utterance section.
For example, when the conversation between the speaker A and the speaker B is as shown in FIG. 2 (a) or FIG. 2 (b), the speech segment determination unit 102 determines the end time te1 from the conversation start time ts1 of the speaker A. The section up to that time, that is, ts1-te2 is determined as the utterance section 1 of the speaker A. Further, the utterance section determination unit 103 determines the section from the start time ts2 to the end time te2 of the conversation of the speaker B, and the section ts2-te2 as the speech section 2 of the speaker B.

  Next, the keyword extraction device 100 (voice recognition unit 103) recognizes the utterance voice of the determined utterance section for each speaker (step S102). This recognition shall be performed by the feature analysis by a frequency band, for example. Furthermore, when performing the above recognition, the speech recognition unit 103 converts the conversation speech of all the speakers into text characters using a known speech recognition technique.

  Next, the keyword extraction device 100 (interrupt detection unit 104) detects an interrupt based on the determined speech period (step S103). Specifically, the interrupt detection unit 104 subtracts the end time of the immediately preceding utterance from the start time of the subsequent utterance, that is, the utterance interval (for example, te1-ts2 in FIGS. 2A and 2B). ). As a result of this calculation, if the value of the speech interval (for example, te1-ts2 = speech interval in FIG. 2B) is negative, the interrupt detection unit 104 determines that there is an interruption of the subsequent speech. .

Next, the keyword extraction device 100 (keyword extraction unit 105) extracts and determines the keywords in the detected voice conversation (interactive voice recognition recognized in step S102) with the interruption (step S104). Specifically, the keyword extraction unit 105 extracts a noun in the preceding utterance immediately before the subsequent utterance, and determines this noun as a keyword in the utterance.
For example, when utterer A starts to speak “This time, New Tokyo Tower ...” at the time ts1 in FIG. 2B, the utterer B is “oh” at the time ts2 in FIG. When the conversation begins, “Where is it possible?”, The keyword extraction unit 105 determines the noun “Shin Tokyo Tower” of the speaker A immediately before ts2 as a keyword. As a result, the keyword extraction unit 105 determines “New Tokyo Tower” as a topic in the conversation without extracting the keyword of “New Tokyo Tower” from the database in which keywords predicted in advance are registered. be able to.

  Note that when the utterance interval shows a positive value (see FIG. 2A), the keyword extraction unit 105 determines that there is no keyword being uttered, and does not extract the keyword.

  Next, the keyword extraction device 100 (keyword search unit 106) performs keyword search for the determined keyword (step S105). Specifically, the keyword search unit 106 first requests the search server 300 for the keyword search via the network 400. Then, the search server 300 performs a keyword search in response to the request, and transmits the search result to the keyword search unit 106. Next, the keyword search unit 106 receives the search result transmitted from the search server 300.

  Next, the keyword search unit 106 displays the received search result on the display unit 107 (step S106). Thereby, the speaker can grasp the information (search result) related to the keyword (for example, New Tokyo Tower) in the conversation.

  Also, in place of the interrupt detection unit 104, operating a silence detection unit that detects silence whose utterance interval is greater than or equal to a predetermined threshold (for example, 3 seconds) can also be characterized by an utterance response that suggests the presence of a keyword. Useful for extraction.

  As described above, according to the present embodiment, the keyword extraction device 100 detects an interrupt as a feature of an utterance response that suggests the presence of a keyword, and extracts a keyword in the conversation. For this reason, the keyword extraction apparatus 100 can extract a keyword in a conversation based on the presence or absence of a speaker's interruption without predicting the keyword in the conversation in advance and registering it in a database or the like.

  In the first embodiment, the keyword extracting apparatus 100 has been described with respect to the case where the processes of steps S101 to S106 in FIG. 3 are sequentially performed, but the present invention is not limited to this. For example, the keyword extraction device 100 may execute the steps in FIG. 3 by changing the order of the steps, or may execute the steps in parallel.

(Embodiment 2)
The keyword extracting apparatus according to the second embodiment extracts keywords in a conversation based on a pitch (sound pitch) pattern that is a feature of an utterance response.
FIG. 4 is a block diagram illustrating a configuration example of the keyword extracting device according to the second embodiment of the present invention. In the second embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals and terms as those in the first embodiment, and redundant description is omitted.
In FIG. 4, the keyword extraction device 100 </ b> A includes a pitch determination unit 201 and a pitch pattern determination unit 202 instead of the interrupt detection unit 104 of the first embodiment in FIG. 1. Furthermore, the keyword extraction device 100A is different from the first embodiment in that it includes a keyword extraction unit 105A instead of the keyword extraction unit 105 of the first embodiment in FIG. The pitch determination unit 201, the pitch pattern determination unit 202, and the keyword extraction unit 105A are processing devices such as a CPU. In addition, the configuration of the entire system including the information terminal 200 is the same as that of FIG.

  The pitch determination unit 201 and the pitch pattern determination unit 202 (also collectively referred to as an utterance response feature extraction unit) perform the utterance of the utterance interval determined by the utterance interval determination unit 102 based on the utterance speech of each speaker. A pitch pattern that is a feature is extracted. Specifically, the pitch determination unit 201 determines the pitch of the speech voice. The pitch determination unit 201 of the present embodiment determines the pitch by dividing the uttered speech every 10 ms, for example.

Based on the determined pitch, the pitch pattern determination unit 202 has a descending pitch at the end of the preceding utterance (see tc1-te1 in FIG. 5), and a succeeding utterance immediately after the preceding utterance is an ascending pitch ( The pitch pattern (characteristic of the utterance) to be determined (see between tc2 and te2 in FIG. 5) is determined. An example of this determination is shown in FIG. The horizontal axis in FIG. 5 represents time, and the vertical axis represents frequency.
In the utterance section ts1-te1, there is a preceding utterance “New Tokyo Tower”, and in the utterance section ts2-te2, there is a subsequent utterance “Is that? A descending pitch is determined at the end of the preceding utterance of “New Tokyo Tower”, and an ascending pitch is determined for the subsequent utterance of “Is that ...?”. This determination is made because the pitch pattern determination unit 202 determines as follows.

  That is, the pitch pattern determination unit 202 has a higher frequency f at the end (at the end) of the utterance section than the frequency f at the midpoint tc1 in the utterance section ts1-te1 of “New Tokyo Tower is” in FIG. This is because it is determined that the pitch is rising. In addition, the pitch pattern determination unit 202 decreases in the utterance interval ts2-te2 of “What?” In FIG. 5 because the frequency f at the end (at the end) of the utterance interval is lower than the frequency f at the midpoint tc2. This is because the pitch is determined.

  In addition, although the pitch pattern determination part 202 of this Embodiment demonstrates the case where a raise pitch or a fall pitch is determined on the basis of the frequency of the middle point of an utterance area, it is not restricted to this. For example, the pitch determination unit 201 may perform determination based on a time point that is a predetermined period (for example, time T) from the end of the utterance period (for example, te1 and te2 in FIG. 5).

  The keyword extraction unit 105A extracts keywords from the preceding utterances shown in the determined pitch pattern. In this extraction, the keyword extraction unit 105A extracts, for example, the last constituent (for example, a noun) in the preceding utterance shown in the pitch pattern as a keyword.

  Next, the operation of the keyword extracting device 100A will be described with reference to FIG. In FIG. 6, for example, the speaker A uses the keyword extraction device 100A to say “Now, Tokyo Tower ...”, and then the speaker B uses the information terminal 200 to say “ Explain on the premise of saying "...?" 7 are the same as steps S101 to S102 and S105 to S106 in FIG. 3, and will be omitted as appropriate.

  First, the keyword extraction device 100A (the utterance section determination unit 102) uses the utterance section for each utterer (the utterance section 1 in FIG. 2A, FIG. 2) for the utterance speech input from the voice input section 100 and the information terminal 200. (See utterance section 2 in (b)) (step S101). Next, the keyword extraction device 100A (voice recognition unit 103) recognizes the utterance voice of the determined utterance section for each speaker (step S102).

  Next, the keyword extraction device 100A (pitch determination unit 201), for example, the utterance section 1 of the preceding utterance of the speaker A (see FIG. 2A) and the utterance section 2 of the subsequent utterance of the speaker B (FIG. 2 ( Based on the uttered voice of (b), the pitch of the uttered voice is determined (step S103A).

  Next, the keyword extraction device 100A (pitch pattern determination unit 202) determines whether there is a pitch pattern that changes from the descending pitch to the ascending pitch when the preceding utterance is shifted to the succeeding utterance based on the determined pitch. Determination is made (step S103B). Specifically, the pitch pattern determination unit 202 determines that the end of the preceding utterance is the descending pitch (see tc1-te1 in FIG. 5), and the succeeding utterance immediately after the preceding utterance is the ascending pitch (tc2 in FIG. 5). The pitch pattern is determined.

  Next, the keyword extraction device 100A (keyword extraction unit 105A) determines the preceding utterance of the utterance voice (recognized in step S102) indicated in the determined pitch pattern (for example, “New Tokyo Tower ]), Keywords are extracted (step S104A). In this extraction, the keyword extraction unit 105A extracts, for example, “Shin Tokyo Tower”, which is the last noun in the preceding utterance shown in the pitch pattern, as a keyword.

  Next, the keyword extraction device 100A (keyword search unit 106) performs keyword search for the determined keyword on the search server 300 via the network 400 (step S105). Next, the keyword search unit 106 displays the received search result on the display unit 107 (step S106). Thereby, the speaker can grasp information (search result) related to the topic word (for example, New Tokyo Tower).

  As described above, according to the present embodiment, keyword extracting apparatus 100A determines a pitch pattern that is a feature of an utterance response that suggests the presence of a keyword, and extracts a keyword in a conversation. For this reason, the keyword extraction device 100A can extract keywords in a conversation from the presence or absence of a pitch pattern without preparing in advance for registering the keywords used in the conversation in a database or the like. .

  In the second embodiment, the keyword extraction apparatus 100A has been described with respect to the case where the processes of steps S101 to S102, S103A to S103B, S104A, and S105 to S106 in FIG. 7 are sequentially performed, but the present invention is not limited to this. For example, the keyword extracting device 100A may execute the steps in FIG. 7 by changing the order of the steps, or may execute the steps in parallel.

(Embodiment 3)
The keyword extraction device according to the third embodiment extracts keywords in a conversation based on a function phrase that is a feature of an utterance response.
FIG. 7 is a block diagram illustrating a configuration example of the keyword extraction device according to Embodiment 3 of the present invention. In the third embodiment, the same reference numerals and terms as those in the first embodiment are assigned to the same parts as those in the first embodiment, and the duplicate description is omitted.
In FIG. 7, the keyword extraction device 100B has a function phrase extraction unit 301 (utterance response feature extraction unit) instead of the interrupt detection unit 104 of the first embodiment in FIG. Furthermore, the keyword extraction device 100B includes a function phrase storage unit 302. Further, the keyword extraction device 100B is different from the first embodiment in that it includes a keyword extraction unit 105B instead of the keyword extraction unit 105 of the first embodiment in FIG. The function phrase extraction unit 301 is a processing device such as a CPU, and the function phrase storage unit 302 is a storage device such as a memory. In addition, the configuration of the entire system including the information terminal 200 is the same as that of FIG.

  The function phrase storage unit 302 stores a predetermined function phrase. This function phrase is a word representing the type of response, and is used in common with conversations regardless of various different conversation contents. For example, as a functional phrase, a question sentence such as “Is it?”, An agreement sentence such as “Like” or “I see” or “It is”, a negative sentence such as “No”, a request sentence such as “Please” Exclamation sentences such as “Oh” and indentation sentences such as “Nadeyanen” are applicable.

  The functional phrase extraction unit 301 extracts the functional phrase that is a feature of the uttered voice from the uttered voice. Specifically, the functional phrase extraction unit 301 compares a word included in the speech to be extracted with a functional phrase in the functional phrase storage unit 302, and extracts a functional phrase included in the speech.

  Next, the operation of the keyword extracting device 100B will be described with reference to FIG. In FIG. 8, for example, after the speaker A uses the keyword extraction device 100 </ b> B to say “This time, New Tokyo Tower can be created.”, The speaker B uses the information terminal 200 to say “Oh, it "Where can I do that?" Note that the processes in steps S101 to S102 and S105 to S106 in FIG. 8 are the same as the processes in steps S101 to S102 and S105 to S106 in FIG.

  First, the keyword extraction device 100B (the utterance section determination unit 102) uses the utterance section for each utterer (the utterance section 1 in FIG. 2A, FIG. 2) for the speech input from the voice input section 100 and the information terminal 200. (See utterance section 2 in (b)) (step S101). Next, the keyword extraction device 100B (voice recognition unit 103) recognizes the utterance voice in the determined utterance section for each speaker (step S102).

  Next, the keyword extraction device 100B (functional phrase extraction unit 301), for example, the utterance section 1 of the preceding utterance of the speaker A (see FIG. 2A) and the utterance section 2 of the subsequent utterance of the speaker B (FIG. 2). A functional phrase representing a question sentence or the like is extracted from the uttered voice of (b). Specifically, the functional phrase extraction unit 301 compares a sequence of words included in the uttered speech to be extracted with the functional phrase in the functional phrase storage unit 302, and extracts a functional phrase included in the uttered speech. To do. In the present embodiment, the functional phrase extraction unit 301 extracts the functional phrase of the question sentence “Is it?” From the utterance voice of “Oh, where can I do it?”. Here, the speech recognition result may be used for the word sequence included in the speech voice.

  Next, the keyword extraction device 100B (keyword extraction unit 105B) extracts a keyword from the utterance voice immediately before the utterance including the extracted functional phrase (recognized in step S102) (step S104B). When extracting this keyword, the keyword extraction unit 105B, for example, from the previous utterance "Now, you can make New Tokyo Tower." Extract as keywords.

  Next, the keyword extraction device 100B (keyword search unit 106) performs keyword search for the extracted keyword on the search server 300 via the network 400 (step S105). Next, the keyword search unit 106 displays the received search result on the display unit 107 (step S106). Thereby, the speaker can grasp information (search result) related to a keyword (for example, New Tokyo Tower) which is a topic in the conversation.

  In addition, according to the present embodiment, the question from the previous utterance, such as when the speaker A asks "What is that?" And the speaker B answers "What is the New Tokyo Tower?" When the functional phrase of the sentence (“What was it?”) Is extracted, the keyword extraction unit 105B can be operated so as to extract the keyword (“New Tokyo Tower”) from the subsequent utterance. is there. At this time, whether the keyword is extracted from the immediately preceding utterance speech or the keyword is extracted from the immediately following utterance speech can be switched as follows. That is, it can be used by switching from the immediately preceding utterance when the pronoun pronoun “it” is included, from the immediately following utterance when including the indicating pronoun “that”, and from the immediately following utterance in other cases. At that time, the feature of the utterance response may be captured by using (using in combination) a pitch pattern in which the preceding utterance is the rising pitch and the subsequent utterance is the descending pitch in the same manner as in the second embodiment.

  As described above, according to the present embodiment, the keyword extraction device 100B extracts functional phrases (question sentences, etc.) that are commonly used regardless of the conversation content (genre), and extracts keywords in the conversation. To do. For this reason, in the keyword extracting device 100B, it is possible to extract a keyword by extracting a commonly used function phrase from the conversation sentence. Therefore, the keyword extraction device 100B is useful because it can extract keywords without preparing in advance a keyword corresponding to each genre conversation and registering it in a database or the like.

  In the third embodiment, the keyword extraction apparatus 100B has been described with respect to the case where the processes of steps S101 to S102, S103C, S104B, and S105 to S106 in FIG. 8 are sequentially performed. However, the present invention is not limited to this. For example, the keyword extracting device 100B may execute the steps in FIG. 9 by changing the order of the steps, or may execute the steps in parallel.

(Embodiment 4)
The keyword extraction device according to the fourth embodiment extracts keywords in a conversation based on changes in the facial expression of a person who has heard spoken speech.
FIG. 9 is a block diagram illustrating a configuration example of the keyword extracting device according to the fourth embodiment of the present invention. In the fourth embodiment, the same reference numerals and terms as those in the first embodiment are assigned to the same parts as those in the first embodiment, and the duplicate description is omitted.

  In FIG. 9, the keyword extraction device 100 </ b> C includes a video input unit 401 and a facial expression recognition unit 402 (also collectively referred to as an utterance response feature extraction unit) instead of the interrupt detection unit 104 of the first embodiment in FIG. 1. Have. Further, the keyword extraction apparatus 100C is different from the first embodiment in that it includes a keyword extraction unit 105C instead of the keyword extraction unit 105 of the first embodiment in FIG. The image input unit 401 is a camera, and the facial expression recognition unit 402 is a processing device such as a CPU. In addition, the configuration of the entire system including the information terminal 200 is the same as that of FIG.

The video input unit 401 is for inputting image data including a user's face portion. When the facial expression recognition unit 402 converts the image data into digital original image data that can be used to estimate the facial expression of the user, the facial expression recognition unit 402 extracts the user's face area included in the original image data, and extracts the user's face area from the extracted face area. The contour position of at least one facial organ such as eyes and mouth constituting the face is extracted. Then, the facial expression recognition unit 402 extracts the contours of the upper and lower ends of the facial organ acquired over a plurality of video frames, and determines the facial expression of the user (for example, neutrality, surprise) , Joy, anger, etc.).
At that time, the facial expression recognition unit 402 associates the time in the utterance section for each speaker obtained from the utterance section determination unit 102 with the recognition result of the facial expression of a person other than the speaker. Further, the facial expression recognition unit 402 extracts facial expression change points from the facial expression recognition result.
For example, in FIG. 10, t10 is the utterance start time of the utterance section 1 by the speaker A, t11 and t12 are equally spaced times following t10, t20 is the utterance start time of the utterance section 2 by the speaker B, t21, t22 is an equally spaced time following t20. Here, the facial expression recognition unit 402 recognizes the facial expression of the speaker B at each of the times t10, t11, and t12 and the facial expression of the speaker A at each of the times t20, t21, and t22. In this example, the expression of the speaker B at time t11 is a surprised expression, and at other times, the expression is neutral regardless of the speaker. That is, the facial expression recognition unit 402 extracts time t11 as a facial expression change point.

  The keyword extraction unit 105C changes the facial expression when the facial expression recognition unit 402 recognizes that the recognized facial expression is a neutral facial expression at the start of the utterance and changes to another facial expression during the utterance. A word uttered at the time corresponding to the point is extracted as a keyword. At that time, the keyword extraction unit 105C may obtain the word at the time corresponding to the facial expression from the section information for each word in the speech recognition result, or may estimate it from the number of syllables included in the uttered speech. The corresponding time here refers to the time from when a word is perceived until the reaction appears in the facial expression (for example, 0.1 seconds), and the end of the word and the expression of the facial expression are associated with each other. It's time.

  Next, the operation of the keyword extracting device 100C will be described with reference to FIG. In FIG. 11, for example, the speaker A uses the keyword extraction device 100C to say “New Tokyo Tower can be done next”, and then the speaker B uses the information terminal 200 to say “What is that? ”On the assumption that“ Note that the processes in steps S101 to S102 and S105 to S106 in FIG. 11 are the same processes as steps S101 to S102 and S105 to S106 in FIG. The voice and video of the speaker B are input using the information terminal 200. For the sake of convenience, the input will be described as being input from the audio input unit 101 and the video input unit 401 in the same manner as the speaker A. .

  First, the keyword extraction device 100C (the utterance section determination unit 102) determines the utterance section for each speaker (see the utterance section 1 and the utterance section 2 in FIG. 10) for the uttered speech input from the speech input section 101 (see FIG. 10). Step S101). Next, the keyword extracting device 100C (voice recognition unit 103) recognizes the uttered voice of the determined utterance section for each speaker (step S102).

  On the other hand, the keyword extraction device 100C (the video input unit 401 and the facial expression recognition unit 402), for example, of the speaker B at the time corresponding to the utterance voice (see FIG. 10) in the utterance section 1 which is the preceding utterance uttered by the speaker A. The facial expression is recognized, and the facial expression of the speaker A at the time corresponding to the utterance voice (see FIG. 10) in the utterance section 2 which is the subsequent utterance uttered by the speaker B is recognized. That is, rather than recognizing the facial expression of the speaker, the facial expression of the person who is listening to the speech, that is, the facial expression of another speaker relative to the speech of the speaker is recognized (step S103D).

  Next, the keyword extraction device 100A (keyword extraction unit 105C) recognizes that the recognized facial expression is a neutral facial expression at the start of utterance and has changed to another facial expression during the utterance. A word uttered at the time corresponding to the facial expression change point is extracted as a keyword (step S104C). In the above example, “New Tokyo Tower” is extracted as a word corresponding to the time when the expression changes from neutral to a surprising expression.

  Next, the keyword extraction device 100C (keyword search unit 106) performs keyword search for the determined keyword to the search server 300 via the network 400 (step S105). Next, the keyword search unit 106 displays the received search result on the display unit 107 (step S106). Thereby, the speaker can grasp information (search result) related to the topic word (for example, New Tokyo Tower).

  As described above, according to the present embodiment, the keyword extraction device 100C extracts keywords in a conversation based on the recognition result of the facial expression of another person who is listening to the uttered voice. For this reason, in the keyword extraction device 100C, the keyword in the conversation is obtained from the feature of the utterance response that is captured as a change in facial expression without preparing the keyword used in the conversation in advance and registering it in the database or the like. Can be extracted.

  It should be noted that the same effect can be obtained by converting the expression of the eyes and the degree of opening of the mouth into numerical values instead of the facial expression recognition unit 402, and detecting changes in facial expressions based only on the magnitudes of those changes.

  In the fourth embodiment, the keyword extraction device 100C has been described with respect to the case where the processes of steps S101 to S102, S103D, S104C, and S105 to S106 in FIG. 11 are sequentially performed. However, the present invention is not limited to this. For example, the keyword extraction device 100C may execute the steps in FIG. 11 by changing the order of the steps, or may execute the steps in parallel.

(Embodiment 5)
The keyword extraction device according to the fifth embodiment extracts keywords in a conversation based on an excitement reaction of a person who has heard an uttered voice.
FIG. 12 is a block diagram illustrating a configuration example of the keyword extracting device according to the fifth embodiment of the present invention. In the fifth embodiment, the same parts as those in the first embodiment are denoted by the same reference numerals and terms as those in the first embodiment, and redundant description is omitted.

  In FIG. 12, the keyword extraction device 100D includes a climax reaction detection unit 501 (also referred to as an utterance response feature extraction unit) instead of the interrupt detection unit 104 of the first embodiment in FIG. Further, the keyword extraction device 100D is different from the first embodiment in that it includes a keyword extraction unit 105D instead of the keyword extraction unit 105 of the first embodiment in FIG. The swell reaction detection unit 501 is a processing device such as a CPU. In addition, the configuration of the entire system including the information terminal 200 is the same as that of FIG.

The swell response detector 501 detects a sway response from voice and sound. Specifically, a swell response is detected by detecting a laughing voice, detecting a voice with a high degree of excitement, or detecting a sound of clapping or kneeling. For the laughing voice, applause, and kneeling sound, the climax reaction detection unit 501 facilitates a learning sample in advance, creates a GMM (gamma mixture model), obtains a likelihood for the input, and performs threshold processing. To detect. The excitement reaction detection unit 501 linearly combines amounts obtained by normalizing the loudness level, the pitch height, and the speaking speed with the average value of the speaker for a highly excited sound. It is detected by digitizing and threshold processing.
At this time, the climax reaction detection unit 501 regards the climax reaction detected near the end of the utterance interval determined by the utterance interval determination unit 102 as an excitement response corresponding to the utterance.

  The keyword detection unit 105D extracts a keyword from the utterance corresponding to the excitement reaction.

  Next, the operation of the keyword extracting device 100D will be described with reference to FIG. In FIG. 13, for example, the speaker A uses the keyword extraction device 100 </ b> C to say “Now, Tokyo Tower ...”, and then the speaker B uses the information terminal 200 to say “Ahaha”. I will explain on the assumption that I laughed. The processes in steps S101 to S102 and S105 to S106 in FIG. 13 are the same processes as steps S101 to S102 and S105 to S106 in FIG.

  First, the keyword extraction device 100D (the utterance section determination unit 102) determines an utterance section for each speaker from the speech input from the voice input unit 101 and the information terminal 200 (step S101). Next, the keyword extraction device 100D (voice recognition unit 103) recognizes the utterance voice in the determined utterance section for each speaker (step S102).

  Next, the keyword extraction device 100D (exciting reaction detection 501) detects the presence of an enlarging reaction, for example, in the vicinity of the utterance section uttered by the speaker A (step S103E). As a result, in the above-described utterance example, the laughter voice GMM is collated with high likelihood immediately after the utterance section of the speaker A, so that it is detected as an excitement reaction.

  Next, the keyword extraction device 100A (keyword extraction unit 105D) extracts, as a keyword, a word (for example, “New Tokyo Tower”) uttered in the utterance section corresponding to the excitement reaction (step S104D).

  Next, the keyword extraction device 100D (keyword search unit 106) performs keyword search of the determined keyword to the search server 300 via the network 400 (step S105). Next, the keyword search unit 106 displays the received search result on the display unit 107 (step S106). Thereby, the speaker can grasp information (search result) related to the topic word (for example, New Tokyo Tower).

  As described above, according to the present embodiment, the keyword extracting device 100D detects the excitement reaction of the person who has heard the uttered voice and extracts the keywords in the conversation. For this reason, in the keyword extraction apparatus 100D, it is possible to predict the keyword used in the conversation in advance from the characteristics of the utterance response that can be regarded as a swell and applause without preparing to register it in a database or the like. The keywords in can be extracted.

  In the fifth embodiment, the keyword extraction device 100D has been described with respect to the case where the processes of steps S101 to S102, S103E, S104D, and S105 to S106 in FIG. 13 are sequentially performed. However, the present invention is not limited to this. For example, the keyword extracting device 100D may execute the steps in FIG. 13 by changing the order of the steps, or may execute the steps in parallel.

  In Embodiments 1 to 3 and 5, the keyword extraction device (keyword extraction unit) has described the case where the noun at the end (immediately before interruption) in the utterance section is extracted as a keyword. However, the present invention is not limited to this. For example, the keyword extraction unit may search for a noun that is conceptually lowest among a plurality of nouns included in the preceding utterance to be searched as a keyword. In this case, the keyword extraction device further includes a dictionary information storage unit (not shown) such as a memory, and the dictionary information storage unit includes a conceptual upper level (for example, Italian cuisine) and a conceptual lower level (for example, pasta). ) Is used to store dictionary information organized and organized. Then, the keyword extraction unit extracts, as keywords, the conceptually lowest nouns included in the dictionary information of the dictionary information storage unit (not shown) from the nouns included in the utterance to be extracted. Thereby, the noun of a low-order concept is extracted as a keyword.

  In Embodiments 1 to 3 and 5, the keyword extraction unit may extract the noun with the highest pitch among the nouns included in the utterance to be extracted as the keyword, or the most frequently used. You may make it extract a noun as a keyword. Alternatively, the keyword extraction unit uses, as a keyword, a noun in which a combination of various parameters indicating the pitch and the number of times of use of each noun is optimal (a predetermined parameter pattern) from the nouns included in the utterance to be extracted. You may make it extract.

Although the present invention has been described in detail and with reference to specific embodiments, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the invention.
This application is based on a Japanese patent application filed on March 29, 2007 (Japanese Patent Application No. 2007-088321), the contents of which are incorporated herein by reference.

  The keyword extracting device of the present invention is useful for extracting important keywords included in a conversation. The keyword extraction device can be applied to uses such as a telephone, an in-vehicle terminal, a television, a conference system, a call center system, and a personal computer.

1 is a block diagram showing a configuration example of an entire system including a keyword extraction device in Embodiment 1 of the present invention. The figure which shows the example of the speech area in Embodiment 1 of this invention. The flowchart which shows operation | movement of the keyword extraction apparatus of FIG. The block diagram which shows the structural example of the keyword extraction apparatus in Embodiment 2 of this invention. The figure which shows the example of the pitch pattern in Embodiment 2 of this invention. 5 is a flowchart showing the operation of the keyword extraction device in FIG. 4. The block diagram which shows the structural example of the keyword extraction apparatus in Embodiment 3 of this invention. The flowchart which shows operation | movement of the keyword extraction apparatus of FIG. The block diagram which shows the structural example of the keyword extraction apparatus in Embodiment 4 of this invention. The figure which shows the example of the speech area in the Embodiment 4 of this invention, speech content, and a facial expression recognition result. 10 is a flowchart showing the operation of the keyword extraction device of FIG. The block diagram which shows the structural example of the keyword extraction apparatus in Embodiment 5 of this invention. The flowchart which shows operation | movement of the keyword extraction apparatus of FIG.

Explanation of symbols

100, 100A, 100B, 100C, 100D Keyword extraction device 101 Voice input unit 102 Speech segment determination unit 103 Speech recognition unit 104 Interrupt detection unit 105, 105A, 105B, 105C, 105D Keyword extraction unit 106 Keyword search unit 107 Display unit 201 Pitch determination unit 202 Pitch pattern determination unit 301 Function phrase extraction unit 302 Function phrase storage unit 401 Video input unit 402 Expression recognition unit 501 Swell reaction detection unit

Claims (8)

A voice input unit for inputting the voice of the speaker,
For the input speech voice, an utterance interval determination unit that determines an utterance interval for each speaker,
A speech recognition unit for recognizing the speech of the determined speech section for each speaker;
An utterance response feature extraction unit that extracts the feature of an utterance response that suggests the presence of a keyword based on the response of another utterer to the utterance voice of each utterer;
A keyword extraction unit for extracting the keyword from the utterance voice of the utterance section identified based on the extracted utterance response characteristics;
Keyword extractor including The utterance voice of each utterer includes the utterance voice of the preceding utterance and the utterance voice of the subsequent utterance,
The utterance response feature extraction unit determines whether the preceding utterance and the following utterance are generated when the following utterance is started in the middle of the preceding utterance based on the speech of the preceding utterance and the following utterance. It consists of an interrupt detection unit that detects overlapping interrupts,
The keyword extraction unit extracts the keyword from the utterance speech of the preceding utterance that overlaps with the subsequent utterance specified based on the detected interruption.
The keyword extraction device according to claim 1. The utterance voice of each utterer includes the utterance voice of the preceding utterance and the utterance voice of the subsequent utterance,
The utterance response feature extraction unit includes:
A pitch determination unit that determines the pitch of the uttered voice based on the uttered voice of the preceding utterance and the subsequent utterance;
A pattern determination unit that determines a pitch pattern in which the trailing utterance at the end of the preceding utterance is a descending pitch and the succeeding utterance immediately after the preceding utterance is an ascending pitch based on the determined pitch;
The keyword extracting unit extracts the keyword from the utterance speech of the preceding utterance indicated in the pitch pattern, identified based on the determined pitch pattern.
The keyword extraction device according to claim 1. The utterance voice of each utterer includes the utterance voice of the preceding utterance and the utterance voice of the subsequent utterance,
The utterance response feature extraction unit extracts a function phrase of a predetermined type from the utterance voice of the subsequent utterance based on the utterance voice of the preceding utterance and the subsequent utterance,
The keyword extraction unit extracts the keyword from the utterance voice of the preceding utterance immediately before the subsequent utterance including the extracted function phrase.
The keyword extraction device according to claim 1. The utterance response feature extraction unit detects an excitement reaction of a person other than the speaker in the vicinity of the utterance section for each speaker,
The keyword extraction unit extracts the keyword from the utterance voice corresponding to the excitement reaction;
The keyword extraction device according to claim 1. The keyword extraction unit, when extracting the keyword, extracts a last constituent in the preceding utterance as the keyword;
The keyword extracting device according to any one of claims 2 to 5. The utterance voice of each utterer includes the utterance voice of the preceding utterance and the utterance voice of the subsequent utterance,
The utterance response feature extraction unit extracts a predetermined type of functional phrase from the utterance speech of the preceding utterance based on the utterance speech of the preceding utterance and the subsequent utterance,
The keyword extraction unit extracts the keyword from the utterance voice of the subsequent utterance immediately after the preceding utterance including the extracted functional phrase.
The keyword extraction device according to claim 1. The utterance response feature extraction unit recognizes facial expressions of other utterers with respect to the uttered voices of the respective speakers, and extracts change points of the recognized facial expressions,
The keyword extraction unit extracts a constituent element in the utterance interval corresponding to the extracted facial expression change point as a keyword;
The keyword extraction device according to claim 1.

Images