JP2009064186A - Interactive system for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an interactive system for a vehicle, in which proper interaction control on which characters such as the sensitivity or tastes of a user are reflected, is performed. <P>SOLUTION: A detection means detects the occupant of a vehicle. A discrimination means discriminates the taste of the occupant based on the speech of the detected occupant. A determination means determines interaction content based on the taste of the occupant decided by the discrimination means. A control means controls interaction with the occupant based on the interaction content determined by the determination means. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an interactive interactive system that uses car navigation in a vehicle as a task.

  In recent years, the use of a dialogue system that achieves a task through dialogue between a human and a machine has been advanced. In particular, the voice interaction system is expected to be applied to tasks such as car navigation for those who cannot keep their hands like car drivers.

JP 2006-195637 A

  In a dialog system, it is important to select a dialog strategy or control dialog to efficiently accomplish a task. In particular, it is very difficult to select or control an appropriate dialogue strategy in an application whose purpose changes variously depending on the user or depending on the time and place, such as car navigation. In a uniform dialogue strategy, satisfaction is often high for one user but low for another user.

  Therefore, there is a need for a dialogue system in which appropriate dialogue control that reflects characters such as user sensitivity and preferences is performed.

  According to one aspect of the present invention, detection means for detecting an occupant of a vehicle, determination means for determining a passenger's preference based on an utterance of the occupant detected by the detection means, and determination by the determination means A vehicle dialogue comprising: a determination unit that determines a conversation content based on the passenger's preference; and a control unit that controls a conversation with the passenger based on the conversation content determined by the determination unit. A system is provided.

  According to this configuration, it is possible to realize dialogue processing that reflects the passenger's preference.

  According to a preferred embodiment of the present invention, the speech recognition means for recognizing the utterance of the passenger, the speech recognized by the speech recognition means is converted into text, and the grammatical dependency between the morphemes constituting the text It is preferable to further include dependency analysis means for analyzing the relationship, and the determination means determines the passenger's preference based on the dependency relationship analyzed by the dependency analysis means.

  According to this structure, a passenger | crew's preference can be determined with high precision.

  According to a preferred embodiment of the present invention, the image processing apparatus further comprises face image recognition means for recognizing the face image of the occupant, wherein the determination means is based on the face image of the occupant recognized by the face image recognition means. It is preferable to determine the preference.

  According to this structure, a passenger | crew's preference can be determined with high precision.

  According to a preferred embodiment of the present invention, there is further provided a preference database for accumulating vocabulary relating to the passenger's preference determined by the determination means, and a topic database for accumulating topic data collected via a network, It is preferable that the determination unit determines the conversation contents by extracting a vocabulary corresponding to the determination result of the determination unit from the preference database and extracting a topic related to the extracted vocabulary from the topic database.

  According to this configuration, a wide variety of topics can be provided to the occupant.

  According to a preferred embodiment of the present invention, the apparatus further comprises a forming unit that generates a self-organizing map of the preference database and forms an occupant utterance content cluster based on the self-organizing map, and the determining unit includes: It is preferable to determine the conversation content based on the utterance content cluster.

  According to this configuration, a wide variety of topics can be provided to the occupant.

  According to a preferred embodiment of the present invention, the preference database and the topic database are preferably configured to store data transferred from an external device via a network.

  According to this configuration, a wide variety of topics can be provided to the occupant.

  According to a preferred embodiment of the present invention, it is preferable to further include a hierarchical interactive content database in which interactive content is classified and stored hierarchically according to priority. Moreover, it is preferable that the said determination means determines the said priority based on the said passenger | crew's preference which the said determination means determined.

  According to this configuration, it is possible to provide a dialogue that meets the needs of the occupant.

  According to a preferred embodiment of the present invention, it is preferable that each of the topic database and the preference database is associated with a target facility provided by a navigation system.

  According to this configuration, it is possible to select a destination facility that suits the passenger's preference and topic via the navigation system.

  Further, the topic database is associated with a roadside facility provided by the navigation system, and the determining means is based on the topic database and the preference database, and the dialogue content regarding the roadside facility according to the position of the vehicle. Is preferably determined.

  According to this configuration, it is possible to interact with facility information around the vehicle.

  ADVANTAGE OF THE INVENTION According to this invention, the dialogue system in which appropriate dialogue control reflecting characters, such as a user's sensitivity and liking, is performed is provided.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to the following embodiment, It shows only the specific example advantageous for implementation of this invention. In addition, not all combinations of features described in the following embodiments are indispensable as means for solving the problems of the present invention.

  FIG. 1 is a block diagram showing a configuration of a vehicle dialogue system in the present embodiment.

  The vehicle dialogue system basically includes a service center 100 that provides services to the vehicle, and an in-vehicle unit 200 that is mounted on the vehicle and communicates with the service center 100 via the network 10.

  The service center 100 includes a server 101, a user determination database (DB) 102, a preference determination DB 103, a user preference DB 104, a topic DB 105, and a hierarchical interactive content DB 106. In the present embodiment, as shown in the figure, these are communicably connected via the network 10, but each DB may be connected directly to the server 101 or built in the server 101. As shown in the figure, a mobile phone 20, a personal computer (PC) 21, a home robot 22, and the like can also be connected to the network 10.

  In the in-vehicle unit 200, 201 is a vehicle interior microphone 201 that collects the sound of the passenger (user), and 202 is a sound detection unit 202 that detects the sound of the passenger (user) input from the vehicle interior microphone 201. 203 denotes an in-vehicle camera 203 and 204 that captures an image including the user's face, and an image detection unit that detects the user's face image from the image captured by the in-vehicle camera 203. Reference numeral 205 denotes a multimodal information processing unit that analyzes the sound detected by the sound detection unit 202 and the face image detected by the image detection unit 204.

  Reference numeral 206 denotes a user determination unit that performs user determination using the analysis result in the multimodal information processing unit 205 and the user determination DB 102. Reference numeral 207 denotes a user preference determination unit that performs user preference determination processing using the preference determination DB 103 and updates the user preference DB 104. Reference numeral 208 denotes a user preference data classification unit that classifies user preferences determined by the user preference determination unit 207. It is.

  Reference numeral 209 denotes a request level determination unit that determines a request level for a target word of a user's utterance based on the user preference DB 104. 210 is a dialogue planning unit that generates a dialogue plan for the user based on the determination result of the request level determination unit 209. 211 is an utterance interruption determination unit that determines whether or not to perform voice output by interruption based on the current position of the vehicle specified by the navigation device 212 using GPS and the map data 213.

  A dialog output control unit 214 controls the dialog output according to the processing results of the dialog planning unit 210 and the speech interruption determination unit 211. The voice synthesizer 215 synthesizes voice according to the control command from the dialogue output controller 214 and outputs synthesized voice through the speaker 216. The image display unit 217 displays an image on the display 218 in accordance with a control command from the dialogue output control unit 214.

  FIG. 2 is a flowchart showing the processing contents of the vehicle dialogue system in the present embodiment.

  First, processing performed by the server 101 in the service center 100 will be described.

  In step S1, user data storage processing is performed. Here, voice frequency pattern data used for voiceprint analysis and facial image feature quantities used for face image recognition are collected as user data for each user through the mobile phone 20, the PC 21, the home robot 22, and the like. And registering in the user determination DB 102.

  Next, in step S2, preference determination data registration processing is performed. Here, the vocabulary representing the emotion for determining the preference is registered in the preference determination DB 103 through the mobile phone 20, the PC 21, the home robot 22, and the like. In the present embodiment, as shown in FIG. 6, vocabulary representing emotions for determining preference is classified and registered as “positive” and “negative” in the preference determination DB 103.

  Next, in step S3, topic data classification processing is performed. Details of the topic data classification process are shown in the flowchart of FIG.

  First, all target words classified as “positive” are read from the user preference DB 104 (step S31). When there is a target word (positive word) classified as “positive” (step S32, YES), information related to each target word is extracted from the Internet using, for example, a known search engine (step S33). The entire sentence of the extracted information is stored in the topic DB 105 in association with the target word and a predetermined large classification (step S34).

  Thereafter, it is determined whether or not all positive words have been processed (step S35). If no, the processes of steps S33 and S34 are repeated for another positive word. When the processing is completed for all positive words in this way, the process proceeds to step S36. If there is no positive word in step S32, the process proceeds to step S36.

  In step S36, it is determined whether there is a no reaction word in the user preference DB 104. A “no-reaction” word is a word that is not classified as “positive” or “negative”. If the no-reaction word is not included in the user preference DB 104, the topic data classification process is directly exited. On the other hand, when the no-reaction word is included in the user preference DB 104, information related to each no-reaction word is extracted from the Internet using, for example, a known search engine (step S37). The summary sentence is stored in the topic DB 105 in association with the no-reaction word and the major classification (step S38).

  Thereafter, it is determined whether or not processing has been performed for all no-reaction words (step S39). If not, the processing of steps S37 and S38 is repeated for another no-reaction word. When processing is completed for all no-reaction words in this way, the topic data classification processing is exited.

  An example of the topic DB 105 generated by the topic data classification process in step S3 is shown in FIG. Each topic is classified into large categories such as “sports”, “facility”, and “meal”. If “soccer” is a positive word, soccer-related information is registered in full text. On the other hand, when “baseball” is a no-reaction word, baseball-related information is registered as a summary.

  Next, the description will be returned to the flowchart of FIG. 2, and the process performed by the in-vehicle unit 200 will be described.

  First, the voice detection unit 202 detects the user's utterance input to the vehicle interior microphone 201 (step S4), and the image detection unit 204 detects the user's face image from the image captured by the vehicle interior camera 203. (Step S5). Thereafter, the multimodal information processing unit 205 calculates voice frequency pattern data used for voiceprint analysis from the voice detected in step S4, and performs face image recognition from the face image data detected in step S5. A feature amount of the face image to be used is calculated (step S6). Then, the user determination unit 206 determines the user by comparing and collating the voice frequency pattern data and the facial image feature amount calculated in step S5 with the data stored in the user determination DB 102 (step S7). ). That is, voice recognition and face image recognition are performed in steps S6 and S7.

  Next, the user preference determination unit 207 performs a user preference determination process determined in step S7 (step S8). FIG. 5 shows a flowchart according to an example of the user preference determination process.

  First, morphological analysis is performed on the text of the user utterance obtained by speech recognition, and the word registered in the preference determination DB 103 is extracted as a keyword (target word) from the result (step S81). Next, dependency analysis is performed to extract a dependency set that is a set of the extracted keyword and its evaluation expression word (step S82). Note that the morphological analysis and dependency analysis described above may use techniques known in the field of natural language processing.

  Next, it is determined what the evaluation expression word obtained in step S82 is classified in the preference determination DB 103 (steps S83 to S88). Here, for the subsequent processing, when the evaluation expression word is “positive”, 2 is given to the target word as a required degree point (evaluation value). On the other hand, when the evaluation expression word is “no reaction”, 1 is given as the required degree point to the target word, and when it is “negative”, 0 is given as the required degree point to the target word.

Then, the target word and the determination result are registered in the user preference DB 104 (step S89). For example, as shown in FIG. 6, when the user's utterance is “I watched a baseball game while having dinner last night, but soccer is interesting”, the user preference DB 104 stores the target word and its evaluation expression. Extracted as a dependency group that is a pair with a word,
“Meals” → “Yes”,
"Baseball" → "I saw",
“Soccer” → “Interesting”
Are registered together with their evaluation values (positive, negative, no reaction). Specifically, as evaluation values in this case, 1 is given to the target words “meal” and “baseball”, and 2 is given to “soccer”.

  At this time, as shown in FIG. 6, related words used in the conversation when the target word is spoken may be registered as attribute information of each dependency group.

  7 and 8 are diagrams illustrating another example of the user preference determination process.

  In this other example, not the vocabulary representing the emotion for judging the preference but the facial expression data representing the emotion for judging the preference are registered in the preference judging DB 103 as shown in FIG. The

  First, morphological analysis is performed on the text of the user utterance obtained by speech recognition, and the words registered in the preference determination DB 103 are extracted as keywords (target words) from the results (step S811). Next, the facial expression of the user at the time of the utterance is detected (step S812).

  Then, it is determined what the facial expression obtained in step S812 is classified in the preference determination DB 103 (steps S813 to S818). Here, as in the above example, for the subsequent processing, when the evaluation expression word is “positive”, 2 is given as the required degree point. On the other hand, when the evaluation expression word is “no reaction”, 1 is given as the required degree point, and when it is “negative”, 0 is given as the required degree point.

  Then, the target word and the determination result are registered in the user preference DB 104 (step S819).

  The above is an example of the user preference determination process in step S8.

  Next, the user preference data classification unit 208 performs user preference data classification processing (step S9). In this user preference data classification process, for example, as shown in FIG. 9, word target extraction added to the user preference DB is performed (step S91) and converted into self-organizing map learning data (step S92). . Using the self-organizing map learning data obtained in step S92, word classification processing by self-organizing map learning described later is performed (step S93), and further, large classification and middle classification class extraction processing is performed ( Step S94).

  10 and 11 show the contents of the word classification processing by self-organizing map learning in step S93.

First, the relevance of the attribute information of the target word is set as the input vector Xj (step S931). Next, the winning node c having the reference vector m _c (t) that best matches the input vector Xj is searched according to the equation 1 (step S932).

The winning node c and its neighboring node i are updated according to Equation 2. This is performed for all input signals x ₁ ,..., X _m . As learning (number of updates) proceeds, the range and size of learning are reduced and converged.

  This equation means a one-layer neural network. As the learning (the number of updates) is advanced, the neighborhood size σ and the learning rate coefficient α are decreased. Typical examples are shown in Equations 3, 4, and 5.

  Then, it is determined that the target word belongs to the classification of the winning node c (step S933).

  FIG. 12A shows an example of the word classification processing result based on the self-organizing map in step S93 when the neighboring node is set to a hexagon. According to this example, it is processed into four major categories of sports, meal, information, and entertainment, and four major categories of Japan national soccer, soccer J-league, baseball major league, and baseball domestic league under the major category of sports. I understand that.

Next, a case where unknown data x _p is input to the self-organizing map after learning will be described. When learning with respect to the input data is completed, the output state (reference vector) _mi (T) of each node becomes constant. That is, the characteristic of each node is expressed by the reference vector m _i (T). When unknown data _xp is input to the self-organizing map after learning, a node q to which the unknown data on the self-organizing map belongs is calculated according to Equation 6.

  For example, as shown in FIG. 12B, if unknown data related to the “run” word is input, the related major classification / medium classification is determined based on the degree of relevance. Further, the user preference feature can also be determined on the self-organizing map after learning. For example, as shown in FIG. 12C, A is positive for sports and negative for entertainment, and B is positive for tourism and negative for sports. The determination result is shown.

  Next, the large classification & middle classification extraction process in step S94 is shown in FIG. First, a large classification & middle classification class is extracted (step S941), and a large classification & middle classification class is added to the user preference DB (step S942). An example is shown in FIG.

  Next, the request level determination unit 209 performs a request level determination process (step S10). FIG. 15 shows a flowchart of the request level determination process.

  First, the result of the user preference determination process in step S8 is extracted for the target word of the user's utterance (step S101). In the case of the example shown in FIG. 6, the user's utterance is “Last night, while watching a baseball game while eating, but soccer is interesting”, as evaluation values, the target words “meal” and “baseball” 1 is given to “Soccer”, and 2 is given to “soccer”.

  Next, the total value of the request points for each major classification of the target word of the user utterance is calculated (step S102). Specifically, in the above example, the major category “sport” is “baseball” 1+ “soccer” 2 = 3, the major category “meal” is 1, and the major category “facility” is 0.

  Then, the request priority of the large classification is calculated according to the request point obtained in step S102. In the case of the above example, the request priority is “sport”, “meal”, and “facility” in descending order.

  Next, the dialogue planning unit 210 performs dialogue planning processing. FIG. 16 shows an example of this dialogue planning process.

  First, the major category with the highest request priority is selected (step S111), and the vocabulary of the user preference DB 104 corresponding to the selected major category is extracted (step S112).

  It is determined whether or not the vocabulary extracted in step S112 includes a positive word (step S113). If a positive word is included, the full text of the topic stored in the topic DB 105 related to the positive word is extracted. (Step S114). On the other hand, if the vocabulary extracted in step S112 does not include a positive word, it is determined whether the vocabulary extracted in step S112 includes a no reaction word (step S115), and the no reaction word is included. In this case, a summary of topics accumulated in the topic DB 105 related to the no-reaction word is extracted (step S116).

  The full text or summary of the topic extracted in step S114 or step S116 is stored in the hierarchical interactive content DB 106 (step S117).

  Then, it is determined whether all major classification processes have been completed (step S118). If all major classification processes have not yet been completed, the next highest priority major classification is selected (step S119). Returning to step S112, the process is repeated.

  An example of the hierarchical interactive content DB 106 created in this way is shown in FIG.

  Next, the speech interrupt determination unit 211 performs speech interrupt determination processing. The contents of this utterance interruption determination process are shown in FIG.

  First, the traveling position of the vehicle is detected based on the data supplied from the navigation device 212 (step S121). Next, roadside facilities within a predetermined distance are extracted from the travel position of the vehicle detected in step S121 using the map data 213 (step S122). Next, the roadside facility registered as positive or no reaction in the user preference DB 104 is further extracted from the roadside facility extracted in step S122 (step S123).

  If there is a roadside facility extracted in step S123 (YES in step S124), the information on the roadside facility is set as information of interest to the user through voice / image by interruption through the dialog output control unit 214. Output (step S125).

  On the other hand, if there is no roadside facility extracted in step S123 (step S124, NO), voice or image based on priority is output by the hierarchical interactive content DB 106 through the interactive output control unit 214 (step S126). Specifically, here, information is presented in full text for the positive word with the highest priority, and information is presented in summary for the no-reaction word.

It is a block diagram which shows the structure of the dialog system for vehicles in embodiment. It is a flowchart which shows the processing content of the dialogue system for vehicles in embodiment. It is a flowchart which shows the topic data classification | category process by the server of the service center in embodiment. It is a figure which shows typically an example of topic DB in embodiment. It is a flowchart which shows an example of the user preference determination process in embodiment. It is a figure explaining an example of the user preference determination process in embodiment. It is a flowchart which shows another example of the user preference determination process in embodiment. It is a figure explaining another example of user taste judging processing in an embodiment. It is a flowchart which shows an example of the user preference determination process in embodiment. It is a flowchart which shows an example of the word classification | category process by the self-organization map in embodiment. It is a figure explaining an example of the word classification | category process by the self-organization map in embodiment. It is a figure explaining an example of the word classification | category process result by the self-organization map in embodiment. It is a flowchart which shows an example of the extraction process of the large classification & middle classification class in embodiment. It is a figure explaining an example of user taste DB after word classification processing by a self-organization map in an embodiment. It is a flowchart which shows an example of the request | requirement degree determination process in embodiment. It is a flowchart which shows an example of the dialogue planning process in embodiment. It is a figure which shows an example of the hierarchical dialogue content DB in embodiment. It is a flowchart which shows an example of the speech interruption determination process in embodiment.

Explanation of symbols

10: Network 20: Mobile phone 21: Personal computer 22: Home robot 100: Service center 101: Server 102: User determination DB
103: DB for taste determination
104: User preference DB
105: Topic DB
106: Hierarchical dialogue content DB
200: vehicle-mounted unit 201: vehicle interior microphone 202: sound detection unit 203: vehicle interior camera 204: image detection unit 205: multimodal information processing unit 206: user determination unit 207: user preference determination unit 208: user preference data classification unit 209 : Request level determination unit 210: Dialog planning unit 211: Speech interruption determination unit 212: Navigation device 213: Map data 214: Dialog output control unit 215: Speech synthesis unit 216: Speaker 217: Image display unit 218: Display

Claims (10)

Detecting means for detecting a vehicle occupant;
A determination unit that determines the preference of the occupant based on the utterance of the occupant detected by the detection unit;
Determining means for determining the content of the dialogue based on the passenger's preference determined by the determining means;
Control means for controlling a dialogue with the occupant based on the dialogue content decided by the decision means;
A vehicle dialogue system comprising: Voice recognition means for recognizing the utterance of the passenger;
Dependency analysis means for converting speech recognized by the speech recognition means into text and analyzing grammatical dependency relationships between morphemes constituting the text;
Further comprising
2. The vehicle dialogue system according to claim 1, wherein the determination unit determines the preference of the occupant based on the dependency relationship analyzed by the dependency analysis unit. A face image recognition means for recognizing the face image of the occupant;
2. The vehicle dialogue system according to claim 1, wherein the determination unit determines the preference of the occupant based on the face image of the occupant recognized by the face image recognition unit. A preference database that accumulates vocabulary related to the passenger's preference determined by the determination means;
A topic database that accumulates topic data collected over the network;
Further comprising
The determining means extracts a vocabulary according to a determination result of the determining means from the preference database, and determines the conversation content by extracting a topic related to the extracted vocabulary from the topic database. The vehicle dialogue system according to any one of claims 1 to 3. Generating a self-organizing map of the preference database, and forming a speech content cluster of an occupant based on the self-organizing map;
5. The vehicle dialogue system according to claim 4, wherein the decision means decides dialogue contents based on the utterance content cluster.   5. The vehicle dialogue system according to claim 4, wherein the preference database and the topic database are configured to store data transferred from an external device via a network.   The vehicle dialogue system according to claim 1, further comprising a hierarchical dialogue content database in which dialogue contents are classified and stored hierarchically according to priority.   8. The vehicle dialogue system according to claim 7, wherein the determination unit determines the priority based on the passenger's preference determined by the determination unit.   The interactive system for vehicles according to claim 4, wherein each of the topic database and the preference database is associated with a target facility provided by a navigation system. The topic database is associated with a roadside facility provided by the navigation system,
10. The vehicle dialogue system according to claim 9, wherein the decision means decides a dialogue content related to a roadside facility according to the position of the vehicle based on the topic database and the preference database.