JP2021077268A - Information presentation device and information presentation method - Google Patents

Abstract

To provide an information presentation system and an information presentation method capable of obtaining necessary information by utilizing past dialog history while using no direct keyword.SOLUTION: An information presentation system includes an action storage unit for storing a recall direction phrase included in a request utterance of a user obtained by performing interactive processing, implementation content indicating content in which the recall direction is changed and retrieved when a concept indicating a recall direction corresponding to the recall direction phrase is changed in order to obtain reply content to the request utterance, and feedback indicating whether or not the reply content is satisfactory as information desired by the user; and a recall direction management unit for calculating a degree of reliability indicating a degree of reliability for the recall direction that the recall direction phrase of the user to obtain information including the reply content desired by the user based on the recall direction phrase, implementation content stored in the action storage unit, and the feedback.SELECTED DRAWING: Figure 1

Description

An embodiment of the present invention relates to an information presenting device and an information presenting method.

In recent years, with the enhancement of the Internet environment, social network services (hereinafter referred to as SNS) have become widespread, and it has become possible to easily communicate between a plurality of users using texts and images. For example, typical SNS applications include LINE (registered trademark), Facebook (registered trademark) messenger, and Slack (registered trademark). These SNSs not only exchange information between one-to-one users, but also share information transmitted and received between a large number of users in a predetermined group (dialogue between a plurality of users) among all users in the group. It also has a function.

Further, examples of the man-machine interactive SNS include Google Assistant (registered trademark), Amazon Alexa (registered trademark), and Line Clova (registered trademark).
In addition, each of the above-mentioned applications is installed in personal computers and smart devices, and smart speakers such as Google Home (registered trademark), Amazon Echo (registered trademark), and Clova Wave (registered trademark), and voice synthesis is performed in each of them. , Information presentation using voice is also widely used.

Various services are being developed for man-machine interactive SNS. Regarding the usage method, it is often used not only in the situation where the user and the system are one-on-one, but also in the situation where there is an unspecified number of users vs. the system (reservation and timetable by interactive service without going to the agency when planning a trip You can search, inquire about insurance plans, etc.).

In the man-machine interactive SNS, in order to realize the dialogue between the user and the system, rules are constructed in advance assuming the flow of dialogue regarding what kind of response utterance is to be made in response to the request utterance from the user. Rule-based methods are often used.

Various ideas have been devised for constructing dialogue rules, and a method of using a labeled dialogue set as an alternative to a detailed dictionary such as an ontology has been proposed for the purpose of reducing the cost of manually constructing rules (ontology). Patent Document 1).

As a technology to assist the generation of dialogue rules for the purpose of improving the quality of dialogue, response utterances are based on the attributes of words (evaluation information such as positive and negative) uttered by the user during the dialogue and the attributes of the user. A method for customizing the presented content has been proposed (Patent Document 2).

Japanese Patent No. 4755478 Japanese Patent No. 6381775

However, in the conventional interactive SNS, when the user wants to refer to arbitrary information in the past, he / she can browse all the past information, and even if the search is possible, he / she cannot remember the keyword or in a group dialogue. There is a problem that it is difficult to intuitively reproduce the contents, such as the common recognition that triggers the search is unclear and cannot be referred to.

Also, in the case of dialogue between people, when referring to the content of past dialogue, not only the specific content of the story, but also the surrounding environment such as "when", "where", "who", "what", etc. Often, conversations are held using the environmental information of the above, and it is not uncommon for dialogues to be established.

On the other hand, in the case of a general dialogue service, the man-machine type is premised on a general-purpose service for an unspecified number of people, so even if an attempt is made to retrieve information obtained in the past, the desired information can be obtained. Must follow a predetermined procedure and cannot be referred to using the above-mentioned environmental information or the like.

Efficient information grasping can be expected by adding new dialogue rules using the framework of man-machine dialogue, but although there are rule-based methods that assume dialogue scenarios in advance, it is done manually. There are many parts to be maintained, which is quite difficult in terms of the amount of work.

Even in a message application, it is preferable from the interface point of view that information can be retrieved by a mechanism like a man-machine type.

The present invention has been made in view of such circumstances, and an object of the present invention is an information presenting device capable of obtaining necessary information by utilizing past dialogue history without using direct keywords. , To provide a method of presenting information.

In order to solve the above-mentioned problems, one aspect of the present invention presents information capable of constructing dialogue rules and controlling dialogue based on the recall orientation that expresses the intention of memory recall based on the dialogue history of the user and the environmental information. Included in the dialogue management unit that is a system and performs dialogue processing on the terminal device used by the user, the dialogue history storage unit that stores the dialogue history that is the dialogue history, the environment in which the dialogue was performed, and the dialogue. The environment storage unit that stores the recall-oriented phrases and the recall-oriented words, the recall-oriented phrases included in the user's request speech obtained by performing the dialogue processing, and the recall content in order to obtain the response contents to the request speech. When the concept indicated by the recall orientation corresponding to the orientation phrase is changed, the implementation content indicating the content searched by changing the recall orientation and the feedback indicating whether or not the response content is satisfied as the information desired by the user. Based on the behavioral memory unit that stores the information, the recall phrase stored in the behavioral memory unit, the implementation content, and the feedback, the recall orientation indicated by the user's recall-oriented phrase is that of the user. It is an information presentation system equipped with a recall-oriented management unit that calculates the reliability indicating the degree of reliability in order to obtain information including the desired response content.

Further, in one aspect of the present invention, the recall-oriented management unit responds to environmental information similar to the environment of the dialogue based on the recall-oriented phrase included in the dialogue stored in the dialogue history storage unit. To get.

Further, one aspect of the present invention is a dialogue that generates a dialogue rule by applying a word included in the dialogue content stored in the dialogue history storage unit and a word stored in the environment storage unit to a sentence template. It has a rule building department.

Further, in one aspect of the present invention, the dialogue management unit determines a response utterance to a request utterance according to the recall orientation calculated by the recall orientation management unit.

Further, in one aspect of the present invention, the environmental storage unit stores surrounding environmental information including members, weather, place, date and time.

Further, one aspect of the present invention is an analysis unit that obtains numerical data according to the dialogue content by using any of morphological analysis, keyword extraction, and vectorization for the dialogue content stored in the dialogue history storage unit. And have.

Further, one aspect of the present invention has an attribute storage unit that stores demographic data including the age, gender, and place of residence of the user, and the recall orientation, and the recall orientation management unit is stored in the attribute storage unit. The recall-oriented optimization process is performed by changing the recall-oriented phrase of the request utterance so that the answer requested by the user can be obtained by using the stored information.

Further, in one aspect of the present invention, the recall-oriented management unit writes and stores the reliability of the recall-oriented in the attribute storage unit for each user.

Further, in one aspect of the present invention, the recall-oriented management unit updates the reliability of the recall-oriented based on the search result and feedback stored in the behavior memory unit.

Further, one aspect of the present invention includes a grouping estimation unit that groups based on the characteristics of the user, and a grouping storage unit that stores the results grouped by the grouping estimation unit, and the model construction unit. Uses a model for estimating the recall orientation of users who share the same recall orientation of the users to be changed, based on the grouping result stored in the information of the grouping storage unit.

Further, one aspect of the present invention uses a model in which dialogue rule construction and dialogue management are changed from the grouping storage unit according to the recall orientation common to the users.

Further, in one aspect of the present invention, the recall-oriented management unit constructs a dialogue rule for a user whose modification model is not prepared or insufficient by using a model having a tendency similar to that of the user. Generate a model for optimizing dialogue management.

Further, one aspect of the present invention has a synthesis unit that generates content by processing information including response content.

Further, in one aspect of the present invention, the synthesis unit has a structure for generating contents that have been processed differently depending on the terminal device of the transmission destination.

Further, one aspect of the present invention is an information presentation method in an information presentation system capable of constructing dialogue rules and controlling dialogue based on a recall orientation that expresses a memory recall orientation based on a user's dialogue history and environmental information. The information presentation system includes a dialogue history storage unit that stores dialogue history, which is the history of dialogue, an environment storage unit that stores the environment in which the dialogue was conducted, and recall-oriented phrases and recall-oriented phrases included in the dialogue. When the recall-oriented phrase included in the user's request utterance obtained by performing the dialogue processing and the concept indicated by the recall-oriented phrase corresponding to the recall-oriented phrase are changed in order to obtain the response content to the request utterance, the recall is concerned. It has an action storage unit that stores an implementation content that indicates the content searched by changing the intention and a feedback indicating whether or not the response content is satisfied as the information desired by the user, and is a dialogue management unit. However, the user performs an interactive process on the terminal device used by the user, and the recall-oriented management unit uses the recall phrase stored in the action storage unit, the implementation content, and the feedback. This is an information presentation method for calculating the reliability indicating the degree to which the recall-oriented phrase indicated by the recall-oriented phrase can be trusted in order to obtain information including the response content desired by the user.

The present invention focuses on the construction of dialogue rules, and aims to improve the efficiency of information reference from the past dialogue history by incorporating information that promotes memory recall or contributes to ambiguous search into the construction of dialogue rules. Specifically, the dialogue rules are constructed using the dialogue / search history, words based on the common recognition within the group included in the dialogue / search history, and environmental information (weather, room, etc.) for reminding the memory, and the user. Based on the results of comparing and analyzing the tendencies of each user or between users, and by calculating the orientation regarding memory recall, it is possible to construct the dialogue rules and control the dialogue.

One aspect of the present invention is characterized in that, in the recall orientation management unit, a parameter for estimating the recall orientation of the user is determined by any or a combination of the information stored in the storage unit by various storage means described later. To do.

In one aspect of the present invention, the recall-oriented management unit performs learning using a framework such as natural language processing, machine learning, neural network, and reinforcement learning with the addition of new information, and optimizes the estimation algorithm. Equipped with functions (machine learning can be performed to optimize the parameters required for formulation).

One aspect of the present invention is characterized in that the dialogue rule building unit constructs a dialogue rule according to the information of the recall-oriented and storage unit calculated by the recall-oriented management unit.

One aspect of the present invention is characterized in that the dialogue management unit determines the response utterance to the request dialogue according to the recall orientation calculated by the recall orientation management unit.

(Description of information and features acquired by the environmental storage unit)
One aspect of the present invention is characterized in that the environmental storage unit stores information other than the dialogue (which changes depending on the content of the dialogue) such as the members who participated in the dialogue and the date and time, place, weather, and temperature. And.

As one aspect of the present invention, the contents of the environmental storage unit can be converted into numerical values or symbols as feature quantities, processed into a format in which different information such as a single substance or a combination is combined, and held.

(Description of information and features acquired in the dialogue history storage unit)
One aspect of the present invention is characterized in that the dialogue history storage unit stores dialogue contents and the like on the user's system.

As one aspect of the present invention, the dialogue content of the dialogue history storage unit can be converted into numerical values or symbols representing co-occurrence, similarity, etc. and used (using known techniques such as tf-idf and word2vec). ).

As one aspect of the present invention, the dialogue history storage unit handles all the dialogues performed through the system, and the dialogue history using the dialogue rules constructed by using the recall orientation is obtained in advance in addition to the dialogue contents. It is characterized in that it records a phrase that matches or is similar to the phrase (related to the environmental information described later) used when referring to the past dialogue set in, or stores the total number of detections or each request content. ..

(Description of information and features acquired in the attribute storage unit)
As one aspect of the present invention, the attribute storage unit holds user attribute information other than dialogue (particularly, one that does not change due to dialogue) such as demographic data such as age, gender, and location of the user. It is a feature.

As one aspect of the present invention, the attribute storage unit is characterized in that, in addition to the above information, parameters calculated by the recall-oriented management unit are written and stored in the attribute storage unit for each user.

(Description of information and features acquired by the behavioral memory unit)
One aspect of the present invention is characterized in that the behavior storage unit records the behavior of both the system and the user.

As one aspect of the present invention, the behavioral memory unit estimates the dialogue rules constructed and the specific implementation contents (association of request utterance and response utterance, degree of misunderstanding) based on the estimation result of recall orientation described later. It is characterized by recording the presentation.

As one aspect of the present invention, the behavioral memory unit clearly allows the user's evaluation (“was it ?, yes / no” or the like selected on the system) for the result of the dialogue processing according to the recall-oriented estimation result. It is characterized by memorizing the presentation).

(Description of information and features acquired in the grouping storage unit)
One aspect of the present invention is characterized in that the grouping storage unit realizes and records grouping for users having similar tendencies based on the information of the storage means.

One aspect of the present invention is characterized in that the grouping storage unit is further provided with a change model for changing each of dialogue rule construction and dialogue management according to a recall orientation common to users.

As one aspect of the present invention, the recall-oriented management unit extracts a change model having a similar tendency for a user whose change model is not prepared or is insufficient, and optimizes dialogue rule construction and dialogue management. It is characterized by generating a modification model for optimization.

One aspect of the present invention is characterized in that the presentation control unit changes the presentation method according to the response utterance determined by the dialogue management unit based on the estimated contents derived by the recall-oriented management unit.

One aspect of the present invention is to change the presentation form to the user based on the information of the presentation control unit, and the presentation control unit can present visual information to the user in addition to the synthesis unit to the system. If there is a visualization unit, the display is switched.

It is also possible to present to the user without going through the voice synthesis process.

As described above, according to the present invention, it is possible to obtain necessary information by utilizing the past dialogue history without using a direct keyword.

Further, according to the present invention, by using the above mechanism, it is possible to extract desired information based on a word that recalls a pronoun or a memory and perform a dialogue without using a direct expression such as a proper noun. it can. This enables efficient reference of past dialogue history.

In addition, in addition to the history of references by recall based on actual environmental information and the tendency of environmental information that the user uses for recall, using the evaluation from the user for the presentation of the system, misunderstandings and memories that frequently occur in dialogue. The recall-oriented management unit can be optimized in consideration of the tendency of difference, and by comparing and analyzing the recall-oriented between users, it is possible to provide a process to which the recall-oriented is applied even to a user with little information.

A block diagram showing a configuration example of the information presentation system 1 according to the embodiment of the present invention. A block diagram showing a configuration example of the terminal device 10 according to the embodiment of the present invention. The figure which shows the structural example of the digital data transmitted to the information presenting apparatus 20 The figure which shows the image of the presentation form in a terminal apparatus 10. The figure which shows the image of the presentation form in a terminal apparatus 10. The figure which shows the configuration example of the information presenting apparatus 20 Diagram showing a configuration example of the dialogue management unit 212 The figure which shows the data structure example of the dialogue history storage part The figure which shows the data structure example of the environment storage part The figure which shows the data structure example of the attribute storage part The figure which shows the data structure example of the action memory part The figure which shows the data structure example of a grouping storage part The figure explaining the configuration example of the language knowledge memory part 226 Block diagram showing a configuration example of the recall-oriented management unit 240 The figure explaining the example of the optimization of the model building part 242 Block diagram showing a configuration example of the dialogue rule construction unit 250 The figure which shows the data structure example of the dialogue rule storage part Flowchart showing system operation

(Embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Overview of information presentation system)
FIG. 1 is a block diagram showing a configuration example of the information presentation system 1 according to the embodiment of the present invention.

The information presentation system 1 envisions a dialogue system in which humans or computers and humans communicate with each other by presenting voices, images, texts, and the like.
The information presentation system 1 includes a terminal device 10 (terminal device 10a, terminal device 10b, terminal device 10c, terminal device 10d) and an information presentation device 20 (information presentation device 20a, information presentation device 20b, information presentation device 20c, information presentation). The device 20d) and the communication network 30 are included.

The terminal device 10 is a terminal used by a user who requests transmission / reception and presentation of information, for example, a general-purpose computer, a personal computer, a tablet terminal, a smartphone, a smart speaker, or the like. The terminal device 10 of the information presentation system 1 acquires a message from the user via the input device, and transmits the message to the information presentation device 20 via the communication network 30. Specifically, the terminal device 10 acquires the voice uttered by the user by an input unit (such as a microphone) included in the terminal device 10 itself. The terminal device 10 converts the acquired voice into voice data. The terminal device 10 transmits the voice data to the information presenting device 20 via the communication network 30 by the transmission unit included in the terminal device 10.

FIG. 2 is a diagram showing a configuration example of the terminal device 10 according to the embodiment of the present invention.
The terminal device 10 includes an input unit 110, an interface unit 120, a storage unit 130, a sensor unit 140, an output unit 150, a communication unit 160, and a control unit 170.
The input unit 110 acquires a request from the user. For the input unit 110, for example, a microphone can be used. When the input unit 110 is a microphone, the input unit 110 generates an audio signal corresponding to the input voice. Further, the input unit 110 may be an input device such as a keyboard or a mouse.
The interface unit 120 operates the terminal device 10.
The storage unit 130 stores parameters set by using the interface unit 120, an external control terminal, or the like.
The sensor unit 140 acquires information such as GNSS (Global Navigation Satellite System) position information and room temperature.
The output unit 150 outputs the information acquired from the information presenting device 20. The output unit 150 is, for example, a liquid crystal display device, a speaker, or the like, and can output various data, contents, and the like. As a result, various information is presented to the user.
The communication unit 160 has a transmission unit 161 and a reception unit 162, and communicates with an external device. For example, the communication unit 160 transmits request information for making an inquiry based on the information input from the input unit 110. This request information is information that requests a response to a question or request. For example, the request information includes data that expresses a question, such as "What is the price of the book you talked to Mr. A last time?", Or an operation request for a specific device, such as "Set to the same cooling temperature as before." It is the data to represent.
The transmission unit 161 transmits various information to the information presenting device 20.
The receiving unit 162 receives various information transmitted from the information presenting device 20.
The control unit 170 controls each unit in the terminal device 10.
The control unit 170 generates voice data corresponding to the voice signal generated by the input unit 110, and causes the communication unit 160 to transmit request information using the voice data.

FIG. 3 is a diagram showing a configuration example of digital data (collected data) transmitted from the terminal device 10 to the information presenting device 20. The data transmitted to the information presenting device 20 includes data representing a user's request input from the user via the input unit 110, data obtained by the sensor unit 140, and data stored in the storage unit 130. , At least one is included.
Further, as the data transmitted to the information presenting device 20, based on the GNSS information of the terminal device 10, the weather forecast data at the place where the terminal device 10 exists is used, and the weather forecast API or the like is used from an external weather information server. The communication unit 160 may acquire the data and transmit the data to the information presenting device 20. The above-mentioned requirement information is also used as a part of this digital data.

The collected data includes information such as time, user ID, utterance data, session ID, location information, temperature, and installation location.
The time indicates the date and time when the utterance was made.
The user ID is information that identifies the user, and in particular, is a user ID that identifies the user who is using the terminal device 10. As this user ID, the ID assigned to the terminal device 10 can be used, and when the terminal device 10 is a smart speaker, the ID at the time of user registration when using this smart speaker is used. be able to.
The utterance data is information representing the utterance content obtained based on the content uttered by the user, and is also referred to as a message. As for the utterance data, for example, when the user's utterance content is input by the microphone used as the input unit 110, the utterance data is converted from the voice data to the text data by recognizing the uttered content by voice. can get. The utterance data is not limited to voice data, and when the input unit 110 is a keyboard, mouse, or the like, text data or figures (emoticons, pictograms, stamps, etc.) input from the input unit 110 are used. Can be used as dialogue data.
The session ID is identification information that identifies this dialogue group as one dialogue group from the start of one dialogue to the end of the dialogue. If the sessions are the same, the same session ID as the session ID stored in the dialogue history storage unit 221 is used.
The position information indicates the position of the user who spoke at the time of the dialogue. For the user's position, for example, the current position (for example, latitude and longitude) may be specified by the GNSS function of the terminal device 10, and the position of the terminal device 10 may be used as information indicating the user's position. For example, when the terminal device 10 is a smartphone, since the smartphone is generally carried by the user, the position of the terminal device 10 may be set as the user's position.
The temperature indicates the ambient temperature of the terminal device 10. When the terminal device 10 is used indoors, it indicates a room temperature, and when it is used outdoors, it indicates an air temperature.
The installation location is the name of the location where the terminal device 10 is installed or the location where it is mainly used. For example, when the terminal device 10 is a smart speaker, it indicates a place where the smart speaker is installed.

Returning to FIG. 1, the information presenting device 20 determines the response information for the request information acquired from the terminal device 10 and generates the content. The information presenting device 20 transmits the generated content to the terminal device 10. The information presentation device 20 is a server device, and for example, either a general-purpose computer or a personal computer can be used.

The information presenting device 20 analyzes the received request information, determines a response by the dialogue management unit described later, optimizes the content according to the presentation form, and presents the content to the user. When the request information transmitted from the terminal device 10 is voice data, the information presenting device 20 converts the voice data into character data and then determines a response by the dialogue management unit 212.

In the present embodiment, the case where the information presenting device 20 converts the voice data transmitted from the terminal device 10 into character data will be described, but the present invention is not limited to this. For example, the terminal device 10 may convert the voice data into character data and transmit the character data to the information presenting device 20 as request information.
Alternatively, the information presenting device 20 transmits the voice data acquired from the terminal device 10 to an external server having a conversion function for converting the voice data into character data, and receives the character data transmitted from the external server to obtain the character data. May be obtained.

In the above embodiment, the information presentation request from the user is input by the microphone included in the terminal device 10, but the present invention is not limited to this.
For example, if it is a device or sensor capable of transmitting a human intention, a device or sensor other than a microphone may be used. For example, the request information is input by using a configuration in which character information is input by a keyboard or a touch panel instead of voice, or a configuration in which information indicated by a human gesture is input by a camera. May be good.
In addition to the above, there is a configuration in which figures (emoticons, pictograms, stamps, etc.) and video content are input from the input unit 110 by the camera, and moving image content saved in advance without using a sensor. You may use it and enter the request information. For example, when inputting video content as request information, it is possible to input video content including an utterance scene such as "What is the price of the book you talked to Mr. A last time?". As a result, the utterance content included in the moving image content may be extracted as request information. Further, for example, when a pictogram is input as request information, a word for displaying the pictogram may be used as request information. For example, if a certain pictogram is a pictogram corresponding to the character "regards", and if the pictogram is input as request information, the information is acquired assuming that the request information "regards" is input. can do. Further, when a certain pictogram is a pictogram representing a "book" and the pictogram is input as request information, the information can be acquired as if the request information "book" was input. Here, only an image such as at least one pictogram may be input as request information, or an image and voice or text data may be combined and input as one request information.

Further, in the above-described embodiment, the processing of the terminal device 10 and the information presenting device 20 is separated via the communication network 30, but the processing is not limited to this.
For example, the terminal device 10 may be configured to include a part or all of the information presenting device 20 (so that the system can be realized without going through a network).

The communication network 30 is, for example, the Internet or a LAN (Local Area Network).

In the present embodiment, the information presentation system 1 operates on any interactive system such as a conversation between humans and a conversation between a human and a computer. For example, it may be a conversation including a chatbot in which a computer, which is a kind of an automatic answering system, a question answering system, and a dialogue system, extracts or generates a response content based on a rule. In addition, conversations such as LINE and Facebook Messenger, in which humans realize dialogues via a network, may be the target of this embodiment.

Details of the dialogue system via the network including LINE and chatbots can be explained by using existing technology, so detailed explanations are omitted.

(Presentation example by terminal device 10)
Next, an example of presentation based on the dialogue rules constructed using the recall orientation will be described.
FIG. 4 is a diagram showing an example of presentation in the terminal device 10 according to the embodiment of the present invention. Note that FIG. 4 is a conceptual diagram that visualizes the exchange by voice for the purpose of explanation. The left side is the terminal device, the right side is the user, and the user is speaking to the terminal device.

FIG. 4A shows a case where a user Un specifies a proper noun (for example, “book A”) by uttering it to extract information. In this figure, the terminal device 1000 obtains and outputs an answer to a question from a server connected to the outside based on a voice input including a proper noun from a user Un.
FIG. 4B shows a case where a dialogue is realized by using three items of date and time (when), place (where), and person (who) as environmental information. As shown in this figure, the terminal device 10a in the present embodiment derives and outputs the response content even if the content uttered by the user Ua does not include a proper noun (for example, "book A"). be able to.

In the above embodiment, an embodiment premised on voice dialogue has been described, but the present invention is not limited to this, and when the terminal device 10 can present (output) information other than voice, it promotes grasping and understanding of the system response. Therefore, the content may be presented. There are various implementation methods, and as a simple method, a text with the same content as the read content is displayed, or a related item such as past voice or moving image is presented (voice output or image display). You may take.

(Configuration example of information presentation device 20)
FIG. 5 is a diagram showing a configuration example of the information presentation device 20 according to the embodiment of the present invention. The information presentation device 20 includes an information processing unit 210, an information storage unit 220, a grouping estimation unit 231, a recall-oriented management unit 240, and a dialogue rule construction unit 250.

The information processing unit 210 includes an acquisition unit 211, a dialogue management unit 212, a presentation control unit 213, and a synthesis unit 214, through which the presentation form of the terminal device 10 is controlled.

The information storage unit 220 stores the dialogue history storage unit 221 that stores the data obtained from the information processing unit 210, the environment storage unit 222 that mainly stores the data obtained from the terminal device 10, the attribute storage unit 223, and other action storage. It is composed of a unit 224, a grouping storage unit 225, and a language knowledge storage unit 226.

The recall-oriented management unit 240 uses the information of the recall analysis unit 241 that extracts words and phrases from the dialogue history and analyzes the orientation, and the information storage unit 220 to estimate parameters related to memory recall and build a model. It is configured to include a model building unit 242.

The dialogue rule construction unit 250 includes an utterance construction unit 251 and a relationship construction unit 252 that construct dialogue rules using the information of the information storage unit 220 and the recall-oriented management unit 240.

(Configuration example of acquisition unit 211)
The acquisition unit 211 is an external input interface that acquires various information such as collected data transmitted from the terminal device 10. The acquisition unit 211 outputs various information such as request information obtained from the terminal device 10 to the dialogue management unit 212.

Further, the acquisition unit 211 may be configured to acquire various information such as request information directly from a microphone, a keyboard, various sensors, or the like.

(Configuration example of dialogue management unit 212)
FIG. 6 is a diagram illustrating a configuration example of the dialogue management unit 212 according to the embodiment of the present invention. The dialogue management unit 212 has a function of determining a response from the system to the user, and performs dialogue processing with respect to the terminal. The dialogue management unit 212 includes an analysis unit 2121 that receives various information from the acquisition unit 211, an action selection unit 2122 that determines state management and response guidelines, and a generation unit 2123 that generates response contents based on the action selection unit 2122. Consists of. For example, the dialogue management unit 212 generates an answer corresponding to a question or the like given by the user by referring to the information stored in the information storage unit 220.

The analysis unit 2121 converts the information obtained from the acquisition unit 211 into data that can be processed in the subsequent steps of the dialogue management unit 212. The type of information to be input differs depending on the interface used by the user. For example, when the input interface is a microphone, the input information is voice information (information including voice data), image information if the input interface is a camera, and character information if the input interface is a keyboard.

The analysis unit 2121 converts the information input from the acquisition unit 211 into numerical data such as morphological analysis, keyword extraction, and vectorization so that the information can be processed in the subsequent steps. Natural language processing technology and machine learning technology can be used for this conversion processing. As an example according to the embodiment of the present invention, a method such as keyword extraction using tf-idf or vectorization using word2vec or doc2vec may be used. The analysis unit 2121 outputs the information converted into the numerical data to the action selection unit 2122.

Further, the analysis unit 2121 determines whether or not the information input from the acquisition unit 211 is a dialogue using the recall orientation, and outputs the determination result together with the above information (numerical data) to the action selection unit 2122. .. For example, the determination of whether or not the dialogue is based on the recall orientation can be determined based on the utterance content included in the utterance data among the digital data (collected data) obtained from the user via the terminal device 10 or the like. .. More specifically, the analysis unit 2121 extracts words and part of speech by performing parsing or morphological analysis of the utterance content, and determines whether or not the extracted words and part of speech correspond to the recall orientation. To do. Whether or not it corresponds to recall orientation is based on whether or not there is a demonstrative pronoun such as "before" and whether or not there is a general name such as "book" when these elements are present. , Judged as recall-oriented. In this case, the language knowledge storage unit 226 stores in advance the recall-oriented phrases (words) and part of speech together with information on whether or not they correspond to the recall-oriented, and the analysis unit 2121 refers to the language knowledge storage unit 226. However, when the word or part of speech to be determined is stored in the language knowledge storage unit 226, it may be determined to be recall-oriented.
The analysis unit 2121 determines whether or not the information input from the acquisition unit 211 is a dialogue using recall orientation, and outputs the determination result together with the above information (numerical data) to the dialogue history storage unit 221. For example, memorize it as a recall-oriented phrase such as "yesterday" or "book".

The action selection unit 2122 defines the user's state based on the information storage unit 220 and the dialogue rule construction unit 250, and determines the response guideline. The state corresponds to, for example, the current topic, and if the requested utterance is "What is the first Shinkansen from Tokyo to Shin-Osaka?", The state (topic) is "Shinkansen". Can be defined as. Here, what to look for can be defined as a state. Regarding what the object to be examined is, for example, by parsing the utterance requested by the user, the object to be examined may be specified (defined) from the relation of part of speech, dependency, and the like.
When the action selection unit 2122 establishes a response guideline, if the request utterance is "Are there seats on the Shinkansen from Tokyo to Shin-Osaka?", The state (topic) is based on the result of syntactic analysis. ) Can be defined as "Shinkansen", and it can be determined that inquiring about the vacant seats of the Shinkansen is the response guideline. Further, for example, when the request utterance is "Is there a ticket for movie B in a movie theater?", The action selection unit 2122 has a state (topic) of "movie theater" based on the result of syntactic analysis. It can be defined as, and it is decided as a response guideline to inquire about the vacant seats when the movie B of the XX movie theater is screened.
The action selection unit 2122 determines the user's state (topic) from the relationship between a plurality of messages in the context, based on the dialogue history stored in the dialogue history storage unit 221 and the numerical data obtained from the analysis unit 2121. Define.

The action selection unit 2122 determines the response information for the request information by using a database of a list of requests (request information) and responses (response information) and a learned model learned by machine learning or reinforcement learning. .. Since the processing content performed by the action selection unit 2122 can use the same configuration as the dialogue system, which is a known technique, detailed description thereof will be omitted. The action selection unit 2122 outputs the generated information to the generation unit 2123.

When the analysis unit 2121 determines that the dialogue is a recall-oriented dialogue, the action selection unit 2122 parses the content of the requested utterance to obtain a content that matches the recall-oriented phrase (word). If there is no matching content, the optimum response content is determined using the recall-oriented model constructed by the recall-oriented management unit 240.
Here, the environmental storage unit 222 stores the past dialogue contents and the recall-oriented phrases included in the dialogue contents in association with each other. The action selection unit 2122 specifies the dialogue content corresponding to the recall-oriented phrase, and determines the response information based on the identified dialogue content, taking into account the content answered in the past. For example, if there is a dialogue content of "1200 yen" as the "price of the book A" in the past dialogue content, it is stored in the environmental storage unit 222.

The generation unit 2123 determines the content of the response based on the response information of the action selection unit 2122. For example, when the action selection unit 2122 determines the response information of "1200 yen" in response to the request information "What is the price of the book that you talked to Mr. A last time?", The generation unit 2123 provides this response information. By adding information such as "desu" to "1200 yen", the response content is generated by performing the process of adjusting the appearance as the answer (sentence). In addition to the case where the generation unit 2123 responds using only the language, the generation unit 2123 can also generate the response content using two types of language and image. When two types of language and image are used, the generation unit 2123 is a part of the response content using only the language so that the part where the response content can be expressed by the image is not included in the duplicate expression in the language. By omitting, it is also possible to generate a response content using two types of language and image. Specifically, there is a method of generating a response sentence using machine learning (using a known technique). There is also a method of completing a sentence by applying necessary information to a template created in advance via an external API (Application Programming Interface). In this embodiment, either method is used.
Further, when the generation unit 2123 holds the request and response database, the response content can be determined without going through the generation unit 2123. The result of the dialogue management unit 212 including the information of the generation unit 2123 is output to the presentation control unit 213.

(Configuration example of presentation control unit 213)
Returning to FIG. 5, the presentation control unit 213 determines the presentation method based on the information (response content) input from the dialogue management unit 212. As for the presentation method, for example, when the response content is output from the speaker, the response content can be output by voice, so the presentation control unit 213 decides to use voice as the presentation method. Further, when the response content is output using the display panel and the speaker, the response content can be output using the voice and the image, so it is decided to use the voice and the image. The presentation control unit 213 outputs the presentation information including the response content generated by the generation unit 2123 and the determined presentation method to the synthesis unit 214.

(Structure example of synthesis unit 214)
The synthesis unit 214 generates the processed content based on the presentation information input from the presentation control unit 213. For example, the synthesis unit 214 generates the content by generating the data corresponding to the presentation method of the response content generated by the generation unit 2123. The synthesis unit 214 performs voice synthesis processing on the response content (text data) of "1200 yen" when the presentation method is voice, and generates a voice file. Get content by doing. The compositing unit 214 transmits the generated content to the terminal device 10 by a communication function provided in the information presenting device 20.

Further, the information presenting device 20 obtains an explicit item (such as "Did it match? Yes / No", etc., in order to acquire the evaluation of each user for the response information output from the synthesis unit 214 and transmitted to the terminal device 10. ) Can also be sent to the user. The evaluation information from the user input to the terminal device 10 is stored in the action storage unit 224 of the information storage unit 220.

The presentation form by the information presenting device 20 can be arbitrarily changed by the user. These changes made by the user are also output as action information to the action storage unit 224 in the information storage unit 220.

Further, the acquisition unit 211, the dialogue management unit 212, and the synthesis unit 214 output the acquired input, the analysis result, and the information used in the optimization presentation to the dialogue history storage unit 221. For example, the acquisition unit 211 outputs the time, the user ID, the message, and the message ID to the dialogue history storage unit 221 in response to the input from the user. The dialogue management unit 212 outputs a recall-oriented phrase and a session ID to the dialogue history storage unit 221 for each dialogue. The synthesis unit 214 outputs the time, the user ID, the message, and the message ID based on the content to be output to the terminal device 10.

(Configuration example of dialogue history storage unit 221)
FIG. 7 is a diagram showing an example of a data structure of dialogue history data stored in the dialogue history storage unit 221 according to the embodiment of the present invention. The dialogue history storage unit 221 includes information on the time, the user ID, the message indicating the utterance content, the recall-oriented phrase, the session ID, the message ID (nth), and the message ID (n-1th), and includes information on the immediately preceding utterance, etc. Accumulate (memorize) the dialogue history as one of the components.
If the dialogue history storage unit 221 is a voice input to the terminal device 10, the utterance content is stored as a message even if the other party of the dialogue is the information presenting device 20, and a plurality of dialogue history storage units 221 are stored in the vicinity of the terminal device 10. If the user is interacting with, the dialogue is also stored as a message. For example, when the user A and the user B are interacting with each other in the vicinity of the terminal device 10, the spoken message can be classified for each user based on the user's voiceprint or the like. Then, each utterance content is stored as a message. Therefore, the information presenting device 20 is stored in the dialogue history storage unit 221 even when there is a request utterance "How much is the book that came out when I talked with Mr. A at the company last week?" You can search for answers based on the messages you received.

The dialogue history data stored in the dialogue history storage unit 221 may be stored for all the voices input to the terminal device 10, or by switching the input unit 110 of the terminal device 10 on / off. The dialogue content during the period when the input unit 110 is on may be stored. For example, when the input unit 110 is a microphone, the function of the microphone may be switched on or off. This switching may be based on a switch that is turned on and off by a user operation, or an operation command for turning on or off may be performed by voice input.
Further, when the input unit 110 is a keyboard, a mouse, or the like, text data or figures (emoticons, pictograms, stamps, etc.) input from the input unit 110 can be stored as dialogue history data as dialogue contents. it can.

The time represents the time spoken by the user from the input unit 110.
The user ID is identification information that identifies the user who made the utterance. Here, the user indicated by "U_001" is identification information that identifies a user who uses any of the terminal devices 10, and "C_001" is an information presenting device 20 that generates response information output from the terminal device 10. Represents. As this user ID, the user ID corresponding to the identification information assigned to the terminal device 10 that has input the content uttered by the user is used, and is written by the dialogue management unit 212. For the dialogue performed by outputting the content from the information presenting device 20, the user ID assigned to the self-information presenting device 20 is written by the synthesis unit 214. That is, it is possible to grasp a series of flows in which the information presenting device 20 responds to the request information from a certain user.

The message is information representing the utterance content obtained based on the content uttered by the user, and is written by the acquisition unit 211. For example, the message is text data.
The recall-oriented phrase stores information indicating whether or not the recall-oriented phrase is included in the utterance content based on the analysis result of the analysis unit 2121. For example, the analysis unit 2121 determines whether or not the information input from the acquisition unit 211 is a dialogue using the recall orientation, and if it is a dialogue using the recall orientation, an expression element corresponding to the recall orientation. Is stored in the dialogue history storage unit 221 as a recall-oriented phrase. Here, as an example, "yesterday" and "book" are memorized as recall-oriented phrases for the message "What is the price of the book I talked about yesterday?".
The session ID is identification information that identifies this dialogue group as one dialogue group from the start of one dialogue to the end of the dialogue. The session ID is written by the dialogue management unit 212.

The message ID (nth) is information for identifying a message, and different information is given to each message. The message ID (nth) also includes information indicating the order of the messages in the time series, and is, for example, a number determined to be arranged in ascending or descending order according to the time series.
The message ID (n-1st) indicates a message ID (nth) assigned to the message immediately preceding the message indicated by the message ID (nth) in chronological order.
By using the message ID (nth) and the message ID (n-1st), the order of the time series of the messages can be specified.
The message ID is written by the acquisition unit 112 for the message uttered by the user and by the synthesis unit 214 for the message by outputting the content from the information presenting device 20.

(Structure example of environmental storage unit 222)
FIG. 8 is a diagram showing an example of a data structure of the environmental storage unit 222 according to the embodiment of the present invention. The environment storage unit 222 stores the date and time, place, weather, etc. on the user side when the dialogue is performed for each user as one of the components. Specifically, the environmental storage unit 222 includes information on a session ID, a user ID, a date, a time (start), a time (end), a place, a weather, and a recall-oriented phrase.
The session ID is identification information that identifies this dialogue group as one dialogue group from the start of one dialogue to the end of the dialogue. If the sessions are the same, the same session ID as the session ID stored in the dialogue history storage unit 221 is used.
The user ID is information that identifies the user, and in particular, is a user ID that identifies the user who is using the terminal device 10. As this user ID, the ID assigned to the terminal device 10 can be used, and when the terminal device 10 is a smart speaker, the ID at the time of user registration when using this smart speaker is used. be able to.
The date represents the date on which the dialogue took place.
Time (start) indicates the time when the dialogue started.
Time (end) indicates the time when the dialogue ended.
The location indicates the position of the speaking user at the time of the dialogue. For example, when the user speaks at home, "home" is stored as the place, and when the user speaks in the conference room A, "meeting room A" is stored. The location may be specified, for example, by specifying the current position (for example, latitude and longitude) by the GNSS function of the terminal device 10 that speaks, and specifying the name of the place corresponding to the current position.
The weather indicates the weather in the area including the position of the user who spoke at the time of the dialogue. This weather information may be acquired from a weather information API, an external weather server, or the like. As for this weather, it may be temperature, humidity, etc. instead of the weather, as long as the environment at the place where the dialogue was held can be grasped.

(Configuration example of attribute storage unit 223)
FIG. 9 is a diagram showing an example of the data structure of the attribute storage unit 223 according to the embodiment of the present invention. The attribute storage unit 223 stores information about the user other than the dialogue (for example, the location of the user) as one of the components, including age and gender. Specifically, the attribute storage unit 223 includes information such as a user ID, age, gender, location, recall orientation, and reliability.
The user ID is identification information that identifies the user. This user ID is, in particular, identification information that identifies a user who makes a request utterance (question).
Age indicates the age of the user.
Gender is the gender of the user.
The location is the location of the user. These ages, genders, and locations can be used for the demographic data, but other information such as occupations can also be used.

The recall-oriented information is information based on the recall-oriented phrase extracted by the recall analysis unit 241 from the message stored in the dialogue history storage unit 221. For example, the recall-oriented phrase itself extracted by the recall analysis unit 241 may be used as the recall-oriented phrase, but here, a case where information based on the recall-oriented phrase is used will be described. When a recall-oriented phrase is stored as a recall-oriented phrase, a specific word is registered, so that the amount of data stored in the attribute storage unit 223 increases. In addition, even if the specific data of the date and time "3 days ago" and "4 days ago" and the specific data of the proper nouns "Mr. A" and "Mr. B" are stored separately, the amount of data can be saved. As an effect on the increase, it is not always possible to obtain a large effect. Therefore, the specific data of the date and time "3 days ago" and "4 days ago" are memorized as "date and time" as a recollection intention, and the specific data of the proper nouns "Mr. A" and "Mr. B" are stored. , You may memorize it as a "proper noun (personal name)" as a recollection orientation. As a result, it becomes possible to grasp the reliability of the recall-oriented phrase for each attribute such as "date and time" and "proper noun (personal name)". In FIG. 9, "proper nouns (books)" and "price" are illustrated as recall-oriented. The "proper noun (book)" indicates the attribute to which the name of the book belongs, and the "price" is an attribute indicating the price of goods and services.

The degree of reliability is a degree indicating the correctness of the user's perception of the recall orientation. In other words, the reliability indicates the degree to which the recall-oriented phrase indicated by the recall-oriented phrase used by the user can be trusted to obtain the information desired by the user. It can be said that the higher this degree is, the more reliable the recall orientation is, that is, the higher the value that can be used in the process of information processing used as an answer to a request utterance.
The reliability is shown here using any number from 0 to 100. The closer it is to 100, the higher the reliability, and the closer it is to 0, the lower the reliability. Here, the reliability of the recall-oriented "proper noun (book)" is 90, and the reliability of the "price" is 95. When comparing these two reliabilitys, the "price" The reliability of is higher.
By storing the user ID, the recall orientation, and the reliability in association with each other, it is possible to utilize as information the degree to which the recall orientation included in the request utterance can be trusted for the request utterance indicated by the user ID. It becomes. For example, for a user whose date and time tends to be ambiguous, the date and time period can be extended or changed to another period so that the search range can be expanded instead of the date and time itself included in the question.

In addition to the above, the attribute storage unit 223 may store information that can be acquired from the user's terminal device 10 with the permission of the user. For example, the usage history of the application used in the terminal device 10 may be stored as attribute information. When grouping users, the usage history of this application can be grouped so that users with similar application usage tendencies are in the same group.

(Structure example of behavior memory unit 224)
FIG. 10 is an example of the data structure of the behavior storage unit 224 according to the embodiment of the present invention. The action storage unit 224 stores the response content presented by the information processing unit 210 and the reaction (feedback) from the user indicating whether or not the response content is satisfied as the information desired by the user. For example, the action storage unit 224 stores a recall-oriented phrase used by the user, an evaluation regarding the response presented by the system, and the like. More specifically, the action storage unit 224 stores information including time, user ID, action ID, recall-oriented phrase, execution content, message ID, search result, and feedback information.
The recall-oriented phrase indicates the recall-oriented phrase used to perform the search.
The content of the implementation is data showing the recall orientation used when conducting the search. The content of the implementation includes information indicating which recall-oriented phrase and how the search range was expanded (changed) when the search was performed after the search range was expanded.
The action ID is information given for each process in which the search is performed, and is information for identifying the search process.
The search result is information indicating whether or not the search was obtained from the environmental storage unit 222 as a result of the search based on the implementation content. "False" indicates that no answer was obtained, and "True" indicates that the answer was not obtained. Indicates that it was obtained.
The feedback is information indicating the result obtained from the user as to whether or not the answer to the request utterance is transmitted to the user's terminal device 10 and the answer is satisfied as the answer to the request utterance, and "True" is satisfied. Indicates that "False" was not satisfied.

When the request utterance received from the terminal device 10 includes a recall-oriented phrase and the information presenting device 20 outputs the content as a response to the recall-oriented phrase, the action storage unit 224 receives the content for the user. Whether the information was useful (whether the answer was satisfactory to the request) is stored as a history for each message.
For example, here, "last week" and "Mr. A" are extracted as recall-oriented phrases in the request utterance from the user ID "U_001". "Reference from a person (member)" indicating that the answer was searched using "member)" is memorized, and "False" indicating that the answer was not found even after performing this search is memorized. To. Regarding the request utterance from the user ID "U_002", the range of the recall-oriented phrase related to the date and time of the recall phrase included in the request utterance was changed to "two weeks ago" and the search was performed again. The content of the implementation "Expand the range of date and time and retry" is memorized, and as a result of the search, "True" indicating that the answer is found is memorized, and the content based on the answer is sent to the user. However, "True" indicating that feedback was obtained indicating that a satisfactory answer was obtained is stored.

(Structure example of grouping storage unit 225)
FIG. 11 is a diagram showing an example of a data structure of the grouping storage unit 225 according to the embodiment of the present invention. The grouping storage unit 225 stores the result of grouping for each user. As a simple method, there is a method of finding a shared item from an individual or multiple viewpoints and flagging it based on a numerical value of the dialogue history, age, gender, and the like.
In this figure, it is data in which a group ID, a user ID, an age, a gender, and the like are associated with each other.
The group ID is information that identifies the group. Users with the same group ID indicate that they belong to the same group.
The user ID is information that individually identifies the user.
Age is the age of the user.
Gender is the gender of the user.

(Structure example of recall-oriented management unit 240)

Here, in grouping, apart from the grouping shown in FIG. 11, a method of grouping based on the tendency of the words recalling the memory as shown in FIG. 13 can also be used.

The grouping estimation unit 231 executes the grouping process by using the information of the information storage unit 220. For example, the grouping estimation unit 231 uses the information (age, gender, etc.) stored in the attribute storage unit 223 of the information storage unit 220 to select one user whose age is "30's" and "male". Grouping is performed so that users who are in their "twenties" and who are "female" are grouped into one group. It is also possible to use machine learning in addition to the above method. For example, grouping may be performed by performing clustering classification, which is a kind of unsupervised learning, based on attribute information and behavior information.

(Structure example of language knowledge storage unit 226)
FIG. 12 is a diagram illustrating a configuration example of the language knowledge storage unit 226 according to the embodiment of the present invention.
Language knowledge data is stored in the language knowledge storage unit 226.
Linguistic knowledge data includes data items of words, part of speech, superordinate concepts, and subordinate concepts.
For example, the part of speech of the word "Rokuho Zensho" is "proper noun", the superordinate concept is "book", and the subordinate concept is "Rokuho Zensho XX year edition".

FIG. 13 is a diagram showing a configuration example of the recall-oriented management unit 240 according to the embodiment of the present invention. The recall-oriented management unit 240 estimates the user's recall orientation and determines the guidelines for dialogue rule construction and dialogue management. The recall-oriented management unit 240 includes a recall analysis unit 241 and a model construction unit 242.

The recall analysis unit 241 collects phrases (words) related to memory recall from the dialogue history storage unit 221 and analyzes the collected information (words) based on the collected information (words). Estimate words and phrases similar to environmental information from. Further, the recall analysis unit 241 may convert them into feature quantities and hold them for each user or group. Here, memory recall is a process of recalling a specific memory from the memories stored in the brain. If you are aware of a specific event but cannot remember its name, etc., you can say that you are in a state where you can remember but cannot recall it.
The recall analysis unit 241 refers to the dialogue history stored in the dialogue history storage unit 221 and extracts a recall-oriented phrase from the message included in the dialogue history. Here, the words included in the message may be extracted and used as a recall-oriented phrase.
In addition, the recall analysis unit 241 analyzes based on the words contained in the message, determines whether or not the utterance is related to recall from the dialogue history, and from the message determined to be the utterance related to recall, the environment. A word or phrase in an environment similar to the environment stored in the storage unit 222 is extracted. Words and phrases similar to the environment refer to items such as where the message was uttered and what the weather was when the message was uttered, and are related to the message contained in the dialogue history storage unit 221. Extract words and phrases that can be judged to be high.

In addition, the recall analysis unit 241 performs a search using the recall-oriented phrase, and depending on whether or not a desired answer is obtained for the user, the reliability of the recall-oriented phrase for the recall-oriented phrase used in the search is high. Is calculated or updated. For example, when the recall analysis unit 241 finds the answer requested by the user as a result of the search, the reliability is updated so as to maintain the reliability or increase the degree of reliability, and when the answer requested by the user is not found. Update the reliability so that the reliability is reduced. As for whether or not the answer requested by the user is found, the value indicated by the feedback associated with the recall-oriented phrase stored in the action storage unit 224 may be used.

Further, the recall analysis unit 241 refers to the relationship between the word stored in the language knowledge storage unit 226 and the part of speech, and extracts the word associated with the pronoun included in the message. For example, the linguistic knowledge storage unit 226 stores the relationship between words and part of speech. The recall analysis unit 241 identifies the part of speech of the word included in the message by referring to the language knowledge storage unit 226. Then, when the word included in the utterance content is a pronoun, the recall analysis unit 241 extracts the word associated with the pronoun and the recall-oriented phrase by performing morphological analysis and dependency analysis on the word that is the pronoun. To do. For example, pronouns are often omitted in dialogues. Therefore, if there is an utterance "Where did you sell it?" After the utterance "What is the price of the book you talked about before?" By associating the phrases "before (previous dialogue)" and "book (proper noun)", it can be used as if there was an utterance "Where did you sell the book you talked about before?" it can. In this way, even if pronouns are omitted in the dialogue, the phrase used as a recall-oriented phrase is linked from the dialogue before and after the remark that seems to be continuous or continuous. Then, the abbreviated pronouns and the like can be supplemented.

Based on the information of the information storage unit 220, the model construction unit 242 constructs and updates mathematical formulas and rules necessary for estimating the recall orientation for each user and group as a model. In addition, the model building unit 242 builds a model (memory recall estimation model) for estimating parameters related to the user's memory recall.

In addition, the model construction unit 242 can also use a trained model learned by machine learning, reinforcement learning, or the like when constructing a memory recall estimation model. In that case, the model construction unit 242 prepares a basis function for parameter estimation based on the information stored in the information storage unit 220, and uses the information stored in the information storage unit 220 as teacher data for the memory recall estimation model. Perform optimization.
Here, optimization is a concept broader, narrower, or similar than the concept indicated by the recall-oriented phrase so that the user can obtain the answer desired by searching using the recall-oriented phrase based on the recall-oriented phrase. It is a process to change to a recall-oriented phrase with a different target range. To change the recall-oriented phrase, when changing the concept indicated by the recall-oriented phrase to be broader, when changing the target range indicated by the recall-oriented phrase, reselecting the recall-oriented phrase so that it becomes a similar recall-oriented phrase. If there is.

The model construction unit 242 can have a plurality of models such as a unique model which is a model specialized for each user and a general-purpose model which is a model that can be used for a similar user group without specifying an individual. is there. Since the accumulated data is small for users who have just started using the system, it is possible to select a general-purpose model having the most similar tendency from the general-purpose models and perform processing. For example, referring to the information stored in the attribute storage unit 223, at least a part of the attributes such as age, gender, and location can be selected from the user group such as user A, user B, user C, and so on. It may be used to group and select a general-purpose model of users with similar tendencies. In addition, the degree of correctness of the user's recognition of the recall orientation, which is required based on the degree of trust in the recall orientation (often mistaken for the recall orientation "date and time", a little more often for the recall orientation "price", etc. Grouping may be performed according to a certain reliability), and a model of a user with a similar reliability may be selected.
Here, the attribute storage unit 223 stores the recall-oriented phrase and the reliability of the recall-oriented phrase in association with the user ID. The recall-oriented phrase and the reliability stored in the attribute storage unit 223 are obtained by the recall analysis unit 241 of the recall-oriented management unit 240. When the recall analysis unit 241 searches using the recall-oriented phrase and finds an answer, the recall analysis unit updates the reliability so as to maintain the reliability or increase the reliability, while the answer is not found. If so, update the reliability so that the degree of reliability of the recall-oriented phrase decreases. By updating the reliability in this way, the degree of reliability of the recall-oriented phrase can be brought closer to the actual user's behavior and orientation, so that the recall-oriented phrase can be optimized. ..

As an example of optimization using the model of the model construction unit 242, as shown in FIG. 14, "Last week" and "Mr. A" 2 such as "Where was the store that Mr. A went to last week?" It is assumed that a recall utterance using two recall-oriented phrases is presented (reference numeral (a)).

When there is no information corresponding to the environmental memory unit 222 regarding the recalled utterance (utterance using the recalled phrase), for example, from the past dialogue of the user, among the recalled "date and time" and the recalled "person (member)". If the recall-oriented "date and time" seems to have more ambiguous parts (low reliability), the model construction unit 242 considers the possibility that the user misidentifies the date and time part, and the recall-oriented "date and time" By changing "last week", which is a recall-oriented phrase corresponding to "date and time", to, for example, "two weeks ago", the period is changed so that "two weeks ago" is the target period to be searched. The dialogue management unit 212 performs dialogue processing using the utterance content after the recall-oriented phrase corresponding to this "date and time" is changed (reference numeral b).
In addition, the model construction unit 242 optimizes when the recall-oriented phrase corresponding to the answer to the request utterance is not stored in the dialogue history storage unit 221, and the recall-oriented phrase corresponding to the answer to the request utterance is Even if it is stored in the dialogue history storage unit 221, if it is not the answer requested by the user, optimization may be performed.
For example, in FIG. 14 (reference numeral a), the answer obtained by performing the search using the recall-oriented phrase “last week” was transmitted to the user's terminal device 10, and the answer requested by the user was not obtained from the terminal device 10. Evaluation results (feedback) indicating that may be obtained. In such a case, as shown in FIG. 14 (reference numeral b), the target range to be searched is changed from "last week" to "two weeks ago" to perform optimization. May be good.
The answer obtained by re-searching by performing this optimization is evaluated again by the user, and when the evaluation result indicating that the answer requested by the user is obtained is obtained, it is referred to as "date and time". It can be estimated that there was an error in the information presented by the user regarding the "date and time" among the "persons (members)". In such a case, the reliability of the user's recall-oriented "date and time" is lowered. Therefore, the recall analysis unit 241 updates the reliability of the recall-oriented "date and time" of the user who made the request utterance to a value lower than the current value. As a result, the dialogue management unit 212 will select the "date and time" and the "person (member)" when performing the first search among the recall-oriented "date and time" and the recall-oriented "person (member)" when searching next time. If the reliability of "date and time" is lower than that of "person (member)", the search is performed using the recall orientation of "person (member)" without using "date and time". Alternatively, it is also possible to perform processing such as searching after changing to a recall-oriented phrase that widens the period indicated by the recall-oriented phrase corresponding to the "date and time".

In addition, as another method of changing the search range based on the recall-oriented, for example, the recall-oriented phrase related to the recall-oriented "date and time" is fixed, and the recall-oriented phrase related to the recall-oriented "person (member)" is spoken on the same day. By replacing it with a recall-oriented phrase that indicates a member of (for example, replacing it with "Mr. B" who belongs to the same group instead of "Mr. A"), or by using multiple recall-oriented phrases to be changed (date and time and person (member) It can also be said that the replacement is carried out with both). As a result, the target range to be searched can be expanded or the target range can be changed.
In addition, when the reliability is updated using the evaluation result and the search range is changed in advance based on the reliability, the search is performed by the shortest route in order to obtain the answer requested by the user. Becomes possible.

In the information storage unit 220, new dialogue information is accumulated in the dialogue history storage unit 221 each time the user engages in a dialogue, and based on this new dialogue, the environment storage unit 222 and the attribute storage unit 223. , New information is added to the action storage unit 224 and the grouping storage unit 225, and the stored information is updated. Therefore, the model construction unit 242 can update the data model (memory recall estimation model) recalled based on the new information. The update frequency etc. can be set arbitrarily by the administrator.

Further, the memory recall estimation model constructed by the model construction unit 242 can be a single model or a combination of a plurality of models or can be constructed separately. Further, the dialogue rule construction unit 250 also enables processing using a single or a plurality of memory recall estimation models constructed by the model construction unit 242.

(Structure example of dialogue rule construction unit 250)
FIG. 15 is a diagram showing a configuration example of the dialogue rule construction unit 250 according to the embodiment of the present invention. The dialogue rule construction unit 250 constructs the dialogue rule by utilizing the information of the recall-oriented management unit 240 and adding and modifying the dialogue rule. Specifically, the dialogue rule building unit 250 includes a speech building unit 251 and a relationship building unit 252.
The utterance construction unit 251 constructs the requested utterance from the user by using the information storage unit 220.
The utterance construction unit 251 stores a sentence template (template) in advance. The template is, for example, data representing the format of the sentence "Is B of A C?".
The relationship building unit 252 links the request utterance and the response utterance.

As an example of the utterance construction unit 251 according to the embodiment of the present invention, as the most naive method, the environment storage unit 222 is associated with the information of the dialogue history storage unit 221 for each user based on the information of the information storage unit 220. There is a method of applying the information of. For example, the linked information can be obtained by searching using both the session ID and the user ID as keys.

When the above model is used, the registered dialogue rules are replaced with the words and phrases registered in the environmental information. For example, if the dialogue rule "What is the price of the Rokuho Zensho?" Is associated with the recall orientation of "date and time" and "member" as environmental information, "what is the price of the book I talked about yesterday?" A possible method is to replace and register, such as "What is the price of the book I talked to Mr. A?" For example, the utterance construction unit 251 uses the template "What is [G] of [D] [E] [F]?". The template may be stored in advance by the utterance construction unit 251. Here, in the template, [D] is associated with the data of the recall-oriented "date and time", [E] is associated with the part of speech "verb", [F] is associated with the part of speech "noun", and [G] is associated with the data of "book proper noun". ing. Then, based on such a template, by applying the contents of the environmental information to each of [D], [E], [F], and [G], "What is the price of the book I talked about yesterday?" You can compose the sentence "What is the departure time of the train you checked last week?" And replace the sentence "What is the price of the Rokuho Zensho?" With this composed sentence. The utterance construction unit 251 does not replace the word associated with the reply. For example, the word "1,200 yen" may correspond to a reply related to "price", and the word "9:30" may correspond to a reply related to "departure time", so such a word is not replaced. The utterance construction unit 251 additionally stores the sentence generated by the replacement in the dialogue rule storage unit 227.

FIG. 16 is a diagram showing an example of dialogue rule data stored in the dialogue rule storage unit 227.
The dialogue rule data stores the dialogue rule and the recall orientation. For example, as an example of dialogue rule data, a recall-oriented "date and time" and a "member" are associated with the dialogue rule "What is the price of the Six Codes?". In addition, the recall-oriented "price" is associated with the dialogue rule "1,200 yen" and is stored.

When the speech construction unit 251 constructs a dialogue rule by the above method, the utterance construction unit 251 may have an ontroge or a thesaurus that represents a superordinate concept or a subordinate concept of a word registered in the language knowledge storage unit 226, which is a component of the information processing unit 210. It is also possible to use the data to replace it separately from the environmental information. For example, the utterance construction unit 251 ranks the words of the higher concept corresponding to the "Rokuho Zensho" among the words included in the dialogue rule "Rokuho Zensho" by referring to the language knowledge storage unit 226. You can get the concept word "book". The utterance construction department 251 uses the obtained words to replace "Rokuho Zensho" with "book" in the dialogue rule "Rokuho Zensho?" To change the dialogue rule "Book price?" Can be generated. The utterance construction unit 251 additionally stores the generated dialogue rule in the dialogue rule storage unit 227.

In addition, by storing the dialogue rules as described above, it is possible to perform template matching using tree search or the like, but the past dialogue history can be identified as learning data without using a specific sentence format. By creating a vessel, it is possible to perform the same processing as template matching.

The relationship building unit 252 associates the message stored in the dialogue history storage unit 221 with the response utterance corresponding to the response of the request utterance constructed by the utterance building unit 251.
For example, the relationship building unit 252 corresponds to the response of the request utterance constructed by the utterance building unit 251 from the message "What is the price of the book that was talked to Mr. A the other day?" Stored in the dialogue history storage unit 221. When the environmental storage unit 222 is searched (searched) from the contents of "before", "Mr. A", and "book", the book is picked up. Check which books are in order, and uniquely determine the books based on the feedback from users. Then, the request utterance is converted into "What is the price of XX (book)?", And if it is registered as a dialogue rule between the registered request and response, the price can be withdrawn.

(System operation)
Next, an example of the operation of the information presentation system will be described with reference to the drawings.

FIG. 17 is a flowchart showing an example of the operation of the information presentation process of the information presentation system according to the embodiment of the present invention.

At the start of the dialogue, the information presenting device 20 identifies the user ID based on the identification information assigned to the terminal device 10 that collects the dialogue contents (step S101), and the dialogue history storage unit 221 is stored based on the user ID. Based on whether or not there is a message associated with the user ID with reference to the user ID, it is determined whether or not there is a usage history of the system in the past (step S102). Further, even if the information presenting device 20 has a usage history, the reliability score has not been calculated yet, the recall phrase has not been used yet, and the reliability cannot be calculated without responding to the evaluation request from the system. If any of the above is true, it may be determined that there is no usage history because it is considered insufficient. The information presenting device 20 reads the unique model and the general-purpose model related to recall orientation if there is a history of participation in the past (step S103), and reads only the general-purpose model if there is no usage history because the unique model is not formed. (Step S104).

After reading the model data, the information presenting device 20 enters the input standby state and determines whether or not an action (utterance) has been performed by the user (step S105). When the utterance is acquired (step S105-YES), the dialogue processing is started (step S106), the utterance content is stored in the dialogue history storage unit 221, and on the other hand, when the utterance is not acquired (step S105-NO), After a certain period of time has elapsed, the determination in step S105 is performed again. When the dialogue processing is started, the information presenting device 20 determines whether or not the dialogue is a recall-oriented dialogue by the analysis unit 2121 (step S107). When the dialogue does not use the recall orientation (step S107-NO), the analysis unit 2121 stores the utterance content in the dialogue history storage unit 221 (step S140), and proceeds to step S109.
On the other hand, in the case of the dialogue using the recall orientation (step S107-YES), the action selection unit 2122 searches for the response information corresponding to the request information included in the dialogue content from the environmental storage unit 222 (step S108). As a result of the search, if there is a matching content (step S109-YES), the matching content is acquired as response information. When the response information is acquired, the generation unit 2123 determines the response content based on the response information (step S110). When the response content is determined, the information presenting device 20 outputs the content based on the response information according to the response content (step S111).
The information presenting device 20 acquires an evaluation from the user (information indicating whether or not the answer requested by the user has been obtained) for the response content (content) from the terminal device 10 (step S112), and acts using the evaluation result as action information. It is updated by writing to the storage unit 224 (step S113). Here, it is determined whether or not the answer requested by the user has been obtained, the process is terminated when the answer requested by the user is obtained, and when the answer requested by the user is not obtained, step S150. You may want to move to.

On the other hand, if there is no matching content as a result of the search (step S109-NO), the action selection unit 2122 is optimized by the model construction unit 242 to be recollection-oriented and constructed by the recollection-oriented management unit 240. The optimum response content is determined using the model (step S150).

Here, in the optimization process, if necessary, the presentation form is determined and applied, and the response content is determined. When the response content is determined, the content based on the response content is transmitted to the user's terminal device 10 (step S151). Based on the transmitted result, the response content is updated by adding the response content to the dialogue information, and the recall-oriented phrase and the implementation content are updated by writing in the action memory unit 224 (step S153), and the process returns to the input standby in step S105. ..

In step S150, when it is determined that the reference is by recall in the dialogue process, but there is no dialogue rule that matches the corresponding recall word and phrase, the unique model and the general-purpose model are used. These are models of the tendency of recall, and the system side presents such as "Maybe XXX?" In consideration of memory differences such as location, person (member), date and time. For example, if there is no dialogue rule for the request utterance "Where is the store that Mr. A said last week?", By using a unique model or a general-purpose model, you can replace it with a higher-level conceptualization of words or another concept. Content based on response information such as "maybe two weeks ago?" Is transmitted to the terminal device 10.

When the above processing is performed, the action information is updated as an action on the system side. When this action is taken, the user's feedback is obtained by presenting "Did you meet ?, Yes / No" at the break of the dialogue.

After updating the behavior information and the dialogue information, the recall-oriented model can be updated sequentially. Also, depending on the machine specifications, it is possible to perform batch processing.

The terminal device 10 or the information presentation device 20 in the above-described embodiment may be realized by a computer. In that case, the program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client in that case. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized by using a programmable logic device such as FPGA (Field Programmable Gate Array).

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs and the like within a range that does not deviate from the gist of the present invention.

1 ... Information presentation system, 10, 10a, 10b, 10c, 10d ... Terminal device, 20, 20a, 20b, 20c, 20d ... Information presentation device, 30 ... Communication network, 110 ... Input unit, 112 ... Acquisition unit, 120 ... Interface unit, 130 ... storage unit, 140 ... sensor unit, 150 ... output unit, 160 ... communication unit, 161 ... transmission unit, 162 ... receiver unit, 170 ... control unit, 210 ... information processing unit, 211 ... acquisition unit, 212 ... Dialogue management unit, 213 ... Presentation control unit, 214 ... Synthesis unit, 220 ... Information storage unit, 221 ... Dialogue history storage unit, 222 ... Environmental storage unit, 223 ... Attribute storage unit, 224 ... Behavioral memory unit, 225 ... Grouping Memory unit, 226 ... Language knowledge memory unit, 227 ... Dialogue rule storage unit, 231 ... Grouping estimation unit, 240 ... Recollection-oriented management unit, 241 ... Recollection analysis unit, 242 ... Model construction unit, 250 ... Dialogue rule construction unit, 251 ... Speaking construction unit, 252 ... Relationship building unit, 1000 ... Terminal device, 2121 ... Analysis unit, 2122 ... Action selection unit, 2123 ... Generation unit

Claims (15)

It is an information presentation system that enables dialogue rule construction and dialogue control based on the recall orientation that expresses the intention of memory recall based on the user's dialogue history and environmental information.
A dialogue management unit that performs dialogue processing on the terminal device used by the user,
A dialogue history storage unit that stores dialogue history, which is the history of dialogue,
An environmental memory unit that stores the environment in which the dialogue was conducted, and the recall-oriented phrases and recall-oriented phrases contained in the dialogue.
When the concept of the recall-oriented phrase included in the user's request utterance obtained by performing the dialogue process and the concept of the recall-oriented phrase corresponding to the recall-oriented phrase are changed in order to obtain the response content to the requested utterance. Implementation content that expresses the content searched by changing the recall orientation, and
Feedback indicating whether or not the response content was satisfied with the information desired by the user, and
Behavior memory unit that memorizes
In order to obtain information including the response content that the user wants, the recall-oriented phrase indicated by the user's recall-oriented phrase is based on the recall phrase stored in the action memory unit, the implementation content, and the feedback. The recall-oriented management department that calculates the reliability, which indicates the degree of reliability,
Information presentation system equipped with. The recall-oriented management department
The information presentation system according to claim 1, wherein a recall-oriented phrase corresponding to environmental information similar to the environment of the dialogue is acquired based on a recall-oriented phrase included in the dialogue stored in the dialogue history storage unit. Claim 1 or claim 1 having a dialogue rule construction unit that generates a dialogue rule by applying a word included in the dialogue content stored in the dialogue history storage unit to a sentence template and a word stored in the environment storage unit. The information presentation system according to claim 2. The dialogue management department
The information presentation system according to any one of claims 1 to 3, which determines a response utterance to a request utterance according to the recall orientation calculated by the recall orientation management unit. The information presentation system according to any one of claims 1 to 4, wherein the environmental storage unit stores ambient environmental information including members, weather, place, and date and time. Claims 1 to 1 include an analysis unit that obtains numerical data according to the dialogue content by using any of morphological analysis, keyword extraction, and vectorization for the dialogue content stored in the dialogue history storage unit. The information presentation system according to any one of 5. It has an attribute storage unit that stores demographic data including the user's age, gender, and place of residence, and recall orientation.
The recall-oriented management unit performs the recall-oriented optimization process by changing the recall-oriented phrase of the request utterance so that the answer requested by the user can be obtained by using the information stored in the attribute storage unit. The information presentation system according to any one of claims 1 to 6. The information presentation system according to any one of claims 1 to 7, wherein the recall-oriented management unit writes and stores the reliability of the recall-oriented in the attribute storage unit for each user. The information according to any one of claims 1 to 8, wherein the recall-oriented management unit updates the reliability of the recall-oriented based on the search result and feedback stored in the behavior memory unit. Presentation system. Grouping estimation unit that groups based on user characteristics,
A grouping storage unit that stores the results grouped by the grouping estimation unit, and a grouping storage unit.
From claim 1, which includes a model construction unit that uses a model for estimating the recall orientation of users who have a common recall orientation of users to be changed based on the grouping result stored in the information of the grouping storage unit. The information presentation system according to any one of claims 9. The information presentation system according to claim 10, wherein a model is used in which dialogue rule construction and dialogue management are each modified from the grouping storage unit according to the recall orientation common to the users. The recall-oriented management department
Claims 1 to generate a model for optimizing dialogue rule construction and dialogue management for a user whose model is not prepared or insufficient, using a model having a tendency similar to that of the user. The information presentation system according to any one of 11. The information presentation system according to any one of claims 1 to 12, which has a compositing unit that generates content by processing information including response content. The synthesis part
The information presentation system according to claim 13, further comprising a structure for generating contents that have been processed differently according to a destination terminal device. It is an information presentation method in an information presentation system that can construct dialogue rules and control dialogue based on the recall orientation that expresses the intention of memory recall based on the user's dialogue history and environmental information.
The information presentation system
A dialogue history storage unit that stores dialogue history, which is the history of dialogue,
An environmental memory unit that stores the environment in which the dialogue was conducted, and the recall-oriented phrases and recall-oriented phrases contained in the dialogue.
When the concept of the recall-oriented phrase included in the user's request utterance obtained by performing the dialogue process and the concept of the recall-oriented phrase corresponding to the recall-oriented phrase are changed in order to obtain the response content to the requested utterance, the recall is concerned. Implementation content that represents the content searched by changing the orientation, and
Feedback indicating whether or not the response content was satisfied with the information desired by the user, and
Behavior memory unit that memorizes
Have,
The dialogue management department performs dialogue processing on the terminal device used by the user,
Based on the recall phrase stored in the action memory unit, the implementation content, and the feedback, the recall-oriented management unit determines the response content that the user wants as the recall-oriented phrase indicated by the user's recall-oriented phrase. An information presentation method that calculates the degree of reliability that indicates the degree of reliability in order to obtain the information that is included.

Images