JP2018041494A - Control device, control method, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a control device, a control method, and a control program for outputting a message suitable for a user.SOLUTION: A control device (response generation device 100) includes: a search section which searches for advertisement information, which is information on an advertisement to be output as a response message, within a search range on the basis of characteristics of sound in interactions between an interactive agent system and a user; and an output control section which controls the advertisement on the advertisement information searched by the search section to be output as a response message. The control device outputs a message suitable for a user.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a control device, a control method, and a control program.

  2. Description of the Related Art Conventionally, there is known an interactive agent system that outputs a message corresponding to a message to the user terminal in response to receiving the message from the user terminal. In such an interactive agent system, there is provided a technique for outputting various messages in response to a message received from a user terminal and outputting a message suitable for the user of the user terminal.

  For example, based on the user interest category determined based on the user's behavior history on the web, and the message category corresponding to each node, the conversation agent according to the user's preference without imposing a burden on the user Techniques for changing the system interaction rules have been proposed.

JP 2013-122012 A JP 2008-299753 A

  However, the above-described conventional technology does not always output a message suitable for the user. For example, in the above-described conventional technology, there is a possibility that a user who does not like a useless talk may be given an unpleasant feeling by outputting a response message out of the conversation context. Thus, in the above-described conventional technology, a message suitable for the user is not always output.

  The present application has been made in view of the above, and an object thereof is to provide a response generation apparatus capable of outputting a message suitable for a user.

  The control device according to the present application includes: a search unit that searches for advertisement information that is information related to an advertisement that is output as the response message in a search range based on a sound characteristic in a conversation between a dialog agent system and a user; and the search unit An output control unit that controls to output an advertisement related to the searched advertisement information as a response message.

  According to one aspect of the embodiment, there is an effect that a message suitable for the user can be output.

FIG. 1 is a diagram illustrating an example of a response generation process according to the embodiment. FIG. 2 is a diagram illustrating a configuration example of the response generation system according to the embodiment. FIG. 3 is a diagram illustrating a configuration example of the advertising bid device according to the embodiment. FIG. 4 is a diagram illustrating an example of the advertisement information storage unit according to the embodiment. FIG. 5 is a diagram illustrating a configuration example of the response generation device according to the embodiment. FIG. 6 is a diagram illustrating an example of the determination information storage unit according to the embodiment. FIG. 7 is a schematic diagram of a tree structure stored in the determination information storage unit according to the embodiment. FIG. 8 is a diagram illustrating an example of a search information storage unit according to the embodiment. FIG. 9 is a sequence diagram illustrating a response generation processing procedure performed by the response generation apparatus according to the embodiment. FIG. 10 is a hardware configuration diagram illustrating an example of a computer that implements the function of the response generation apparatus.

  Hereinafter, a mode for carrying out a control device, a control method, and a control program according to the present application (hereinafter referred to as “embodiment”) will be described in detail with reference to the drawings. In addition, the control apparatus, the control method, and the control program concerning this application are not limited by this embodiment. In the following embodiments, the same portions are denoted by the same reference numerals, and redundant description is omitted.

[1. Response generation process)
First, an example of a response generation process executed by the control device according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a response generation process according to the embodiment. As shown in FIG. 1, in the present embodiment, the control device includes a voice recognition device 20 and a response generation device 100.

  The user terminal 10 is a terminal device used by a user. The user terminal 10 is, for example, a mobile phone such as a smartphone, a tablet terminal, a PDA (Personal Digital Assistant), a desktop PC (Personal Computer), a notebook PC, or the like.

  The speech recognition device 20 converts the input message received from the user terminal 10, that is, speech speech data into text data. In addition, the voice recognition device 20 analyzes voice data received from the user terminal 10 and specifies information about the user as sound feature information. Specifically, the voice recognition device 20 generates a voice waveform by voice data analysis. Then, the voice recognition device 20 specifies information related to the user who has input the voice based on the peak shape, frequency characteristics, and the like of the generated voice waveform.

  The user information as the sound feature information is various information including information on user attributes such as the user's age, gender, emotion, dialect, physical condition, and peripheral information obtained from environmental sounds around the user. For example, the speech recognition apparatus 20 can determine a character such as “impatient” if the sound tempo or the interval between words is shorter than a predetermined value, and “occupy” if it is longer, based on the speech waveform. Further, the speech recognition device 20 can determine the age, sex, and dialect from the vibration width, peak shape, and frequency in the speech waveform.

  Furthermore, the speech recognition apparatus 20 can also determine a physical condition that is difficult to determine from text information. For example, the voice recognition device 20 can determine that the user is “cold” when a voice waveform characterizing the coughing sound is obtained from the user. Note that the speech recognition apparatus 20 may specify user attributes from speech using any method other than the processing described above.

  The voice recognition device 20 can also determine the peripheral information around the user U01. Specifically, since the user terminal 10 acquires not only a human voice but also surrounding environmental sounds as back noise, as a result, the voice recognition device 20 not only has a voice waveform related to a human voice, An audio waveform related to the surrounding environmental sound can also be generated. For example, when the speech waveform generated from the back noise corresponds to a child's cry, the speech recognition apparatus 20 can determine peripheral information such as “there is a small child” for the user U01. Further, for example, when the back noise includes a train running sound, a station announcement, a departure bell, or the like, the voice recognition device 20 can determine peripheral information such that the user U01 is at the station. Note that the speech recognition apparatus 20 may determine the surrounding information of the user from the environmental sound using any method other than the above-described processing.

  The advertiser terminal 30 is a terminal device used by the advertiser. The advertiser terminal 30 is, for example, a mobile phone such as a smartphone, a tablet terminal, a PDA, a desktop PC, a notebook PC, or the like. Further, the advertiser terminal 30 transmits the advertisement information received from the advertiser to the advertisement bidding device 40.

  The advertisement bidding device 40 presents a bid screen to the advertiser terminal 30. Further, the advertising bidding device 40 stores the advertising information received from the advertiser terminal 30 in a predetermined storage unit.

  The response generation apparatus 100 realizes a dialogue by controlling the output of the response message in accordance with preset dialogue information with respect to an input message that is a user's utterance.

  Hereinafter, in the response generation system 1 according to the embodiment, an example will be described in which advertisement information is searched for and output-controlled as a response message based on user information obtained from the nature of a voice waveform of a user input message.

  First, when the user terminal 10 accepts an input of a message such as “Kima Curry, I love you very much” from the user U01 who is the owner (step S11), the user terminal 10 transmits the voice data to the voice recognition device 20 ( Step S12).

  The voice recognition device 20 converts the voice data of the utterance received from the user terminal 10 into text data, analyzes the voice data received from the user terminal 10, and as feature information indicating the characteristics of the sound included in the voice data, User information is specified (step S13). Here, it is assumed that the voice recognition device 20 specifies the user information “Osaka dialect”.

  Then, the voice recognition device 20 transmits the text data and the specified user information “Osaka dialect” to the user terminal 10 (step S14). The user terminal 10 transmits the received text data and user information “Osaka dialect” to the response generation device 100 (step S15).

  And the response production | generation apparatus 100 performs the search process for searching the advertisement information used as a response message (step S16). Specifically, the response generation device 100 stores the user information and the search tag information in association with each other in advance in a predetermined storage unit, for example, when the user information “Osaka dialect” is received. Specifies “Osaka region” as the corresponding search tag. The search tag indicates various search conditions. That is, the search target and the search range vary depending on the search tag. The search target is information corresponding to a search query when searching for advertisement information, such as, for example, the Osaka region or food-related advertisement information. The search range is information indicating, for example, to what extent the similarity during advertisement search is expanded.

  As will be described in detail later, the response generation apparatus 100 supports not only user information related to dialects but also user information related to age, gender, personality, emotion, physical condition, environmental sound, and the like, and search tags characterizing each user information. In addition, it is stored in advance in a predetermined storage unit. For example, the response generation device 100 stores the gender “female” and the search tag “cosmetics” in association with each other, or stores the personality “dislike talk” and the search tag “narrow search” in association with each other. ing. Note that the response generation device 100 does not necessarily have such a storage unit. For example, the received user information itself may be used as a search tag.

  In addition, as a search process, the response generation apparatus 100 sets a predetermined keyword included in the input message of the user U01 as a search keyword. Here, since the user U01 has input “Kima Curry, I really like you”, the response generation apparatus 100 sets “Kima Curry” as a search keyword.

  Then, the response generation device 100 acquires the advertisement data included in the advertisement information that matches the search keyword and the search tag from the advertisement bidding device 40 (step S17).

  For example, if the advertising bidding apparatus 40 receives “advertising keyword: keema curry, advertising tag: Osaka region, advertising data: curry shop A in Osaka!” As advertising information from the advertiser A store, the search keyword And the advertising keyword match, and the search target “Osaka region” set as the search tag matches the advertising tag “Osaka region”. That is, since the advertisement information of the advertiser A store satisfies the search keyword and the search tag, the advertising bidding apparatus 40 has the advertisement data “curry store A store Osaka advance!” Included in the advertisement information of the advertiser A store! Is presented to the response generation device 100.

  Then, the response generation device 100 acquires the advertisement data presented by the advertisement bidding device 40, and performs output control on the user terminal 10 so as to output the acquired advertisement data (step S18). Specifically, the response generation device 100 transmits the acquired advertisement data to the user terminal 10.

  In this way, the response generation device 100 searches for advertisement information that becomes a response message based on the characteristics of sound in the conversation between the dialog agent system and the user, and outputs the advertisement data acquired as the search result as a response message. Control output. Thereby, the response generation device 100 can output a message suitable for the user and can perform advertisement distribution in accordance with the intention of the advertiser.

[2. Response generation system configuration]
Next, the configuration of the response generation system according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of the response generation system 1 according to the embodiment. As shown in FIG. 2, the response generation system 1 includes a user terminal 10, a speech recognition device 20, an advertiser terminal 30, an advertisement bidding device 40, an API server device 60, a speech synthesis device 70, and a response generation. Device 100. The user terminal 10, the speech recognition device 20, the advertiser terminal 30, the advertisement bidding device 40, the API server device 60, the speech synthesis device 70, and the response generation device 100 are wired or wirelessly via the network N. To be communicable. Note that the response generation system 1 illustrated in FIG. 2 may include a plurality of user terminals 10 and a plurality of advertiser terminals 30.

  Here, an outline of processing in which the response generation system 1 provides a voice service to the user will be described. When the user terminal 10 detects the user's speech after the application is started, the user terminal 10 transmits speech data of the speech to the speech recognition device 20.

  When the speech recognition apparatus 20 receives speech voice data from the user terminal 10, the speech recognition apparatus 20 converts the speech data into text data and transmits the speech text data to the user terminal 10. Upon receiving the utterance text data from the speech recognition server 20, the user terminal 10 transmits the utterance text data to the response generation apparatus 100.

  The advertiser terminal 30 transmits the advertising information received from the advertiser to the advertising bidding device 40. The advertisement information includes advertisement keywords, advertisement tags, advertisement data, and the like. The advertisement bidding device 40 presents a bid screen to the advertiser terminal 30. Further, the advertising bidding device 40 stores the advertising information received from the advertiser terminal 30 in a storage unit described later.

  When receiving the text data of the utterance and the user information acquired by the voice recognition device 20 from the user terminal 10, the response generation device 100 executes the above-described search process and generates a response message. When the response generation apparatus 100 outputs an image search result or a route search result as a response based on the user's utterance, the response generation apparatus 100 specifies a search condition for data necessary for generating the response, and the user terminal 10 is activated. The data request is made to the API server device 60 corresponding to the application.

  The API server device 60 transmits data including an image search result and a route search result to the response generation device 100 in accordance with the search condition received from the response generation device 100. For example, the API server device 60 performs processing for acquiring XML (Extensible Markup Language) data including image search results and route search results, and transmits the acquired XML data to the response generation device 100.

  When the response generation apparatus 100 receives, for example, XML data from the API server apparatus 60, the response generation apparatus 100 extracts the data from the XML data, converts the XML data into HTML data, and responds by voice from the XML data or HTML data. Text data (hereinafter referred to as text data for response utterance display) is extracted. Further, the response generation device 100 transmits the response utterance display text data and the advertisement data acquired by the search processing described above to the speech synthesizer 70. The speech synthesizer 70 transmits an intermediate notation for response utterance generated by performing speech synthesis processing for synthesizing speech from response utterance display text data and advertisement data to the response generation device 100. The response generation device 100 transmits the intermediate notation for response utterance, the text data for displaying the response utterance, and the HTML data to the user terminal 10.

  The user terminal 10 outputs the response voice using the received intermediate notation for response utterance, and displays the response content using text data for displaying the response utterance and HTML data. In this way, the response generation system 1 realizes a voice service that makes an appropriate response to the user's utterance.

  The response generation device 100 realizes output of a response message according to the user's characteristics by combining the response generation processing described above. Specifically, the response generation device 100 receives the user information specified from the voice received in the conversation between the dialog agent system and the user, and specifies the search tag corresponding to the received user information. Further, the response generation apparatus 100 extracts a keyword included in the user's utterance, and uses the extracted keyword as a search keyword. Then, the response generation device 100 transmits the search tag and the search keyword to the advertisement bidding device 40. In such a case, the advertisement bidding device 40 searches for advertisement information corresponding to the received search tag and search keyword, and transmits the advertisement data included in the advertisement information of the search result to the response generation device 100. Then, the response generation apparatus 100 transmits the received advertisement text data to the speech synthesizer 70 to acquire an intermediate expression for response utterance, and transmits the acquired intermediate expression and advertisement text data to the user terminal 10.

[3-1. Configuration of advertising bid device]
Next, the advertising bidding apparatus 40 according to the embodiment will be described with reference to FIG. FIG. 3 is a diagram illustrating a configuration example of the advertisement bidding apparatus 40 according to the embodiment. As shown in FIG. 3, the advertisement bidding apparatus 40 includes a communication unit 41, an advertisement information storage unit 42, and a control unit 43.

  The communication unit 41 is realized by, for example, a NIC (Network Interface Card). The communication unit 41 is connected to the network by wire or wireless.

  The advertisement information storage unit 42 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk.

  The advertisement information storage unit 42 stores various types of advertisement information. Specifically, the advertisement information storage unit 42 stores advertisement information received as a bid from the advertiser terminal 30. Here, FIG. 4 shows an example of the advertisement information storage unit 42 according to the embodiment. In the example illustrated in FIG. 4, the advertisement information storage unit 42 stores the advertisement keyword and the advertisement tag in association with the advertisement ID.

  “Advertisement ID” indicates identification information for identifying advertisement information. The “advertisement ID” is also identification information for identifying the advertiser and the advertiser terminal 30. “Advertising keyword” is a keyword set by the advertiser. For example, the advertiser sets a word characterizing a product or information to be advertised as an advertising keyword. “Ad tags” can be used to identify what people want to advertise, what search range they want to advertise for, and what type of advertisement (for example, This indicates the type of advertisement that the user wants to advertise when searching for an advertisement with a long message length. “Advertisement data” is an advertisement text set by the advertiser, and is submitted in a text data format, for example.

  That is, in FIG. 4, the advertiser (for example, store A) identified by the advertisement ID “C01” sets the advertisement keyword “Kima Curry” and the advertisement tag “Osaka region”, so that the user ”Is specified as an output target candidate, and for example, it is advertisement information for users living in the vicinity of Osaka, users using Osaka dialect, etc., Alternatively, it indicates that the advertisement information is expressed in Osaka dialect, or that a store exists in the Osaka area.

  Returning to FIG. 3, the description will be continued. For example, the control unit 43 stores various programs (corresponding to an example of an advertisement bidding program) stored in a storage device inside the advertising bidding apparatus 40 by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. This is realized by executing (Random Access Memory) as a work area. The control unit 43 is realized by an integrated circuit such as an application specific circuit (ASIC) or a field programmable gate array (FPGA).

  As shown in FIG. 3, the control unit 43 includes a bid accepting unit 44 and a presentation unit 45, and realizes or executes information processing functions and operations described below. Note that the internal configuration of the control unit 43 is not limited to the configuration illustrated in FIG. 3, and may be another configuration as long as information processing described later is performed. Further, the connection relationship between the processing units included in the control unit 43 is not limited to the connection relationship illustrated in FIG. 3, and may be another connection relationship.

  The bid reception unit 44 receives a bid for advertisement information including an advertisement keyword, an advertisement tag, and advertisement data from the advertiser by presenting a predetermined bid screen on the advertiser terminal 30. Then, the bid reception unit 44 pays out the advertisement ID, and stores the advertisement keyword included in the received advertisement information, the advertisement tag, and the advertisement data in the advertisement information storage unit 42 in association with the paid-out advertisement ID. The advertisement bidding device 40 has the same content as the “search tag” of the response generation device 100 described later as the “advertising tag”, so that the bid accepting unit 44 has the “advertising tag”. To the advertiser in a selectable manner. As a result, the advertisement bidding device 40 stores the “advertisement tag” selected by the advertiser in the advertisement information storage unit 42. The “advertisement tag” may be freely input by the advertiser. Moreover, the bid reception part 44 does not necessarily need to receive an advertisement tag.

  The presentation unit 45 searches for advertisement information in response to an advertisement acquisition request from the response generation device 100, and presents advertisement data included in the searched advertisement information. Specifically, the presentation unit 45 receives a search keyword and a search tag from the response generation device 100, and searches for advertisement data in the advertisement information storage unit 42 using the received search keyword and search tag. For example, the presentation unit 45 extracts advertisement data in which the search keyword and the advertisement keyword match and the search tag and the advertisement tag match from the advertisement information storage unit 42, and extracts the extracted advertisement data from the search keyword transmission source Is presented to the response generation apparatus 100.

[3-2. Response generator configuration]
Next, the response generation device 100 according to the embodiment will be described with reference to FIG. FIG. 5 is a diagram illustrating a configuration example of the response generation device 100 according to the embodiment. As illustrated in FIG. 5, the response generation device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

  The communication unit 110 is realized by a NIC or the like, for example. The communication unit 110 is connected to the network by wire or wireless.

  The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. As illustrated in FIG. 5, the storage unit 120 includes a determination information storage unit 121 and a search information storage unit 122.

  Here, FIG. 6 illustrates an example of the determination information storage unit 121 according to the embodiment. In the example shown in FIG. 6, the node ID for identifying each node, the node type indicating the type of the node, and the processing content indicating the processing procedure corresponding to the message are stored in association with each other.

  Although omitted from FIG. 6, it is assumed that information indicating which node each node is connected to is registered in the determination information storage unit 121. For example, the node having the node ID “N1” and the nodes having the node IDs “N2” and “N3” are connected to the determination information storage unit 121, and the node IDs “N2” and “N2” are connected to the nodes having the node ID “N1”. It is assumed that the transition probabilities that are the probabilities of transition to “N3” are “0.5”, respectively. As a result, the determination information storage unit 121 stores data having the tree structure shown in FIG.

  FIG. 7 is a schematic diagram of a tree structure stored in the determination information storage unit 121. A broken line block shown in FIG. 7 indicates a detection node, and a solid line block indicates an operation node. Also, node IDs are assigned to these blocks. Moreover, the arrow which connects each block has shown the edge, and specifically, the start point (side without an arrow) shows a connection origin node, and the end point (side with an arrow) shows a connection destination node. . For example, the arrow connecting the node with the node ID “N11” and the node with the node ID “N12” is an operation in which the connection source node is the detection node with the node ID “N11” and the connection destination node is the node ID “N12”. Indicates a node. In addition, the numerical value (0.5 etc.) described shows a transition probability. Further, the determination tree schematically illustrated in FIG. 7 represents only some of the detection nodes and operation nodes stored in the determination information storage unit 121, and the nodes illustrated in FIGS. 6 and 7 are included in each node. In addition, it is assumed that various detection nodes and operation nodes are connected.

  The search information storage unit 122 stores search information for searching for advertisement information serving as a response message. Here, FIG. 8 illustrates an example of the search information storage unit 122 according to the embodiment. In the example illustrated in FIG. 8, the search information storage unit 122 stores “user information” and “search tag” in association with each other.

  “User information” is user information that may be specified by the speech recognition device 20, for example, information on age, gender, personality, emotion, dialect, physical condition, or environmental sounds around the user. . The user information may be a combination of these information. The “search tag” indicates various search conditions such as a search target and a search range.

  That is, the response generation device 100 searches for advertisement information that is related to the user information. For example, when the user information “Osaka dialect” is received from the speech recognition device 20, the advertisement generation device 100 receives advertisement information for users living in the vicinity of Osaka, users using the Osaka dialect, etc. Search for advertisement information expressed in the above, or advertisement information of an advertiser who has a store in the Osaka region. The association registered in the search information storage unit 122 may be registered as an arbitrary association by the advertiser or the administrator of the advertisement generation device 100.

  As described above, the response generation device 100 does not necessarily have the search information storage unit 122. For example, the search information storage unit 122 may use the user information itself received from the voice recognition device 20 as a search tag. That is, for example, when the user information “60s” is received, the response generation apparatus 100 may search for advertisement information in which “60s” is set as the advertisement tag (advertisement target). As described above, the response generation apparatus 100 can employ any method as long as it searches for an advertisement to be output based on the speech waveform in the conversation.

  Returning to FIG. The control unit 130 is realized by executing various programs (corresponding to an example of a response generation program) stored in a storage device inside the response generation apparatus 100 by using the RAM as a work area by, for example, a CPU or MPU. Is done. The control unit 130 is realized by an integrated circuit such as an ASIC or FPGA, for example.

  As illustrated in FIG. 5, the control unit 130 includes a reception unit 131, a search unit 132, and an output control unit 133, and realizes or executes functions and operations of information processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration illustrated in FIG. 5, and may be another configuration as long as information processing described later is performed. In addition, the connection relationship between the processing units included in the control unit 130 is not limited to the connection relationship illustrated in FIG. 5, and may be another connection relationship.

  The receiving unit 131 receives voice text and user information from the voice recognition device 20. Specifically, the receiving unit 131 receives information obtained from the properties related to the speech waveform uttered by the user as the sound feature information. The information obtained from the property related to the speech waveform is user information related to the user, such as age, sex, personality, emotion, dialect, physical condition, or information related to environmental sounds around the user.

  Hereinafter, an example in which the search unit 132 narrows down advertisements by region based on dialects will be described. For example, as shown in FIG. 7, in response to a response message from the response generation device 100 such as “What kind of curry do you like?”, The user U01 has input “Keyma Curry I love you” It is assumed that the information “Osaka dialect” is specified. In such a case, the receiving unit 131 receives the user information “Osaka dialect” along with the voice text data via the user terminal 10. Then, the reception unit 131 transmits the received text data and user information “Osaka dialect” to the search unit 132.

  The search unit 132 searches for advertisement information based on the characteristics of the voice spoken by the user. Specifically, based on the received user information, the search unit 132 searches for advertisement information by changing a search target or search range of advertisement information.

  Specifically, when receiving the voice text and the user information from the receiving unit 131, the search unit 132 selects a response message using the voice text and the information stored in the determination information storage unit 121. For example, when receiving the speech text, the search unit 132 determines a detection node having a keyword included in the speech text, and a response message corresponding to the operation node connected to the determined detection node is output. The output of the response message data is controlled. Subsequently, when receiving the voice text of the input message for the response message, the search unit 132 has a keyword included in the voice text among the plurality of detection nodes connected to the operation node. Determine. Then, the search unit 132 controls the output of response message data corresponding to the operation node connected to the determined detection node. In this way, the response generation device 100 realizes a conversation with the user using the detection node and the operation node.

  Moreover, the search part 132 performs the following processes, when performing the advertisement search is registered in the process content. Specifically, the search unit 132 accesses the search information storage unit 122 and specifies a search tag corresponding to the received user information. In the case of this example, the search unit 132 receives the user information “Osaka dialect” from the reception unit 131, and thus specifies the search tag “Osaka district” corresponding to the user information “Osaka dialect”. This search tag “Osaka region” designates a search target so as to search, for example, advertisement information for users living in the vicinity of Osaka and advertisement information in the Osaka area such as advertisement information of an advertiser who has a store in the Osaka area. Indicates that

  In addition, the search unit 132 extracts a keyword from the received speech text, and sets the extracted keyword as a search keyword. In such an example, the user U01 has input “Keyma Curry, I love you”, and the search unit 132 sets “Keyma Curry” as a search keyword.

  Then, the search unit 132 acquires the advertisement data by requesting the advertisement bidding device 40 to search for advertisement information corresponding to the search keyword and the search tag. Specifically, the search unit 132 transmits the search keyword and the search tag to the advertisement bidding apparatus 40 as a search condition. For example, the search unit 132 transmits the search keyword “key curry” and the search tag “Osaka region”.

  Here, the presentation unit 45 of the advertisement bidding device 40 receives the search keyword and the search tag, and thus the advertisement data in which the search keyword and the advertisement keyword match and the search tag and the advertisement tag match is sent to the advertisement information. Extracted from the storage unit 42. For example, in the example of FIG. 4, the advertiser identified by the advertisement ID “C01” has set the advertisement keyword “Kima Curry” and the advertisement tag “Osaka region”, and the search condition transmitted by the search unit 132 is Match. Therefore, the presentation unit 45 extracts the corresponding advertisement data “curry shop A store Osaka advance!”.

  In addition, the presentation unit 45 presents the extracted advertisement data to the response generation device 100 that is the transmission source. Thereby, the search unit 132 acquires the advertisement data presented by the advertisement bidding device 40. Then, the search unit 132 transmits the acquired advertisement data to the output control unit 133. In this example, since the advertisement data “curry shop A store Osaka advance!” Is presented by the presentation unit 45, the search unit 132 acquires the advertisement data “curry store A store Osaka advance!” And outputs control. To the unit 133.

  The output control unit 133 performs output control so that the response message is output as voice by the user terminal 10. Specifically, the output control unit 133 transmits advertisement data to the speech synthesizer 70 to receive an intermediate representation (for example, reproduction waveform data), and receives the received intermediate representation and text of the advertisement data as a response generation device 100. Send to.

[4. Response generation process flow)
Next, a response generation process performed by the response generation apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 9 is a flowchart illustrating a response generation processing procedure performed by the response generation apparatus 100 according to the embodiment.

  As illustrated in FIG. 9, first, the user terminal 10 receives a voice related to a user's utterance, an environmental sound around the user, and the like (step S201). Then, the user terminal 10 transmits the received voice data to the voice recognition device 20 (step S202). When receiving the voice data, the voice recognition device 20 converts the voice data into text data and identifies user information from the voice data (step S203). Specifically, the voice recognition device 20 identifies user information regarding the user as sound characteristic information by analyzing voice data. More specifically, the speech recognition apparatus 20 generates a speech waveform from speech data, and specifies user information as information obtained from the properties related to the speech waveform. Then, the voice recognition device 20 transmits the text data and the acquired user information to the user terminal 10 (step S204). The user terminal 10 transmits the received text data and user information to the response generation device 100 (step S205).

  Here, when the response generation apparatus 100 receives text data and user information, if the fact that an advertisement search is performed is registered in the processing content, the response generation apparatus 100 accesses the search information storage unit 122 and corresponds to the received user information. Identify the search tag you want. In addition, the response generation apparatus 100 extracts keywords from the text data, and sets the extracted keywords as search keywords (step S206). Then, the response generation device 100 requests the advertisement bidding device 40 to search for advertisement information corresponding to the search keyword and the search tag by transmitting the search keyword and the search tag to the advertisement bidding device 40 as a search condition. (Step S207). On the other hand, the advertisement bidding device 40 extracts advertisement data in which the search keyword and the advertisement keyword match and the search tag and the advertisement tag match from the advertisement information storage unit 42, and transmits the extracted advertisement data to the transmission source. Is presented to the response generation apparatus 100 (step S208).

  The response generation device 100 acquires the advertisement data presented by the advertisement bidding device 40 (step S209). And the response production | generation apparatus 100 performs output control so that the acquired advertisement data is output as an audio | voice by the user terminal 10 (step S210).

[5. (Modification)
The response generation apparatus 100 according to the above-described example may be implemented in various different forms other than the above embodiment. Accordingly, another embodiment of the response generation device 100 will be described below.

[5-1. Search process (1)]
The search unit 132 of the response generation device 100 described above has shown an example in which advertisements are narrowed down by region based on a dialect used by the user as user information acquired from speech. However, when information about the user's personality or emotion is received as user information, the search unit 132 may change the search target according to the information about the personality or emotion. That is, the search unit 132 may narrow down the advertisement information according to information on personality or emotion.

  Specifically, when it is estimated that the topic of the user tends to switch based on the characteristics of sound in conversation, the search unit 132 advertises advertisement information including a message longer than a predetermined message. If it is estimated that the user's topic tends to be difficult to switch, advertisement information of an advertisement including a message shorter than the predetermined message is searched.

  For example, when the user information that the user has the character of “dislike uselessness” is received as the feature information of the voice of the user who has input the message, the response generation apparatus 100 “the message length is short”. Search for advertising information. In this case, for example, the response generation apparatus 100 stores the user information “I don't like useless talk” and the search tag “short message advertisement” in association with each other.

  Note that the voice recognition device 20 has a personality that, for example, a user whose tone of tempo calculated based on a voice waveform is shorter than a predetermined tone is quick, so he / she does not like useless talk (topic switching is difficult). That is, the user information is specified.

  Here, the response generation device 100 sets a predetermined threshold for the number of characters in the message. Then, the search unit 132 makes a search request to the advertisement bidding device 40 so that the search keyword and the advertisement keyword match and the number of characters less than the set threshold value is searched, thereby corresponding advertisement information. Get the advertising data contained in.

  As described above, when the user who has input the message is “dislike useless talk”, the response generation apparatus 100 acquires the advertisement data having a short message length from the advertisement bidding apparatus 40, and controls the output of the acquired advertisement data. To do. For example, a user who dislikes useless talk may feel uncomfortable when the response message is long. Therefore, the response generation apparatus 100 controls to output advertisement data having a short message length to a user who dislikes useless talk.

  On the other hand, the search unit 132 searches for advertisement information with a long message length when the user who has input the message is “I like useless talk”. That is, the response generation device 100 stores, for example, user information “like a waste story” and a search tag “long message advertisement” in association with each other. Then, the search unit 132 makes a search request to the advertisement bidding device 40 so that the search keyword and the advertisement keyword match and the advertisement information having a number of characters larger than the set threshold value is searched, thereby corresponding advertisement data. To get.

  In this case, the voice recognition device 20 is relaxed because the tone of the tone calculated based on the voice waveform is longer than a predetermined tone, so that the user recognizes “I like useless talk” (the topic is switched). This means that the user information is specified as having a difficult character.

  As described above, the response generation apparatus 100 searches for an advertisement having a long message length and performs output control when the user who has input the message is “I like useless talk”. Contrary to the above-mentioned example of “dislike useless talk”, it is considered that if “I like useless talk”, there is a low possibility of feeling uncomfortable with advertisement information having a long message length. Therefore, the response generation apparatus 100 controls to output advertisement data having a long message length to a user who likes “useless talk”.

  As described above, the response generation device 100 can provide advertisement information suitable for the response message according to the personality or emotion of the user.

  In addition, since emotions have ups and downs, even a user who normally has a character that likes “waste talk”, for example, if the user is temporarily “irritated”, Can cause discomfort. For this reason, the response generation apparatus 100 may normally control the output of advertisement data according to personality, and when a predetermined emotion is received, priority may be given to search processing according to that emotion.

[5-2. Search process (2)]
In addition, when the search unit 132 of the response generation device 100 described above receives information about the user's personality or emotion as user information, the search unit 132 changes the search range according to the information about the personality or emotion. May be.

  Specifically, when it is estimated that the user's topic is likely to be switched based on the characteristics of sound in the conversation, the search unit 132 widens the search range of the advertisement information and the user's topic is switched. When it is estimated that there is a tendency to be difficult, the search range of advertisement information is narrowed.

  For example, when the response generation apparatus 100 receives user information indicating that the user has a disliked character as the feature information of the voice of the user who has input the message, the response generation apparatus 100 resembles the similarity at the time of advertisement information search. To narrow. That is, the search unit 132 uses a predetermined keyword extracted from the input message as a search keyword. For example, when the user inputs “I like Key Maca curry”, the search unit 132 extracts “Key Maca curry” and uses it as a search keyword. Then, the search unit 132 makes a search request to search using the search keyword “key curry”, thereby acquiring the advertisement data in which the advertisement keyword “key curry” is set.

  As described above, the response generation apparatus 100 controls the output of advertisement data searched using a keyword included in the input message of the user as a response message for a user who dislikes useless talk. Since such a response message is faithful to the conversation context, it does not cause discomfort to a user who dislikes useless talk. Specifically, since the response generation apparatus 100 outputs a response message with the content related to “key macurry” to the user having a conversation related to “key curry”, the response generation apparatus 100 does not deviate from the conversation context. There is no discomfort.

  On the other hand, if the search unit 132 receives user information indicating that the personality is “likeless talk”, the search unit 132 spreads the similarity when searching for advertisement information. It can be said that a user who likes “waste talk” wants to enjoy various conversations with the response generation device 100. In other words, it is considered that a discomfort is not given by a response message out of the dialog context. Therefore, the search unit 132 may output control advertisement information as a response message that deviates from the conversation context. For example, when the user inputs “I like Kema curry”, the search unit 132 extracts “Kima curry” as a search keyword and uses a similar search keyword that is a related word to search range. To spread. In other words, the search unit 132 searches for advertisement information with a wide similarity that includes “key curry” in the search range.

  For example, the search unit 132 sets “curry” and “indian cuisine” as similar search keywords similar to “Kima curry”. Note that any known technique is applied to the process in which the response generation apparatus 100 searches for related words. Here, the search unit 132 makes a search request using “Kima Curry”, “Curry”, and “Indian Food”, so that the advertising keywords “Kema Curry”, “Curry”, and “Indian Food” are obtained. Get the set advertisement data.

  As described above, the response generation apparatus 100 outputs advertisement data searched using a related word related to a keyword included in the input message of the user as a response message to a user who likes “useless talk”. Such a response message may deviate from the dialog context as compared to a case where a response message based on a search by a search keyword is output. That is, the response generation apparatus 100 may output a response message regarding “curry” or “indian food” to a user having a conversation regarding “key curry”. As described above, the response generation apparatus 100 can provide various topics to the user who has determined that he / she likes “useless talk” by outputting a response message out of the conversation context. Satisfaction with can be increased.

[5-3. Search process (3)]
Moreover, the search part 132 of the response production | generation apparatus 100 mentioned above may change the search range of advertisement information according to the age, when a user's age is received. For example, when the user information “60s” of the user is received as the feature information of the voice of the user who has input the message, the response generation apparatus 100 narrows the similarity when searching for advertisement information. That is, the search unit 132 uses a predetermined keyword extracted from the input message as a search keyword, and requests to search using the search keyword.

  As described above, when the user who inputs the message is elderly, the response generation apparatus 100 outputs and controls advertisement data highly relevant to the message input by the user as a response message. For example, when the response generation apparatus 100 performs output control of a response message having a low relevance to the input message by performing a search with a wide similarity, the elderly cannot fully understand the content of the response message and feel uncomfortable. You may remember. For this reason, the response generation apparatus 100 controls the output of advertisement data highly relevant to the input message as a response message to the elderly by intentionally narrowing the similarity when searching for advertisement information. Thereby, the response generation device 100 can provide advertisement information according to the user.

  On the other hand, for example, when the user information “20s” of the user who received the message is received as the feature information of the voice of the user who has input the message, the response generation apparatus 100 expands the similarity when searching for advertisement information. That is, as described above, the search unit 132 sets a search keyword and a similar search keyword based on the content of the input message, and requests to search using these keywords.

  As described above, when the user who inputs the message is a young person, the response generation apparatus 100 may output and control advertisement data having a low relevance to the message input by the user as a response message. Since young people tend to acquire various information through the Internet, the response generation device 100 performs a search with a wide similarity to output a response message that is less relevant to the input message. It is unlikely that you will feel uncomfortable. Therefore, the response generation apparatus 100 controls the output of advertisement information having low relevance to the input message as a response message by expanding the similarity when searching for advertisement information. Thereby, the response generation device 100 can increase the output opportunity of the advertisement.

  Note that the above-described combination of user information and search processing is an example, and can be arbitrarily combined. For example, although the search part 132 showed the example which fluctuates a search range according to age, you may fluctuate a search object according to age. For example, the search unit 132 may search for advertisement information with a long message length and control the output of the advertisement data obtained as a result of the search so that the contents can be easily understood for the elderly. On the other hand, if it is a young person, even if it is advertisement information with a long message length, it is considered that there is no problem, so the search unit 132 searches for advertisement information with a long message length for the young person, and as a result The obtained advertisement data may be output controlled.

  In addition, the response generation apparatus 100 has shown an example in which the search range is narrowed for elderly people and the search range is widened for young people. For example, even for young people, the search range is narrowed for children. May be. That is, the response generation device 100 may arbitrarily set the relationship between the age and the search range.

[5-4. Search process (4)]
In addition, the search unit 132 of the response generation device 100 described above may perform a search based on the physical condition of the user. Specifically, the voice recognition device 20 specifies information related to physical condition as user information. For example, when the shape that characterizes the “coughing sound” can be identified in the generated speech waveform, the speech recognition apparatus 20 identifies “coldness” as user information, for example. Thereby, the search unit 132 makes a search request with the user information “cold taste” as a search tag. Thereby, the response generation device 100 can make a bid for the physical condition to the advertiser. For example, if the advertiser is a drugstore owner, the advertiser advertises the product of his store to the user who has a cold by setting the advertisement tag “cold” Can do. In this way, by using the response generation device 100 and the sound feature information, it is possible to increase the advertising effect by enabling a bid related to physical condition. Moreover, since the response production | generation apparatus 100 can provide the advertising information regarding a chemical | medical agent, a pharmacy, etc. with respect to a user with bad physical condition, for example, it can output the response message suitable for a user.

[5-5. Search process (5)]
Further, the search unit 132 of the response generation device 100 described above may perform a search based on environmental sounds around the user. Specifically, the voice recognition device 20 specifies information related to environmental sounds around the user as user information. For example, when the shape characterizing the “baby cry” can be identified in the speech waveform of the sound acquired as the back noise, the speech recognition apparatus 20 identifies “baby parent” as the user information, for example. As a result, the response generation apparatus 100 makes a search request using the user information “parent of baby” as a search tag. Thereby, the response generation device 100 can make a bid for the baby product with respect to the advertiser, for example. As described above, the response generation device 100 can increase the advertising effect by enabling the bid for the environmental sound by using the feature information of the sound. Moreover, since the environmental sound is an environmental sound around the user, the response generation apparatus 100 can provide advertisement information advantageous to the user.

[5-6. Search considering location information)
Further, the search unit 132 of the response generation apparatus 100 described above may perform a search in consideration of position information. For this purpose, the advertising bidding device 40 accepts an output location from which the advertiser wants to output advertising information. And the search part 132 acquires the user's location corresponding to the received user information, when receiving user information. Specifically, the search unit 132 specifies the location of the user by matching the terminal ID included in the received user information with the terminal ID included in the GPS log. Then, the search unit 132 transmits the specified location to the advertising bid device 40 together with the search keyword and the search tag. The advertisement bidding apparatus 40 presents the advertisement data that matches the received search condition to the response generation apparatus 100 of the transmission source. The search unit 132 acquires the presented advertisement data, and controls output of the acquired advertisement data.

  Thereby, since the response production | generation apparatus 100 can distribute the advertising data of the advertiser with respect to the user who exists in the output place which an advertiser desires, it can improve an advertising effect.

[5-7. Device configuration〕
In the said embodiment, the response generation apparatus 100 showed the example which receives the user information specified by the speech recognition apparatus 20. However, the response generation apparatus 100 may specify user information as sound feature information. In this case, the user terminal 10 that has received the voice transmits the sound data to the response generation device 100. The response generation device 100 creates a speech waveform related to the received speech data, and identifies characteristic user information of the speech waveform.

[5-8. Example other than user terminal)
In the above-described embodiment, an example has been described in which the user performs a conversation with the response generation apparatus 100 using the user terminal 10. However, the interactive function with the response generation device 100 of the user terminal 10 may be installed in a robot that performs conversation. Thereby, it can be realized that the robot has a conversation with the response generation device 100 on behalf of the user.

[5-9. program〕
Further, the response generation apparatus 100 according to each embodiment described above is realized by a computer 1000 having a configuration as shown in FIG. 10, for example. Hereinafter, the response generation apparatus 100 will be described as an example. FIG. 10 is a hardware configuration diagram illustrating an example of a computer 1000 that implements the function of the response generation apparatus 100. The computer 1000 includes a CPU 1100, RAM 1200, ROM 1300, HDD 1400, communication interface (I / F) 1500, input / output interface (I / F) 1600, and media interface (I / F) 1700.

  The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each unit. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

  The HDD 1400 stores programs executed by the CPU 1100, data used by the programs, and the like. The communication interface 1500 receives data from other devices via the communication network 50 and sends the data to the CPU 1100, and transmits the data generated by the CPU 1100 to other devices via the communication network 50.

  The CPU 1100 controls an output device such as a display and a printer and an input device such as a keyboard and a mouse via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. Further, the CPU 1100 outputs the generated data to the output device via the input / output interface 1600.

  The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads such a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or PD (Phase change rewritable disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. Etc.

  For example, when the computer 1000 functions as the response generation apparatus 100 according to the embodiment, the CPU 1100 of the computer 1000 implements the function of the control unit 130 by executing a program loaded on the RAM 1200. The HDD 1400 stores data in the storage unit 120. The CPU 1100 of the computer 1000 reads these programs from the recording medium 1800 and executes them, but as another example, these programs may be acquired from other devices via the communication network 50.

[5-10. Others]
Of the processes described in the above embodiment, all or part of the processes described as being automatically performed can be performed manually, or all of the processes described as being performed manually or A part can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the document and drawings can be arbitrarily changed unless otherwise specified.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  Moreover, each embodiment mentioned above can be combined suitably in the range which does not contradict a process content.

[6. effect〕
As described above, in the control device according to the embodiment, the response generation device 100 includes the search unit 132 and the output control unit 133. The search unit 132 searches for advertisement information that is information related to an advertisement to be output as a response message, based on the characteristics of sound in the conversation between the dialog agent system and the user. The output control unit 133 controls to output an advertisement related to the advertisement information searched by the search unit 132 as a response message.

  Thereby, the response generation apparatus 100 can output a message suitable for the user and can perform advertisement distribution in accordance with the intention of the advertiser.

  In addition, the search unit 132 according to the embodiment searches for advertisement information based on the characteristics of the voice spoken by the user.

  Thereby, the response generation device 100 according to the embodiment can obtain voice-specific information that cannot be obtained from the voice text.

  In addition, the search unit 132 according to the embodiment performs advertisement based on at least one of the user's age, sex, personality, emotion, dialect, or physical condition estimated based on the characteristics of the voice spoken by the user. Search for information.

  Thereby, the response generation apparatus 100 according to the embodiment can output a response message with advertisement information suitable for the user, and can perform advertisement distribution in accordance with the intention of the advertiser.

  Moreover, the search part 132 concerning embodiment searches the advertising information of the advertisement corresponding to the area specified by the estimated dialect.

  Thereby, since the response generation device 100 according to the embodiment can output the advertisement information related to the user's hometown, the user's interest in the advertisement information can be increased, and the advertisement related to the region can be provided to the advertiser. Information can be made easier to bid.

  Moreover, the search part 132 concerning embodiment expands the search range of advertisement information, so that the estimated age is low, and narrows the search range of advertisement information, so that the estimated age is high.

  Thereby, the response generation device 100 according to the embodiment can output advertisement information corresponding to the age as a response message.

  In addition, when it is estimated that the user's topic is likely to be switched based on the sound characteristics in the conversation, the search unit 132 according to the embodiment widens the search range of the advertisement information, and the user's topic is When it is estimated that it is difficult to switch, the range for searching for advertisement information is narrowed.

  Thereby, since the response production | generation apparatus 100 concerning embodiment can output the advertising information according to a user's character and emotion as a response message, it can raise the user's satisfaction with respect to a conversation.

  In addition, when it is estimated that the user's topic tends to switch based on the characteristics of sounds in conversation, the search unit 132 according to the embodiment may advertise an advertisement including a message longer than a predetermined message. When information is searched and it is estimated that the user's topic tends to be difficult to switch, advertisement information of an advertisement including a message shorter than a predetermined message is searched.

  Thereby, since the response production | generation apparatus 100 concerning embodiment can output the advertising information according to a user's character and emotion as a response message, it can raise the user's satisfaction with respect to a conversation.

  As described above, some of the embodiments of the present application have been described in detail with reference to the drawings. However, these are merely examples, and various modifications, including the aspects described in the disclosure section of the invention, based on the knowledge of those skilled in the art, It is possible to implement the present invention in other forms with improvements.

  In addition, the “section (module, unit)” described above can be read as “means” or “circuit”. For example, the determination unit can be read as specifying means or a specific circuit.

DESCRIPTION OF SYMBOLS 10 User terminal 20 Speech recognition apparatus 30 Advertiser terminal 40 Advertisement bid apparatus 42 Advertisement information storage part 44 Bid reception part 45 Presentation part 100 Response generation apparatus 121 Judgment information storage part 122 Search information storage part 130 Control part 131 Reception part 132 Search part 133 Output control unit

Claims (9)

A search unit that searches for advertisement information that is information related to an advertisement to be output as a response message within a search range based on the characteristics of sound in a conversation between a dialog agent system and a user;
An output control unit that controls to output an advertisement related to the advertisement information searched by the search unit as a response message;
A control device comprising: The search unit searches the advertisement information in a search range based on a voice feature spoken by the user as a sound feature in the conversation.
The control device according to claim 1. The search unit is a search range based on at least one of the age, gender, personality, emotion, dialect, or physical condition of the user estimated based on the characteristics of the voice spoken by the user, and the advertisement information The control device according to claim 2, wherein the control device is searched. The search unit searches for advertisement information of an advertisement corresponding to an area specified by the estimated dialect used by the user;
The control device according to claim 3. The search unit expands the search range of the advertisement information as the estimated age of the user is low, and narrows the search range of the advertisement information as the estimated age of the user is high.
The control device according to claim 3 or 4, wherein When it is estimated that the user's topic is likely to be switched based on the sound characteristics in the conversation, the search unit expands the search range of the advertisement information and the user's topic is difficult to switch. If it is estimated that there is a tendency, narrow the search range of the advertising information,
The control device according to any one of claims 1 to 5, wherein: The search unit searches for advertisement information of an advertisement including a message longer than a predetermined message when it is estimated that the topic of the user is likely to switch based on the characteristics of sound in the conversation. The advertisement information of an advertisement including a message shorter than the predetermined message is searched when it is estimated that the topic of the user is not likely to be switched. The control apparatus as described in any one. A control method executed by a computer,
A search step of searching for advertisement information that is information related to an advertisement to be output as a response message in a search range based on the characteristics of sound in a conversation between a dialog agent system and a user;
An output control step of controlling to output an advertisement related to the advertisement information searched by the search step as a response message;
The control method characterized by including. A search procedure for searching advertisement information, which is information related to an advertisement to be output as a response message, within a search range based on the characteristics of sound in a conversation between a dialog agent system and a user,
An output control procedure for controlling to output an advertisement related to the advertisement information searched by the search procedure as a response message;
A control program for causing a computer to execute.

Images