JP6054140B2 - Message management apparatus, message presentation apparatus, message management apparatus control method, and message presentation apparatus control method - Google Patents

Description

  The present invention relates to a message presentation system that presents a message.

  In recent years, electronic devices that transmit and receive messages between users or between users and electronic devices have been developed. The electronic device is devised to exchange messages smoothly.

  For example, Patent Literature 3 discloses a life monitoring system that converts text data and voice data to each other and smoothly transmits a message between a watcher and an elderly person.

  Furthermore, in an electronic communication device that exchanges messages between a user and an electronic device, a device is provided so that a more natural and human-friendly message or a message that is more beneficial to the user can be transmitted.

  For example, Patent Literature 1 discloses an electronic communication device that changes a message to be presented based on a user attribute.

  Patent Document 2 discloses a system that detects a keyword in a message transmitted and received between users and drives a character device with a control command associated with the keyword.

JP 2005-092540 A (published April 7, 2005) JP 2000-076167 A (published March 14, 2000) JP 2004-272568 A (published September 30, 2004)

  Even when messages are exchanged between a user and an electronic device, from the viewpoint of usability, the electronic device presents a message to the user's utterance so that a person can talk to another person. Is considered more natural. In addition, in order to transmit the optimum message for the user's utterance, the electronic device changes the message to be presented based on “who (who is the uttering user)” and “what was said (what is the utterance content)” It is desirable to make it.

  However, the techniques disclosed in Patent Documents 1 and 2 present a message in response to the user's electronic message transmission, and do not present a message for the user's natural utterance. Further, in the technologies disclosed in Patent Documents 1 and 2, the user identification information (user ID, e-mail address, etc.) is used to determine who the user is and to change the message to be presented. It is difficult to apply against.

  As a result, in the techniques disclosed in Patent Documents 1 and 2, the conversation content (message to be presented) is based on “who” and “what was said” so that a person speaks in response to the user's utterance. There is a problem that cannot be changed.

  SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and its purpose is to provide a message management apparatus and message presentation that can cause a user to present a message with a more appropriate content to the output apparatus in response to the user's utterance. An apparatus, a message presentation system, a message management apparatus, a control method for the message presentation apparatus, a control program, and a recording medium are provided.

  In order to solve the above-described problem, a message management device according to an aspect of the present invention is a message management device that transmits a message to be presented to a presentation target person to the message presentation device. Based on utterance acquisition means for acquiring utterance data acquired and acquired, keyword extraction means for extracting keywords included in the utterance data acquired by the utterance acquisition means, and the utterance data acquired by the utterance acquisition means A target person specifying means for specifying the presentation target person, the keyword extracted by the keyword extracting means, and a message previously associated with the presentation target person specified by the target person specifying means, Message specifying means for specifying the message to be transmitted to the message presenting device. It is set to.

  The message management apparatus control method according to an aspect of the present invention is a message management apparatus control method for transmitting a message to be presented to a presentation target person to the message presentation apparatus, and acquires the utterance of the presentation target person. An utterance acquisition step of acquiring the utterance data generated in the step, a keyword extraction step of extracting a keyword included in the utterance data acquired in the utterance acquisition step, and the utterance data acquired in the utterance acquisition step. A target person specifying step for specifying the presentation target person based on the keyword extracted in the keyword extracting step, and a message associated with the presentation target person specified in the target person specifying step in advance Is specified as a message to be sent to the message presentation device It is characterized in that it comprises a step.

  Further, a message presentation device according to an aspect of the present invention is a message presentation device that presents a message to a presentation target person, the utterance data generation unit that acquires the utterance of the presentation target person and generates utterance data; Message acquisition means for acquiring a keyword extracted from the utterance data generated by the utterance data generation means, and a message associated in advance with the target person specified based on the utterance data, and the message acquisition means And a message output means for presenting the message acquired by the above method to a person to be presented.

  The control method of the message presentation device according to one aspect of the present invention is a message presentation method for presenting a message to a presentation target person, and obtains the utterance of the presentation target person and generates utterance data generation step. And a message acquisition step of acquiring a keyword extracted from the utterance data generated in the utterance data generation step, and a message associated in advance with the presentation target person specified based on the utterance data; A message output step of presenting the message acquired in the message acquisition step to a presentation target person.

  According to one aspect of the present invention, it is possible to identify a user in response to the user's utterance, and cause the message presenting apparatus to present a message determined based on the utterance content of the user. Thereby, the message presenting apparatus has an effect that an optimum message can be presented to each user according to the content of the user's utterance.

It is a block diagram which shows the principal part structure of the output device and message server which comprise the message presentation system which concerns on embodiment of this invention. It is a figure which shows the operation | movement outline | summary of the said message presentation system. It is a figure which shows the data structure of a message table, and a specific example. (A) is a figure which shows the other example of a message table, (b) is a table which matched the genre and the subject person of the message table of (a). It is a figure which shows the data structure and specific example of group presentation information. It is the figure which showed the flow of the process of the said message presentation system.

Embodiment 1
Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS.

  First, an outline of the operation of the message presentation system 100 will be described with reference to FIG. FIG. 2 is a diagram showing an outline of the operation of the message presentation system 100.

[Operation overview]
The message presentation system 100 according to the present embodiment is
(1) The output device 1 detects a keyword spoken by a person in the vicinity of the output device and information for identifying the person, and transmits it to the message server 2;
(2) The message server 2 determines a message to be output to the output device 1 based on the keyword, information for identifying the person, and the current time, and transmits the message to the output device 1,
(3) The output device 1 is a system that outputs a received message. Hereinafter, a person who speaks in the vicinity of the output device 1 is simply referred to as a “presentation target person”.

  Here, the “current time” refers to a time measured when the message server 2 determines a message to be output to the output device 1. The same applies to the description of “current time” below.

  As shown in FIG. 2, the message presentation system 100 includes an output device 1 and a message server 2. The output device 1 and the message server 2 transmit and receive data to and from each other. The communication mode between the output device 1 and the message server 2 is not particularly limited, but wireless communication is desirable.

  Next, functions of each device will be briefly described.

  The output device 1 records a voice uttered by the presentation target person when the presentation target person approaches the device, acquires the voice data, a function of acquiring image data of the presentation target person, and a message And a function of outputting a message received from the server 2. Hereinafter, data obtained by recording the voice uttered by the person to be presented is simply referred to as “utterance data”.

  In addition, since the output device 1 is a device for recording a user's utterance and taking a picture and presenting a message to the user mainly using voice, the output device 1 is preferably a self-propelled device that can move to the vicinity of the user. . Furthermore, the output device 1 may be mounted on a self-propelled cleaner.

  The message server 2 has a function of specifying a presentation target person based on the utterance data and image data received from the output device 1, and a function of extracting a predetermined keyword from the utterance data (speech uttered by the presentation target person). Further, the message server 2 has a function of measuring the current time and a function of determining the content of a message to be output to the output device 1. Detailed description of each function will be described later.

  Next, the basic operation of the message presentation system 100 will be described with reference to FIG.

  As shown in FIG. 2, when the presentation target person A approaches the output device 1 and utters “immediately”, the output device 1 acquires the utterance of the presentation target person detected while stationary or moving as utterance data. Subsequently, the output device 1 takes a photograph or a moving image in the direction in which the sound is made, and acquires it as image data. The output device 1 transmits the obtained utterance data and image data to the message server 2.

  The message server 2 converts the received utterance data into a character string, and obtains the keyword “just now” from the character string. A specific keyword acquisition method will be described later.

  Further, the message server 2 specifies the presentation target person using the speech data and the image data, and specifies that the presentation target person A is “Taro”. A specific method for identifying the person to be presented will be described later.

  Furthermore, the message server 2 measures the current time (18:00), and based on the current time (18:00), the obtained keyword (now), and the identification result (Taro) of the presentation target person A, the output device The message to be output to 1 is determined to be “return”. The determined message is transmitted to the output device 1 and output.

  Next, the main configuration of each device included in the message presentation system 100 will be described with reference to FIG.

[Main part configuration]
FIG. 1 is a block diagram showing a main configuration of the output device 1 and the message server 2 included in the message presentation system 100 shown in FIG. Note that portions not directly related to the present embodiment (for example, a portion that realizes connection with an external device or audio output, etc.) are omitted from the description of the configuration and the block diagram from the viewpoint of ensuring the simplicity of the description. . However, the message presentation system 100 may include the omitted configuration in accordance with the actual situation.

  The message presentation system 100 includes an output device 1 and a message server 2 as shown in FIG.

(Configuration of output device 1)
The output device 1 includes a detection unit 10, a control unit 20, a communication unit 30, and an output unit 40.

  The detection unit 10 detects the speech of the presentation target person and detects the speech data and image data of the presentation target person. The detected data is transmitted to a detection data acquisition unit 21 described later.

  More specifically, the detection unit 10 includes a sensor microphone 11 and a camera 12.

  The sensor microphone 11 detects and acquires the voice of the person to be presented. The configuration and form of the sensor microphone 11 are not particularly limited as long as the content uttered by the person to be presented can be detected and acquired as utterance data.

  The camera 12 captures a photograph or video of the person to be presented and acquires it as image data. The configuration and form of the camera 12 are not particularly limited as long as the image data that can be used for specifying the person to be presented can be taken.

  However, in order for the camera 12 to obtain image data, it is preferable to detect in which direction the person to be presented is present as viewed from the output device 1 and perform shooting in that direction. Therefore, when the image data is obtained using the camera 12 and is used for specifying the presentation target person in the target person specifying unit 62 to be described later, the detection unit 10 is the display target person viewed from the output device 1. It is desirable to have a sensor that detects the direction in which there is. Specifically, the sensor is an infrared sensor or the like. Alternatively, the sensor microphone 11 may not only record the sound but also detect the direction of the sound.

  The control unit 20 controls the output device 1 in an integrated manner.

  The control unit 20 is realized by, for example, a CPU (central processing unit), and the output device 1 is controlled by the CPU as the control unit 20 by reading a program stored in a ROM or the like into a RAM or the like. This is realized.

  More specifically, the control unit 20 includes a detection data acquisition unit 21 and a message acquisition unit 22.

  The detection data acquisition unit 21 acquires data detected by the sensor microphone 11 and the camera 12 of the detection unit 10. The acquired data is transmitted to the message server 2 via the communication unit 30.

  The message acquisition unit 22 acquires message data from the message server 2 via the communication unit 30. The message data here is data obtained by converting a character string of a message to be output by the output device 1 into a format that can be output by the output unit 40. The acquired message data is transmitted to the output unit 40.

  The message data may be in a format that can be output by at least the output unit 40, and the format is not particularly limited. For example, the message data may be voice data such as artificial voice. Furthermore, data of a plurality of formats may be included.

  The communication unit 30 exchanges data with the server communication unit 50 of the message server 2. The communication unit 30 receives audio data and image data from the detection data acquisition unit 21 and transmits them to the server communication unit 50. In addition, the communication unit 30 receives message data from the server communication unit 50 and transmits the message data to the message acquisition unit 22.

  The output unit 40 outputs the message data received from the message acquisition unit 22. Specifically, the output unit 40 is preferably a device that outputs sound, such as a speaker.

  However, the manner in which the output unit 40 presents message data is not limited to voice. The output unit 40 may receive message data in various formats such as voice, characters, and illustrations, and may combine them to present to a person to be presented. For example, the output unit 40 includes a display and displays characters and illustrations. A message may be presented by a combination of voice and the above display.

  Moreover, you may change the output style of the output part 40 according to the presentation target person who the below-mentioned target person specific | specification part 62 specifies. For example, for a presentation target person who is hard to hear, the output unit 40 may present a message using only letters or a combination of voice and letters.

(Configuration of message server 2)
The message server 2 includes a server communication unit 50, a server control unit 60, a timer unit 70, and a storage unit 80.

  The server communication unit 50 performs data transmission / reception with the communication unit 30 of the output device 1. The server communication unit 50 receives audio data and image data from the communication unit 30, transmits the audio data to the keyword extraction unit 61 and the target person specifying unit 62, and transmits the image data to the target person specifying unit 62. Further, the server communication unit 50 receives message data from the message creation unit 65 and transmits it to the communication unit 30.

  The storage unit 80 stores authentication information 81, a message table 82, and group presentation information 83.

  Although not shown, the storage unit 80 stores group setting information. Here, the group setting information is information indicating a correspondence between a group name and one or a plurality of presentation target persons associated with the group name. Therefore, if the group name is determined by referring to the group setting information, the presentation target person associated with the group name is uniquely determined. The group setting information may be set in advance in the message server 2 or may be set and changed freely by the user.

  In the group setting information, a plurality of presentation target persons are collectively set and stored as one group, and the group name of the group is displayed in the “presentation target” column of the message table 82 described later. Can be stored instead of information. For example, in the group setting information, if the person to be presented with the group name “family” is set in advance as “Father, mother, Taro, Hanako”, the person to be presented in the message table 82 is set as “family”. can do.

  The authentication information 81 is information in which a feature amount indicating a feature that can identify a presentation target person is associated with user identification information. Here, the user identification information is, for example, a user ID, a user name, or the like, and any format can be used as long as each user can be uniquely identified.

  When the presentation target person specified by the target person specifying unit 62, which will be described later, is performed based on the utterance data, the feature amount is information indicating the characteristics of the voice of each individual, such as a waveform of voice data.

  In addition, when the target person to be presented specified by the target person specifying unit 62 is performed based on the image data, the feature amount includes, for example, the positional relationship between the eyes, the nose, and the mouth in the face photograph, the height, etc. This is information indicating the characteristics of each individual on the image.

  Further, in the case where the presentation target person is specified by using both the speech data and the image data, the feature amount is a combination of the above-described information.

  The feature amount and user recognition information registered in the authentication information 81 may be set in advance, or may be freely added / changed by the user.

  In the message table 82, a character string indicating the message content is associated with a keyword, a target person who presents the message content, and a period during which the message content may be output, or an outputable period indicating the time zone. Information. The person to be presented may be one user, may list a plurality of people, or may be a group name. The detailed data structure of the message table 82 will be described later.

  The group presentation information 83 is the presentation target person 1 when the presentation target person to whom the character string of the message content is associated in the message table 82 lists a plurality of persons or indicates the group name. This is information associated with whether or not the message content is presented to one person. The group presentation information 83 is referred to by an output necessity determination unit 64 described later. The detailed data structure of the group presentation information 83 will be described later.

  Although not shown, the storage unit 80 further stores a keyword list.

  The keyword list is list information in which keywords included in the message table 82 are listed without duplication. The keyword list is referred to by a keyword extraction unit 61 described later.

  The server control unit 60 controls the message server 2 in an integrated manner.

  More specifically, the server control unit 60 includes a keyword extracting unit 61, a target person specifying unit 62, a message determining unit 63, and a message creating unit 65.

  The keyword extraction unit 61 extracts keywords included in the voice data based on the utterance data received from the server communication unit 50.

Specifically, the keyword extraction unit 61 extracts keywords by performing the processes shown in (1) to (4) below, for example.
(1) The speech data received from the server communication unit 50 is converted into a character string.
(2) The converted character string is divided into predetermined phrases according to a predetermined rule.
(3) The keyword list memorize | stored in the memory | storage part 80 is read, and the phrase divided | segmented by (2) is searched for what corresponds to a keyword list.
(4) If there is a phrase that matches the keyword list, that phrase is taken as a keyword. The processes (3) and (4) are performed for each phrase divided in (2).

  The keywords extracted by the above processing are transmitted to the message determination unit 63.

  In addition, said (1)-(4) is an example, and if the keyword contained in the said speech data can be extracted, the extraction method will not be limited.

  Based on the utterance data and the image data received from the server communication unit 50, the target person specifying unit 62 is the presentation target person who uttered the voice of the utterance data, or who is the presentation target person reflected in the image data. It is to specify.

Specifically, the target person specifying unit 62 specifies the target person to be presented, for example, by performing the processes shown in the following (1) and (2).
(1) With reference to the authentication information 81, the audio data or the image data is collated with the feature amount of the authentication information 81.
(2) As a result of collation, when the feature amount of the authentication information 81 matches the utterance data or the image data, the user identification information associated with the feature amount is used as the presentation target person who utters the speech data. Alternatively, it is specified that the person to be presented is shown in the image data.

  As a specific example of (1), for example, when collation is performed using utterance data, the waveform of the utterance data may be compared and collated with the waveform of voice data stored as the feature amount of the authentication information 81. . When collation is performed using image data, the image data may be compared and collated with the positional relationship between eyes, nose, mouth, height, and the like of the face photograph stored as the feature amount.

  In addition, said (1) and (2) are examples, and if the presentation object person who uttered the audio | voice of the said speech data or is reflected in the said image data can be identified uniquely, the identification method will not be limited.

  In addition, the target person specifying unit 62 may specify the target person by combining both the utterance data and the image data received from the server communication unit 50.

  The message determination unit 63 searches the message table 82 based on the keyword extracted by the keyword extraction unit 61, the presentation target person specified by the target person specifying unit 62, and the current time measured by the time measuring unit 70 described later, The output device 1 determines a message to be output. A specific message determination method will be described later. The determined message is transmitted to the message creation unit 65.

  More specifically, the message determination unit 63 includes an output necessity determination unit 64.

  The output necessity determination unit 64 determines whether the message presentation target determined by the message determination unit 63 specifies a plurality of people or a group name, or specifies a specific individual, and further determines Result When the person to whom the message is presented designates a plurality of persons or a group name, it is determined whether or not the message has been transmitted to the person to be presented specified by the target person specifying unit 62. Specific processing will be described later together with a message determination method of the message determination unit 63.

  The message creation unit 65 creates message data obtained by converting the message content received from the message determination unit 63 into a format that can be output by the output unit 40 of the output device 1. The created message data is transmitted to the output device 1 via the server communication unit 50.

  The timekeeping unit 70 measures and acquires the current time when the current time is requested from the message determination unit 63.

  Note that the configuration of the timer unit 70 is not particularly limited as long as the current time can be measured. For example, the current time may be acquired via the Internet, or a clock may be built in the message server 2.

  The current time acquired by the time measuring unit 70 is transmitted to the message determining unit 63.

[Configuration of Message Table 82]
FIG. 3 is a diagram illustrating a data structure and a specific example of the message table 82.

  In FIG. 3, the message table 82 is shown as a data structure in a table format, and is not intended to limit the data structure of the message table 82 to the table format. Hereinafter, the same applies to other figures for explaining the data structure.

  As illustrated, the message table 82 includes a “message No” column, a “valid” column, a “keyword” column, a “presentation target” column, a “message content” column, and an “output possible period”. With columns.

  The “message No” column stores an index number assigned to each row of the message table 82.

  The “valid” column stores whether or not each row of the message table is to be referred to and searched when the message table is referenced and searched by the message determination unit 63.

  In the case of FIG. 3, when the “valid” column is “−”, the corresponding row is always a reference target and a search target. In other words, the “valid” column is “−” for messages that are not switched between valid / invalid of reference and search by the “valid” column. On the other hand, a message whose “valid” column is “◯” is a reference target and a search target, and a message whose “valid” column is “×” is not a reference target and a search target. In this “valid” column, ◯ / × can be freely switched by the user.

  The way of holding the data in the “valid” column shown in FIG. 3 is an example. If it is determined whether or not each row is a search target, the “valid” column can be any character string, flag, numerical value, or the like. You may memorize by type.

  The “keyword” column stores a keyword that should be included in the utterance of the user in the vicinity of the output device 1, which is a condition for causing the output device 1 to present a message. The type of data to be stored is not limited, but the data is stored in a format such as a character string.

  The “presentation target person” column stores information on a presentation target person who should present the message content, which is a condition for causing the output device 1 to present a message. The presentation target person may be one user, a plurality of persons, or a group name.

  The “message content” column stores the content of a message transmitted from the message server 2 to the output device 1. The type of data to be stored is not limited. For example, a message to be transmitted to the output device 1 is stored in a character string or the like.

  The “output available period” column stores information indicating a period or time zone in which the message content of each row may be presented. The output possible period may be defined by a period such as a start date and time to an end date and time, or may be a specific date and time such as every Tuesday or the 20th of every month.

  The message table 82 is configured such that if the information in the “keyword” column and the information in the “target person” column are determined, the information in the “message No” column is uniquely determined. In other words, if the keyword and the person to be presented are known, the index number of the message output to the output device 1 is determined, and the message content associated with the number is determined.

  The message table 82 may be predetermined in the message server 2, but may be stored so that it can be rewritten by the user.

[Modification of Message Table 82]
The message server 2 may automatically acquire message contents to be registered in the message table 82 from a news site via, for example, the Internet.

  FIG. 4A is a diagram illustrating another example of the message table 82. Note that the description of the same column as the message table 82 shown in FIG. 4 is omitted.

  The message table 82 shown in FIG. 3 is different from the message table 82 shown in FIG. 4A in that the message table 82 shown in FIG. 4A further has a “genre” column. It is.

  The “genre” column stores genre names in which messages (news) in each row are classified according to their content categories.

  Also, the message table 82 shown in FIG. 4A is different from FIG. 3 in that the message content is uniquely determined based on the keyword, the person to be presented, and the output possible period. In this way, not only the keyword and the presentation target person but also the output possible period is used as a table key so that even the same keyword and the same presentation target person present different news depending on the time zone (or date and time) It is possible to design the message server 2 and the output device 1.

  4B is a table in which the determined genre name is associated with the “presentation target” column of the message table 82 in FIG. The “genre” column in the table of FIG. 4B stores a genre name or a keyword for searching for news when automatically acquiring message contents from a news site. Further, the “presentation target person” column stores, for each genre name, a presentation target person who presents news of the genre in the output device 1.

  In the table of FIG. 4B, as shown, when the genre name is determined, the person to be presented who determines the news of the genre is determined.

The message server 2 can automatically generate data to be stored in the message table 82 according to the following procedures (1) to (3), using the tables shown in FIGS.
(1) Search for news to be acquired at a news site via the Internet, etc., and acquire the contents in the form of a character string or the like. At that time, the genre name stored in FIG. 4B is used as a search word for searching for news.

Instead of searching for news based on the genre name, news may be provided sequentially from a provider such as news or weather information. In this case, the message server 2 receives news in which the genre name stored in FIG. 4B matches the category name, keyword, tag, etc. in the contract (or setting) for providing news. Contract (or set) to do.
(2) Refer to the table in FIG. 4B, and match the genre name used as the search word in (1) or the news category, keyword, tag, etc. provided by the provider in (1). The presentation target person corresponding to the genre name to be determined is determined.
(3) The news acquired in (1) is registered as a message in the message table shown in FIG. At this time, the “valid” column, the “keyword” column, and the “output available period” column are set to predetermined values. The “message content” column stores the news content acquired in (1), and the “genre” column stores the genre name used as the search word in (1). The “presentation target person” column is the presentation target person determined in (2).

[Configuration of group presentation information 83]
FIG. 5 is a diagram illustrating a data structure and a specific example of the group presentation information 83.

  As illustrated, the group presentation information 83 includes a “message No” column, a “presentation target” column, and a “presentation” column.

  The “message No” column corresponds to the index number assigned to each row of the message table 82.

  The “presentation target person” column stores, for example, a user ID and a user name. Note that the person to be presented in the group presentation information 83 is different from the message table 82 in that one person is always designated instead of the group name and a plurality of persons.

  The “presentation” column stores whether or not the message content of the message table 82 corresponding to the message No in each row has been presented to each user in the “presentation target” column.

  The group presentation information 83 divides a message presentation target person into one user unit and associates it with a message No. for a message whose group name is registered in the “presentation target” column of the message table 82. Further, a “presentation” column indicating whether or not the message No and the user have presented the message No. is associated with the message No and the user.

The output necessity determination unit 64 searches the group presentation information 83 based on the message No. of the message determined by the message determination unit 63 and the presentation target person determined by the target person specifying unit 62, and sends it to the presentation target person. On the other hand, it is determined whether or not the message corresponding to the message No. has been presented.
[Message determination method]
Next, a method in which the message determination unit 63 (and the output necessity determination unit 64) determines a message to be transmitted to the message creation unit 65 will be described in detail.

The message determination unit 63 determines a message to be transmitted to the message creation unit 65 by performing the processes shown in the following (1) to (5).
(1) The keyword extracted by the keyword extracting unit 61 and the user identification information specified by the target person specifying unit 62 are received.
(2) The timer 70 is requested to measure the current time, and the timer 70 receives the current time acquired and timed.
(3) A search is made as to whether or not there is a message in the message table 82 that matches the keyword and the user identification information, and further includes the message whose current time is included in the outputtable period of the message table 82. Note that the record of the message table 82 to be searched at this time is valid in the “valid” column of the message table shown in FIG. 3 or is not valid / invalid switching (“◯” or “−” in the message table 82). Only records that are present.
(4) When the message is present, the output necessity determination unit 64 included in the message determination unit 63 lists a plurality of persons to be presented with the message associated with the message, or It is further determined whether or not the group name is designated. When the presentation target person designates a plurality of persons or a group name, the output necessity determination unit 64 performs the process (5). On the other hand, if it is not a plurality of persons, the output necessity determination unit 64 does not perform the process (5), and the message determination unit 63 transmits the message content to the message creation unit 65.
(5) The output necessity determination unit 64 refers to the value of the “presentation” column corresponding to the message No and the user identification information for the group presentation information in the storage unit 80. For the message No and the user identification information, if the message has not been presented (“not yet”), the output necessity determination unit 64 indicates that the message has been presented to the person to be presented with the user identification information. The information is recorded (“Done”) in the presentation information 83 and the message content is transmitted to the message creation unit 65. On the other hand, when the message has been presented (“Done”), the message determination unit 63 does not transmit the message content to the message creation unit 65.

[Process flow]
Next, a flow of processing in which the message presentation system 100 determines a message to be presented by the output device 1 and causes the output device 1 to present the message will be described with reference to FIG.

  FIG. 6 is a diagram illustrating a processing flow of the message presentation system 100.

  First, the sensor microphone 11 of the detection unit 10 detects the voice uttered by the person to be presented, and acquires the voice as utterance data (S100). The acquired utterance data is transmitted to the detection data acquisition unit 21.

  Subsequently, the camera 12 of the detection unit 10 takes a photograph or a moving image in the direction in which the sensor microphone 11 detects the sound. Thereby, the person to be presented who made the utterance is photographed and image data is acquired (S102). The acquired image data is transmitted to the detection data acquisition unit 21.

  The detection data acquisition unit 21 transmits the speech data and the image data received from the sensor microphone 11 and the camera 12 to the message server 2 via the communication unit 30 (S104). The server communication unit 50 receives the utterance data and the image data from the detection data acquisition unit 21 (S106), the utterance data is sent to the keyword extraction unit 61 and the target person specifying unit 62, and the image data is sent to the target person specifying unit 62. Send.

  The keyword extraction unit 61 extracts keywords from the utterance data received from the output device 1 via the server communication unit 50 (S108), and transmits the keywords to the message determination unit 63.

  Based on the utterance data and the image data received from the output device 1 via the server communication unit 50, the message determination unit 63 utters the voice of the utterance data or the user identification of the person to be presented that is reflected in the image data Information is specified (S110). The user identification information is transmitted to the message determination unit 63.

  When the message determination unit 63 receives a keyword from the keyword extraction unit 61 and user identification information from the target person identification unit 62, the message determination unit 63 requests the time measurement unit 70 to acquire and transmit the current time. The time measuring unit 70 acquires the current time (S112) and transmits it to the message determining unit 63.

  The message determination unit 63 that has received the current time reads the message table 82, and searches for a message in which the keyword and the user identification information match and the current time is included in the output available period (S114). ).

  When there is a message that matches the above condition (YES in S116), the output necessity determination unit 64 included in the message determination unit 63 determines whether the message is a single user, a plurality of people, Then, it is checked whether or not the group name is a presentation target person (S120).

  In the case where the message has one user as the subject of presentation (NO in S120), the message determination unit 63 extracts the message content of the message and transmits it to the message creation unit 65.

  On the other hand, if the message examined in S120 has a plurality of persons or a group name as a person to be presented (YES in S120), the output necessity determination unit 64 further provides the message to the person to be presented with the user identification information. Is determined (S130).

  If the message has not been presented to the person to be presented with the user identification information (NO in S130), the output necessity determination unit 64 indicates that the message has been presented to the person to be presented with the user identification information. The information is recorded in the group presentation information 83 (S132), and the message content of the message is transmitted to the message creation unit 65.

  The message creation unit 65 creates message data based on the received message content (S122), and transmits the message data to the message acquisition unit 22 of the output device 1 via the server communication unit 50 (S124).

  The message acquisition unit 22 of the output device 1 receives the message data (S126) and transmits the message data to the output unit 40. The output unit 40 outputs the message data (S128).

[Embodiment 2]
<About types of detection data>
In the message presentation system 100 described above, the detection unit 10 includes the camera 12 and the camera 12 acquires the image data of the person to be presented, but only the utterance data acquired by the sensor microphone 11 in the target person specifying unit 62. If it is possible to identify the person to be presented, the configuration of the camera 12 and the image data acquired by the camera 12 are not necessarily required.

  Further, in the message presentation system 100 described above, the data detected by the detection unit 10 includes at least audio data. For example, the camera 12 captures the face of the person to be presented that has approached the output device 1 and the action. It is good as well. The action here means that a keyword can be acquired from the action itself such as sign language or gesture.

  In this case, the moving image data captured by the camera 12 is transmitted to the message server 2, and the keyword extraction unit 61 determines what the above-described operation of the moving image data means, and generates a keyword from the determination result. Further, as described with reference to FIG. 1, the subject person specifying unit 62 is a presentation subject person who has approached the output device 1 based on the moving image data, for example, the face of the presentation subject person recorded in one frame of the moving image data. Is identified. The subsequent processing flow is the same as in the embodiment.

<Handling time information>
In the message presentation system 100 described above, the message determination unit 63 is a message associated with the keyword extracted by the keyword extraction unit 61 and the presentation target person specified by the target person specifying unit 62, and is acquired by the timing unit 70. The message in which the current time is included in the output available period is determined as a message to be output to the output device 1. However, it is not always necessary to store the output available period in the message table 82.

  If the message table 82 does not store the output available period, the message determination unit 63 determines the message to be transmitted to the output device 1 and the keyword extracted by the keyword extraction unit 61 and the target person specifying unit 62. The message is determined based on the presentation target person specified by. Therefore, since the message table 82 is not searched using the time information acquired by the time measuring unit 70, the time measuring unit 70 and the current time need not be acquired.

  Further, in the message presentation system 100 described above, the message server 2 has the time measuring unit 70, and the message determining unit 63 instructs the time measuring unit 70 to acquire the time when determining the message.

  However, if the time information necessary for the message determining unit 63 to determine the message to be output to the output device 1 is obtained, the time measuring unit 70 does not necessarily acquire the time at the above timing. For example, the time measuring unit 70 may acquire the time when the message server 2 receives the utterance data and the image data from the output device 1, and may transmit the time to the message determination unit 63.

  Furthermore, the file generation time of the data when the sensor microphone 11 or the camera 12 of the output device 1 generates the speech data or the image data may be used instead of the time measured by the time measuring unit 70. In this case, the message server 2 may not include the timer unit 70.

<About the function sharing of each device>
In the message presentation system 100 described above, the output device 1 and the message server 2 are separate devices, but the message server 2 may be built in the output device 1.

  Alternatively, in the message presentation system 100 described above, the output device 1 includes both the detection unit 10 and the output unit 40. However, the output device 1 does not have the detection unit 10, and obtains a message from the detection unit 10 as an external device. The configuration may be such that the utterance data and the image data are received by the unit 22. Alternatively, the output device 1 does not have the output unit 40, the message acquisition unit 22 further transmits the message data received from the message server 2 to the output unit 40, and the output unit 40 which is an external device outputs the message data. The structure of performing may be sufficient.

<About creating message data>
In the message presentation system 100 described above, the message creation unit 65 creates message data in a format that can be output by the output unit 40 of the output device 1 from the value of the message content acquired from the message table 82.

  However, the message server 2 may transmit the character string of the message content determined by the message determination unit 63 as it is as text data to the output device 1 without using the message creation unit 65.

  In this case, when the message acquisition unit 22 that acquired the text data transmits the message data to the output unit 40, the message acquisition unit 22 creates message data in a format that can be output by the output unit 40. With such a configuration, the server communication unit 50 has an advantage that the message data can be transmitted as small data such as text data. On the other hand, the message acquisition unit 22 of the output device 1 may further include a configuration for converting the received message data into a format such as artificial speech.

<About communication style>
In the case where the communication unit 30 transmits data to the server communication unit 50, that is, in the case of uplink communication, and in the case where the communication unit 30 receives data from the server communication unit 50, that is, in the case of downlink communication, The communication modes may be divided. For example, in the case of uplink communication, since audio data and image data having a relatively large data size are transmitted, a communication method capable of transmitting a large amount of data quickly is desirable. On the other hand, for example, when message data is received as data having a small data size in downlink communication, a transfer capacity is not required as in uplink communication.

[Summary]
A message management apparatus (message server 2) according to an aspect of the present invention is a message management apparatus that transmits a message to be presented to a presentation target person to the message presentation apparatus (output apparatus 1), and the utterance of the presentation target person Utterance acquisition means (server control unit 60) for acquiring utterance data generated by acquiring utterances, and keyword extraction means (keyword extraction unit 61) for extracting keywords included in the utterance data acquired by the utterance acquisition means; The target person specifying means (target person specifying unit 62) for specifying the presentation target person based on the utterance data acquired by the utterance acquisition means, the keyword extracted by the keyword extracting means, and the target person A message associated with the target person specified in advance by the specifying means is stored in the message presentation device. And a message identification means (message determination unit 63) for identifying a message to be sent to.

  The message management apparatus control method according to an aspect of the present invention is a message management apparatus control method for transmitting a message to be presented to a presentation target person to the message presentation apparatus, and acquires the utterance of the presentation target person. In the utterance acquisition step (S106) for acquiring the generated utterance data, the keyword extraction step (S108) for extracting the keyword included in the utterance data acquired in the utterance acquisition step, and the utterance acquisition step A subject identification step (S110) for identifying the subject to be presented based on the acquired utterance data, the keyword extracted in the keyword extraction step, and the presentation identified in the subject identification step A message that sends a message associated with the target person in advance to the message presentation device. And a message specifying step (S114~116) be identified as sage.

  According to the above configuration and method, the message management apparatus acquires the utterance data of the person to be presented, extracts a keyword from the utterance data, identifies the person to be presented from the utterance data, and identifies the keyword and the A message associated with the presentation target person in advance can be specified as a message to be presented to the presentation target person by the message presentation device.

  Accordingly, the message to be presented to the message presentation device can be changed based on who the presentation target person is and what the presentation target person has spoken.

  Therefore, the message management device can cause the message presentation device to present an optimum message according to the utterance content of each presentation subject person.

  Furthermore, in the message management device according to one aspect of the present invention, the message is associated with the keyword and the person to be presented, and further associated with an output possible period (message table 82). It further comprises time acquisition means (time keeping unit 70) for acquiring the time when the presentation target person speaks or substantially the same time, and the message specifying means includes the keyword extracted by the keyword extracting means and the target person specifying A message associated with the presentation target person specified by the means and associated with the output possible period including the time acquired by the time acquisition means is specified as a message to be transmitted to the message presentation device. It is desirable.

  According to the above configuration, the message management device further includes a message corresponding to the presentation target specified from the utterance data and the keyword extracted from the utterance data, and the acquired current time is associated with the message. The message included in the output available period can be a message transmitted to the message presentation device.

Here, the “substantially the same time” acquired by the time acquisition means can be arbitrarily selected as long as it can be substantially regarded as “the time when the presentation target person speaks” when determining a message. For example, the following times (1) to (4) can be used.
(1) Time when utterance data is generated (2) Time when utterance data is transmitted to message management device (3) Time when message management device receives utterance data (4) Time when message management device specifies message The message presentation device needs to output a message with good response to the user's utterance. Therefore, the process from the user's utterance to the message determination and message output in the message management apparatus is performed in a few seconds at the longest. Therefore, the times of (1) to (4) may be regarded as indicating substantially the same time, and the time acquisition means may acquire the time of any time of (1) to (4).

  Furthermore, the message management device according to an aspect of the present invention further includes an image acquisition unit (server control unit 60) that acquires image data generated by imaging the presentation target person, and the target person specifying unit includes: It is desirable to specify the person to be presented based on the utterance data acquired by the utterance acquisition unit and the image data acquired by the image acquisition unit.

  According to the above configuration, the message management device can further identify the target person by combining the utterance data of the target person and the image data.

  Thereby, compared with the case where a presentation subject person is specified only based on utterance data, the specific precision of a presentation subject person rises and it becomes possible to specify a presentation subject person more correctly.

  Furthermore, in the message management device according to an aspect of the present invention, one keyword and a plurality of subjects to be presented are associated with one message, and the keyword specifying unit extracts the keyword by the keyword extracting unit. The message is displayed so that the message associated with the keyword is presented to each of the plurality of presentation subjects associated with the message at least once. You may specify as a message transmitted to an apparatus.

  According to the above configuration, the message management device further sends the message only once to any one of the plurality of presentation subjects even when one message is associated with the plurality of presentation subjects. Instead of sending the message, it is possible to send the message to all the corresponding presentation target persons at least once. Accordingly, it is possible to prevent the transmission of a message to the plurality of presentation target persons one by one.

  A message presentation device (output device 1) according to an aspect of the present invention is a message presentation device that presents a message to a presentation target person, and utterance data generation means that acquires the utterance of the presentation target person and generates utterance data. (Sensor microphone 11), a keyword extracted from the utterance data generated by the utterance data generation means, and a message for acquiring a message associated in advance with the presentation target person specified based on the utterance data An acquisition unit (message acquisition unit 22) and a message output unit (output unit 40) for presenting the message acquired by the message acquisition unit to a person to be presented.

  Further, a control method for a message presentation device according to an aspect of the present invention is a control method for a message presentation device that presents a message to a presentation target person, and the utterance that acquires the utterance of the presentation target person and generates utterance data Data generation step (S100), a keyword extracted from the utterance data generated in the utterance data generation step, and a message associated in advance with the presentation target specified based on the utterance data A message acquisition step (S126), and a message output step (S128) for presenting the message acquired in the message acquisition step to the presentation target person.

  According to the above configuration and method, the message presentation device acquires the utterance of the person to be presented, generates utterance data, and is specified based on the keyword extracted from the utterance data in the message management device and the utterance data. The message associated with the presentation target person can be acquired, and the acquired message can be presented to the presentation target person.

  Thereby, the optimal message according to the content of utterance of this presentation subject person can be shown to each presentation subject person.

  Furthermore, the message presentation device according to an aspect of the present invention includes a keyword extraction unit (keyword extraction unit 61) that extracts a keyword included in the utterance data generated by the utterance data generation unit, and the utterance data generation unit. Based on the generated utterance data, the target person specifying means (target person specifying unit 62) for specifying the target person to be presented, the keyword extracted by the keyword extracting means, and the target person specifying means are specified. In addition, message specifying means (message determining unit 63) for specifying a message associated with the presentation target person in advance as a message to be presented to the presentation target person may be further provided.

  According to the above configuration, the message presentation device further acquires the utterance data of the presentation target person, extracts keywords from the utterance data, specifies the presentation target person, and determines the message to be presented. Can be done alone.

  A message presentation system according to an aspect of the present invention includes the message management device and the message presentation device.

  The message management device and the message presentation device may be realized by a computer. In this case, a message that causes the message management device and the message presentation device to be realized by the computer by operating the computer as each of the above means. A control program for the management device and the message presentation device, and a computer-readable recording medium on which the control program is recorded also fall within the scope of the present invention.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims, and the embodiments can be obtained by appropriately combining technical means disclosed in the embodiments. Is also included in the technical scope of the present invention. Furthermore, a new technical feature can be formed by combining the technical means disclosed in the embodiments.

[Example of software implementation]
Finally, each block of the output device 1 and the message server 2, particularly the server control unit 60, may be realized in hardware by a logic circuit formed on an integrated circuit (IC chip), or may be a CPU (Central Processing). Unit) may be implemented in software.

  In the latter case, the output device 1 and the message server 2 include a CPU that executes instructions of a program that realizes each function, a ROM (Read Only Memory) that stores the program, a RAM (Random Access Memory) that expands the program, A storage device (recording medium) such as a memory for storing the program and various data is provided. The object of the present invention is to record the program code (execution format program, intermediate code program, source program) of the control program of the output device 1 and the message server 2 which are software for realizing the above-described functions so as to be readable by a computer. This can also be achieved by supplying a recording medium to the message server 2 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include non-transitory tangible medium, such as magnetic tape and cassette tape, magnetic disk such as floppy (registered trademark) disk / hard disk, and CD-ROM / MO. Discs including optical discs such as / MD / DVD / CD-R, cards such as IC cards (including memory cards) / optical cards, semiconductor memories such as mask ROM / EPROM / EEPROM (registered trademark) / flash ROM Alternatively, logic circuits such as PLD (Programmable Logic Device) and FPGA (Field Programmable Gate Array) can be used.

  The output device 1 and the message server 2 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited as long as it can transmit the program code. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication network, and the like can be used. The transmission medium constituting the communication network may be any medium that can transmit the program code, and is not limited to a specific configuration or type. For example, even with wired lines such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, and ADSL (Asymmetric Digital Subscriber Line) line, infrared rays such as IrDA and remote control, Bluetooth (registered trademark), IEEE 802.11 wireless, HDR ( It can also be used by radio such as High Data Rate (NFC), Near Field Communication (NFC), Digital Living Network Alliance (DLNA), mobile phone network, satellite line, and digital terrestrial network. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention relates to a message presentation system that presents a message having an appropriate content to a user in response to the user's utterance.

1 Output device (message presentation device)
2 Message server (message management device)
10 Detection Unit 11 Sensor Microphone (Speech Data Generation Unit)
12 Camera 20 Control unit 21 Detection data acquisition unit 22 Message acquisition unit (message acquisition means)
30 communication unit 40 output unit (message output means)
50 server communication unit 60 server control unit (speech acquisition means, image acquisition means)
61 Keyword extraction unit (keyword extraction means)
62 Target person specifying part (Target person specifying means)
63 Message determination unit (message specifying means)
64 Output Necessity Determination Unit 65 Message Creation Unit 70 Timekeeping Unit (Time Acquisition Unit)
80 Storage Unit 81 Authentication Information 82 Message Table 83 Group Presentation Information 100 Message Presentation System

Claims (10)

A message management device that transmits a message to be presented to a presentation target person to a message presentation device,
Utterance acquisition means for acquiring utterance data generated by acquiring the utterance of the person to be presented;
Keyword extraction means for extracting keywords included in the utterance data acquired by the utterance acquisition means;
Subject identification means for identifying the subject to be presented based on the utterance data acquired by the utterance acquisition means;
Message specifying means for specifying the keyword extracted by the keyword extracting means and a message associated with the presentation target person specified by the target person specifying means in advance as a message to be transmitted to the message presentation device; equipped with a,
The message is associated with the keyword and the person to be presented, and further associated with the output possible period.
Further comprising time acquisition means for acquiring the time when the presentation subject speaks or substantially the same time,
The message specifying means is associated with the keyword extracted by the keyword extracting means and the presentation target person specified by the target person specifying means, and includes the time acquired by the time acquisition means A message management device that identifies a message associated with a possible output period as a message to be transmitted to the message presentation device.   A message management device that transmits a message to be presented to a presentation target person to a message presentation device,
  Utterance acquisition means for acquiring utterance data generated by acquiring the utterance of the person to be presented;
  Keyword extraction means for extracting keywords included in the utterance data acquired by the utterance acquisition means;
  Subject identification means for identifying the subject to be presented based on the utterance data acquired by the utterance acquisition means;
  Message specifying means for specifying the keyword extracted by the keyword extracting means and a message associated with the presentation target person specified by the target person specifying means in advance as a message to be transmitted to the message presentation device; With
  One message and a plurality of target audiences are associated with one message,
  When the keyword is extracted by the keyword extracting unit, the message specifying unit sends the message associated with the keyword to each of the plurality of presentation subjects associated with the message. A message management device that identifies the message as a message to be transmitted to the message presentation device so that the message is presented at least once. Image acquisition means for acquiring image data generated by imaging the person to be presented is further provided,
The target person specifying unit specifies the target person to be presented based on the utterance data acquired by the utterance acquisition unit and the image data acquired by the image acquisition unit. Message management device. A message presentation device for presenting a message to a presentation target person,
Utterance data generation means for acquiring the utterance of the person to be presented and generating utterance data;
The speech data generating means keywords extracted from the speech data generated by the target person specified on the basis of the speech data, and a pre-correlated obtained message in the output period, and the presented objects Message acquisition means for acquiring a message in which the time when the person uttered or almost the same time is included in the output possible period ;
A message output device comprising: message output means for presenting the message acquired by the message acquisition means to a presentation target person. A message presentation device for presenting a message to a presentation target person,
Utterance data generation means for acquiring the utterance of the person to be presented and generating utterance data;
A message acquisition means for acquiring a keyword extracted from the utterance data generated by the utterance data generation means, and a message associated beforehand with the presentation target person specified based on the utterance data;
Message output means for presenting the message acquired by the message acquisition means to a presentation target person,
One message and a plurality of target audiences are associated with one message,
The message output means presents the message acquired by the message acquisition means at least once to each of the plurality of presentation subjects associated with the message. . Keyword extracting means for extracting a keyword contained in the utterance data generated by the utterance data generating means;
Subject identification means for identifying the subject to be presented based on the utterance data generated by the utterance data generation means;
A message specifying means for specifying the keyword extracted by the keyword extracting means and a message previously associated with the presentation target person specified by the target person specifying means as a message to be presented to the presentation target person; The message presentation device according to claim 4, further comprising: A control method of a message management device that transmits a message to be presented to a presentation target person to a message presentation device,
An utterance acquisition step of acquiring utterance data generated by acquiring the utterance of the person to be presented;
A keyword extraction step of extracting a keyword included in the utterance data acquired in the utterance acquisition step;
A subject identification step that identifies the subject to be presented based on the utterance data acquired in the utterance acquisition step;
Message specification that specifies the keyword extracted in the keyword extraction step and the message associated in advance with the presentation target person specified in the target person specifying step as a message to be transmitted to the message presentation device and a step seen including,
The message is associated with the keyword and the person to be presented, and further associated with the output possible period.
A time acquisition step of acquiring a time when the presentation target person speaks or substantially the same time as the utterance time;
In the message specifying step, the keyword extracted in the keyword extracting step and the time acquired in the time acquiring step and associated with the presentation target specified in the target specifying step A message management device control method characterized by specifying a message associated with the output possible period including a message to be transmitted to the message presentation device.   A control method of a message management device that transmits a message to be presented to a presentation target person to a message presentation device,
  An utterance acquisition step of acquiring utterance data generated by acquiring the utterance of the person to be presented;
  A keyword extraction step of extracting a keyword included in the utterance data acquired in the utterance acquisition step;
  A subject identification step that identifies the subject to be presented based on the utterance data acquired in the utterance acquisition step;
  Message specification that specifies the keyword extracted in the keyword extraction step and the message associated in advance with the presentation target person specified in the target person specifying step as a message to be transmitted to the message presentation device Including steps,
  One message and a plurality of target audiences are associated with one message,
  In the message specifying step, when the keyword is extracted in the keyword extracting step, the message associated with the keyword is assigned to each of the plurality of presentation target persons associated with the message. The message management device control method is characterized in that the message is specified as a message to be transmitted to the message presentation device so that the message is presented at least once.   A method for controlling a message presentation device for presenting a message to a presentation target person,
  An utterance data generation step of acquiring the utterance of the person to be presented and generating utterance data;
  The keyword extracted from the utterance data generated in the utterance data generation step, the presentation target specified based on the utterance data, and a message previously associated with the output available period, and the presentation A message acquisition step of acquiring a message in which the time when the subject speaks or substantially the same time is included in the output possible period;
  And a message output step of presenting the message acquired in the message acquisition step to a presentation target person.   A method for controlling a message presentation device for presenting a message to a presentation target person,
  An utterance data generation step of acquiring the utterance of the person to be presented and generating utterance data;
  A message acquisition step of acquiring a keyword extracted from the utterance data generated in the utterance data generation step, and a message associated beforehand with the presentation target specified based on the utterance data;
  A message output step of presenting the message acquired in the message acquisition step to a presentation target person,
  One message and a plurality of target audiences are associated with one message,
  In the message output step, the message acquired in the message acquisition step is presented at least once to each of the plurality of presentation subjects associated with the message. Control method of presentation device.

Images