EP3605316A1 - Dispositif de traitement d'informations, procédé et programme de traitement d'informations - Google Patents
Dispositif de traitement d'informations, procédé et programme de traitement d'informations Download PDFInfo
- Publication number
- EP3605316A1 EP3605316A1 EP18776223.2A EP18776223A EP3605316A1 EP 3605316 A1 EP3605316 A1 EP 3605316A1 EP 18776223 A EP18776223 A EP 18776223A EP 3605316 A1 EP3605316 A1 EP 3605316A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- message
- user
- information
- information processing
- case
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
- 230000010365 information processing Effects 0.000 title claims abstract description 171
- 238000003672 processing method Methods 0.000 title claims abstract description 8
- 238000001514 detection method Methods 0.000 claims abstract description 136
- 238000006243 chemical reaction Methods 0.000 claims abstract description 103
- 230000004044 response Effects 0.000 claims abstract description 51
- 238000012545 processing Methods 0.000 claims description 82
- 238000004458 analytical method Methods 0.000 claims description 14
- 238000005516 engineering process Methods 0.000 abstract description 19
- 238000012790 confirmation Methods 0.000 description 44
- 238000010586 diagram Methods 0.000 description 44
- 230000009471 action Effects 0.000 description 24
- 230000007704 transition Effects 0.000 description 9
- 230000005540 biological transmission Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 6
- 238000013459 approach Methods 0.000 description 5
- 230000006835 compression Effects 0.000 description 5
- 238000007906 compression Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000000034 method Methods 0.000 description 4
- 238000010411 cooking Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000005406 washing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/02—Methods for producing synthetic speech; Speech synthesisers
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
- G10L15/1815—Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/42—Mailbox-related aspects, e.g. synchronisation of mailboxes
Definitions
- the present technology relates to an information processing apparatus, an information processing method, and a program, and relates to, for example, an information processing apparatus, an information processing method, and a program for enabling a message to be more reliably conveyed to a user whom the message is desired to be conveyed to.
- Patent Document 1 proposes that a user whom information is desired to be conveyed to is detected by image recognition, and the information is conveyed when the user has been detected.
- Patent Document 1 Japanese Patent Application Laid-Open No. 2003-208190
- the present technology has been made in view of such a situation, and enables information to be more reliably conveyed to the user.
- An information processing apparatus includes a presentation unit configured to present information to a first user, a detection unit configured to detect a reaction indicating that the first user has received the information, a search unit configured to search for a second user in a case where the detection unit has not been able to detect the reaction, and a request unit configured to request the second user found by the search unit to convey a message to the first user.
- An information processing method includes presenting information to a first user, detecting a reaction indicating that the first user has received the information, searching for a second user in a case of having not been able to detect the reaction, and requesting the found second user to convey a message to the first user.
- a program causes a computer to execute processing including the steps of presenting information to a first user, detecting a reaction indicating that the first user has received the information, searching for a second user in a case of having not been able to detect the reaction, and requesting the found second user to convey a message to the first user.
- information is presented to a first user, a reaction that the first user has received the information is detected, a second user is searched for in a case where the reaction has not been detected, and conveyance of a message to the first user is requested to the searched second user.
- the information processing apparatus may be an independent apparatus or may be internal blocks configuring one apparatus.
- the program can be provided by being transmitted via a transmission medium or by being recorded on a recording medium.
- information can be more reliably conveyed to a user.
- Fig. 1 is a diagram illustrating a configuration of an embodiment of an information processing system to which the present technology is applied.
- the information processing system has a configuration including an information acquisition unit 11, an information processing unit 12, an external service providing unit 21, and an out-of-home network 22.
- the information acquisition unit 11, the information processing unit 12, and the external service providing unit 21 are connected via the out-of-home network 22, and are configured to give and receive data to and from one another.
- the information acquisition unit 11 and the information processing unit 12 are illustrated as devices installed in home. However, as in the information processing system illustrated in Fig. 2 , the information processing unit 12 may be installed outside the home and configured to give and receive data to and from the information acquisition unit 11 via the out-of-home network 22.
- the information acquisition unit 11 and the information processing unit 12 are installed in home and connected via a network constructed in home, for example, a local area network (LAN) configured in a wireless or wired manner, and are configured to give and receive data to and from each other via the network.
- a network constructed in home, for example, a local area network (LAN) configured in a wireless or wired manner, and are configured to give and receive data to and from each other via the network.
- LAN local area network
- the information processing unit 12 is connected to the external service providing unit 21 via the out-of-home network 22, and is configured to give and receive data to and from the external service providing unit 21.
- information is acquired by the information acquisition unit 11.
- the acquired information is, for example, a reaction of a user, an utterance of a user, a conversation between users, information for identifying a user, a surrounding situation, or the like.
- the information acquired by the information acquisition unit 11 is supplied to and processed by the information processing unit 12.
- the information processing unit 12 executes, for example, processing of conveying a message from a predetermined user (called sender) to a user specified by the sender as a recipient of the message, and requesting a user (called messenger) other than the recipient to convey the message to the recipient in a case where the message cannot be directly conveyed to the recipient. Furthermore, the information processing unit 12 makes an inquiry about predetermined information to the external service providing unit 21, and provides an inquiry result to the user.
- the external service providing unit 21 may be a server so-called search server, or a server that manages a shopping site that offers shopping and the like.
- the information processing unit 12 receives a message from a sender (father in Fig. 3 , hereinafter described as sender 31), and conveys the message to a recipient (mother in Fig. 3 , hereinafter described as recipient 32).
- the information processing unit 12 receives an email of "I'll come home late today” from the sender, and thus conveys a message such as "You've got an email from Father “I'll come home late today”” to the recipient 32 by voice, for example.
- the recipient 32 is, for example, cooking, and cannot hear the message from the information processing unit 12 and does not take any reaction.
- examples of the situation where the recipient 32 does not show any reaction include a situation where the recipient 32 does not notice the message and does not take any reaction, and a situation where the recipient 32 does notice the message but does not take any reaction for any reason such as being busy.
- the information processing unit 12 repeatedly conveys the message to the recipient 32, detects the reaction of the recipient 32 each time, and determines whether or not there is a reaction. In a case where such determination is repeated, for example, five times, but it is determined that the recipient 32 has not taken a reaction, words for confirming whether or not the message has been conveyed are added and the message is conveyed again.
- the information processing unit 12 conveys a message such as "You've got an email from Father. He says “I'll come home late today”. Would you like to replay?" to the recipient. In this case, words such as "Would you like to reply?" for confirming whether or not the message has been conveyed are added.
- the message from the information processing unit 12 has been determined not to reach the recipient 32.
- the information processing unit 12 searches surroundings at time T3. This search is performed to find a user (referred to as a messenger 33) near the information processing unit 12, in other words, within a range where the message is conveyable from the information processing unit 12.
- a user who is in the room is set as a search target.
- the found child is set as the messenger 33 who conveys the message to the recipient 32 (who conveys that there is the message).
- the messenger 33 is a user who is given a role of conveying the message to the recipient 32 on behalf of the information processing unit 12 or conveying that there is the message from the information processing unit 12 to the recipient 32, when the message from the sender 31 is not conveyed to the recipient 32.
- the sender 31 may also be included in the messenger 33.
- the messenger 33 can be a user other than the recipient 32
- the messenger 33 In a case where the messenger 33 has been found at time T3, the messenger 33 is notified of a message state at time T4. Then, at time T5 ( Fig. 4 ), the information processing unit 12 sends a message such as "there is something to be conveyed to "mother”" to the messenger 33. Whether or not the messenger 33 (the child in this case) has taken any reaction, for example, has raised a hand or has replied to the message is determined.
- the messenger 33 In a case where the messenger 33 has been determined to take a reaction to the message, the message is determined to be conveyed. In a case where such determination has been made, at time T6, a request type in a message mode is calculated, and in the case of this example, a request for getting the mother is created.
- the request type is set according to the age of the messenger 33, the content of the message from the sender 31 (the message desired to be conveyed to the recipient 32), or the like, as will be described below.
- the information processing unit 12 sends a message such as "Would you go and get "mother”?" to the messenger 33.
- the messenger 33 understands the message from the information processing unit 12 and takes an action. In this case, the child who is the messenger 33 goes and gets the mother who is the recipient 32.
- the messenger 33 goes to the recipient 32 and the recipient 32 approaches the information processing unit 12, the message is conveyed to the recipient 32.
- the information processing unit 12 conveys the message "You've got an email from Father. He says “I'll come home late today”” to the recipient 32 (mother).
- Fig. 5 is a diagram illustrating a configuration of an example of the information processing unit 12.
- the information processing unit 12 illustrated in Fig. 5 includes a configuration of the information acquisition unit 11.
- the configuration of the information processing unit 12 in a case where the information acquisition unit 11 and the information processing unit 12 are integrated will be described. Therefore, part of the configuration illustrated in Fig. 5 may be the information acquisition unit 11 and separately configured from the information processing unit 12.
- the information processing unit 12 includes a voice acquisition unit 101, a voice output unit 102, a voice processing unit 103, an image acquisition unit 104, a display unit 105, an image processing unit 106, a communication unit 107, an operation input unit 18, a control unit 109, and a database 110.
- the voice acquisition unit 101 includes, for example, a microphone, and acquires a sound uttered by the user, an ambient sound, and the like.
- the voice output unit 102 includes, for example, a speaker, and outputs a voice when conveying a message to the user by voice.
- the voice processing unit 103 processes the voice acquired by the voice acquisition unit 101, and generates a voice to be output by the voice output unit 102.
- the voice acquisition unit 101 acquires content uttered by the recipient 32, and the voice processing unit 103 performs determination by analyzing the acquired voice.
- the voice processing unit 103 when sending a message to the recipient 32 at the time T1, the voice processing unit 103 generates voice data of the message to be output and the voice output unit 102 outputs the message.
- the image acquisition unit 104 includes, for example, a camera, and captures and acquires images of the user and the surroundings.
- the display unit 105 includes, for example, a display, and displays an image when conveying a message to the user with the image.
- the image processing unit 106 processes the image acquired by the image acquisition unit 104, and generates an image to be displayed on the display unit 105.
- the image acquisition unit 104 captures an image
- the image processing unit 106 performs a search by analyzing whether or not a user who can be the messenger 33 has been captured in the captured image.
- the image processing unit 106 when sending a message to the recipient 32 at the time T3, the image processing unit 106 generates image data (or text data or the like) of the message to be output, and the display unit 105 displays the message.
- the communication unit 107 gives and receives data to and from the external service providing unit 21 via the in-home network 22 ( Fig. 1 ), and gives and receives data to and from another information acquisition unit 11 or another information processing unit 12 in a case where a plurality of information acquisition units 11 and information processing units 12 are provided.
- the operation input unit 108 includes a touch panel, a keyboard, and the like, and receives an operation from the user.
- the control unit 109 controls each unit in the information processing unit 12.
- the database 110 includes a plurality of databases, and stores data and tables required when the above-described processing is executed.
- the database 110 stores a plurality of data and tables required when the above-described processing is executed. Here, description will be added regarding the plurality of databases included in the database 110.
- Fig. 6 is a diagram illustrating an example of a user attribute information database 110a.
- the user attribute information database 110a is a database that manages name, gender, age, and hearing in association with one another. For example, names such as “mother”, “child”, and “father” are managed as the name. Note that, here, description will be continued using "mother” or the like, which is not a name in general, as the name, for the sake of description.
- the name “mother” in the user attribute information database 110a is associated with information of the gender “female”, the age “38”, and the hearing "normal”.
- the name “child” is associated with information of the gender “male”, the age "5", and the hearing "good”.
- the name “father” is associated with information of the gender “male”, the age "40", and the hearing "poor”.
- Such a user attribute information database 110a is constructed by acquiring information from the user, for example, acquiring information regarding a reaction from the user when a message has been sent.
- Fig. 7 is a diagram illustrating an example of a detection information database 110b.
- the detection information database 110b is a database that manages information detected by the information processing unit 12.
- items of the detection information database 110b illustrated in Fig. 7 "object recognition information”, "position detection”, “existence direction”, “distance”, “face detection”, “nod detection”, “head direction”, “volume”, “voice recognition result”, “semantic analysis result”, “gaze detection”, and “gesture detection” are provided. Furthermore, these items are provided as detection items for subjects such as “mother”, “child”, “father”, “noise source A”, and “noise source B”.
- the detection information database 110b not only a detection result regarding a person but also a detection result regarding a sound is recorded in the detection information database 110b.
- a sound may hinder transmission when a message is conveyed by voice and is thus a detection target.
- the detection information database 110b illustrated in Fig. 7 detection results that a burner is detected as the noise source A, the burner is located 5 m away in the south direction, and the volume is high (large) are written.
- Such a detection information database 110b is updated at predetermined intervals. Furthermore, detection processing may be performed as needed when, for example, a message is output, and a result of the detection processing may be written. Furthermore, usually, the detection processing is performed at long intervals and the result is written, and when a message is output, the detection processing may be performed at short intervals and the result may be written.
- Fig. 8 is a diagram illustrating an example of a voice output information database 110c.
- the voice output information database 110c is a database that manages a message output as voice.
- the voice output information database 110c is a database that manages an output content ID, output content, a notification destination, a notification source, time, and urgency in association with one another.
- the output content of "Mother's favorite singer A will appear on an Internet TV program PPP from 19:30 today", the notification destination of "mother”, the notification source of "system”, the time (time of notification) of "12:00”, the urgency of "normal” are managed in association with one another.
- the notification source is the "system”
- this case indicates that the information processing unit 12 has communicated with the external service providing unit 21, for example, and has conveyed the acquired information to the user.
- the output content ID of "2” the output content of "contact with school immediately”, the notification destination of "mother”, the notification source of “school”, and the time of "12:15”, and the urgency of "high” are managed in association with one another.
- Data is added to such a voice output information database 110c when a message is conveyed to the user by voice.
- a function to delete the data in chronological order or a function to delete the data when the message has been successfully transmitted to the user may be provided.
- Fig. 9 is a diagram illustrating an example of a voice output trial result information database 110d.
- the voice output trial result information database 110d is a database that manages a result indicating whether or not a message has been successfully transmitted to the user (recipient 32) as a result of outputting the message.
- the voice output trial result information database 110d is a database that manages an output content ID, the number of achieved output trials, an output trial time, and TTS control in association with one another.
- TTS is an abbreviation of text-to-speech, and indicates processing regarding text-to-speech for artificially synthesizing human voice and converting texts (sentence) into a voice.
- the same ID as the output content ID of the voice output information database 110c illustrated in Fig. 8 is assigned to the same message.
- the output trial being performed five times, the output trial time of 10 sec, the TTS control of "Normal” are managed in association with one another.
- the output trial being performed zero times (not yet output), the output trial time of "not yet”, and the TTS control of "high pitch, time shortened” are managed in association with one another.
- Such a voice output trial result information database 110d is updated when a message is output. Furthermore, the detection information database 110b ( Fig. 7 ) is also updated when the voice output trial result information database 110d is updated.
- the voice output trial result information database 110d For example, for the message with the output content ID of "1", a result that the output trial has been performed five times is described in the voice output trial result information database 110d.
- the trial of five times means that the message has been repeatedly output because the recipient 32 (the mother in this case) had not shown a reaction that she had received the message.
- information of "No" is written in the items of face detection and nod detection.
- the face of the recipient 32 can be detected.
- the information processing unit 12 can detect the nodding.
- the face detection and the nod detection can be performed by capturing the user (recipient 32) by the image acquisition unit 104 ( Fig. 5 ), and analyzing the captured image (video) by the image processing unit 106, for example.
- the recipient 32 when a message is output, the recipient 32 is detected from the captured image of the recipient 32, whereby the face detection is performed.
- the nod detection is performed by comparing captured images of the recipient 32 before and after outputting the message or analyzing the video, detecting the position of the head of the user, and determining whether or not the user has performed an action corresponding to a nodding action from the movement of the head.
- information of "0" is written in the item of volume.
- information of "None” is written in the items of voice recognition result and semantic analysis result.
- the information processing unit 12 can detect the voice as the volume.
- the recipient 32 has uttered some voice
- utterance of words indicating that the message has been received such as "OK”
- the recipient 32 having reacted to the reception of the message can be detected.
- the reaction to the reception of the message is detected when the voice recognition result is semantically analyzed.
- the reaction not to the reception of the message is detected when the voice recognition result is semantically analyzed.
- Information of the volume, the voice recognition result, and the semantic analysis result is output by acquiring the voice of the user (recipient 32) by the voice acquisition unit 101 ( Fig. 5 ) and analyzing the acquired voice by the voice processing unit 103.
- the volume is detected. Furthermore, what kind of words has been uttered can be detected by recognizing the voice.
- whether or not the reaction is to the reception of the message, or whether or not the utterance is not related to the reception of the message can be determined by semantically analyzing the voice recognition result. For example, when the result of the semantic analysis is a result such as "yes” or "ok", it can be determined that the reaction is to the reception of the message.
- the voice processing unit 103 includes a database and a function for performing such voice recognition and semantic analysis.
- information of "No" is written in the item of gaze detection.
- detection of an action of looking at the information processing unit 12 is set as the reaction to the reception of a message, in other words, in a case where the gesture of looking at the information processing unit 12 is one of reactions to the reception of a message
- detection of the gaze is performed by capturing the user (recipient 32) by the image acquisition unit 104 ( Fig. 5 ) and analyzing the captured image (video) by the image processing unit 106.
- the information of the gaze detection in the detection information database 110b is "Yes” in a case where the detection direction is directed to the information processing unit 12, and is "No” in a case where the detection direction is not directed to the information processing unit 12.
- information of "No” is written in the item of gesture detection.
- a gesture predefined as the reaction to the reception of a message is detected, as in the gaze detection
- information of "Yes” is written in the item
- information of "No” is written in a case where the gesture is not detected.
- the gesture for example, there is an action such as raising a hand or waving a hand.
- the detection information database 110b is also updated along with the update of the voice output trial result information database 110d.
- the voice output information database 110c has been illustrated in Fig. 8 and the voice output trial result information database 110d has been illustrated in Fig. 9 .
- the voice output information database 110c and the voice output trial result information database 110d may be configured as one database.
- Such databases are managed by the database 110. Furthermore, the database 110 also manages a plurality of tables described below.
- Fig. 10 is a diagram illustrating an example of a conveyance confirmation utterance generation template table 110e.
- the conveyance confirmation utterance generation template table 110e is a table that is referred to when a conveyance confirmation utterance is performed at the time T2 when a reaction of the recipient 32 has not been able to be obtained even though a message has been repeatedly conveyed five times at the time T1 in the description with reference to Figs. 3 and 4 , for example.
- the conveyance confirmation utterance generation template table 110e is a table that is referred to when determining what kind of conveyance confirmation utterance is to be performed when a conveyance confirmation utterance is performed.
- the conveyance confirmation utterance generation template table 110e is a table that manages a template ID and a template in association with each other. Furthermore, the conveyance confirmation utterance generation template table 110e illustrated in Fig. 10 manages, as the template, a template for motion response promotion and a template for utterance response promotion.
- the template ID for motion promotion of "M-1-1” is a template such as "(name info), please raise your hand in a case where you can immediately respond to (system info)".
- a message for respond promotion such as "Mother, please raise your hand if you can immediately respond to the message” is output.
- templates such as "Please turn this way” and “Please nod” are prepared in addition to the template of "Please raise your hand”. Which template is selected from among these templates can be made in consideration of the state of the recipient 32, for example.
- the template such as "Please turn this way” or "Please nod” is selected.
- the template such as "Please raise your hand” or "Please nod” is selected.
- the template ID of "V-1-1" is a template such as "(name info), can you or can't you respond to the message of (output info)?".
- a message for response promotion such as "Mother, can you or can't you respond to the message of "I'll come home late today” from Father?" is output.
- the template for utterance promotion is formed in a question form such as "can you or can't you?" to obtain a replay.
- the conveyance confirmation utterance generation template table 110e is referred to when a response confirmation message is output at the time T2 ( Fig. 3 ), as described above. However, for example, the conveyance confirmation utterance generation template table 110e is also referred to in a case where the messenger 33 (child) is detected and a message has been output to the messenger 33 (time T5 ( Fig. 4 )) but there has been no reaction, and the conveyance confirmation utterance is performed for the messenger 33.
- Fig. 11 illustrates a table that is referred to when such a table is referred to and the conveyance confirmation utterance is performed, and whether or not a reaction from the recipient 32 has been obtained is determined.
- Fig. 11 illustrates an example of a response to conveyance confirmation utterance determination table 110f.
- the response to conveyance confirmation utterance determination table 110f is a table that manages a template ID, response expectation, and possibility of continuation in association with one another.
- the template ID the same ID as the template ID of the conveyance confirmation utterance generation template table 110e is assigned.
- the response expectation with the template ID of "M-1-1" in the response to conveyance confirmation utterance determination table 110f is referred to.
- the response expectation in the response to conveyance confirmation utterance determination table 110f is an action or an utterance expected as a response of the recipient 32 as a result of the conveyance confirmation utterance.
- the template with the template ID of "M-1-1” is selected in the conveyance confirmation utterance generation template table 110e and the message for response promotion such as "Mother, please raise your hand in a case where you can immediately respond to the message" is output, the content of "detect an arm has been raised higher than the head in the gesture detection" is referred to as the response expectation of the template ID of "M-1-1" in the response to conveyance confirmation utterance determination table 110f.
- the possibility of continuation in the response to conveyance confirmation utterance determination table 110f is information indicating whether or not to continuously convey a message or the like when the response expected as the response expectation has been obtained. For example, in the above-described case, in the case where "an arm has been raised higher than the head in the gesture detection" has been detected, information of the possibility of continuation is "possible", and thus information desired to be provided to the recipient 32, such as a message, is continuously conveyed to the recipient 32.
- the response expectation is "the voice recognition result is "no", “wait a minute”, “later please”, or “not now”
- the information of the possibility of continuation is “not possible”. Therefore, in a case where the response from the recipient 32 is, for example, an utterance such as "no”, information transmission with a message or the like to the recipient 32 is temporarily stopped. In such a case, transmission of information is resumed after a predetermined time has elapsed or when an utterance such as "What was the previous message?" is performed from the recipient 32.
- a plurality of response expectations is written in the response to conveyance confirmation utterance determination table 110f. Determination such as there having been a response, or information presentation being continued or not continued can be performed by a combination of the response expectations.
- Fig. 12 is a diagram illustrating an example of a message call request generation template table 110g.
- the message call request generation template table 110g is, for example, a table referred to when creating a message for calling the messenger 33 at time T5 ( Fig. 4 ) .
- the message call request generation template table 110g is a table that manages a message call request template. Furthermore, the message call request generation template table 110g manages the message call request templates for infants and for adults.
- templates such as "(name info), please listen”, “(name info), can you come here", and "there is something to ask you to convey to (target user)" are prepared.
- an appropriate template at the point of time is selected from the templates prepared as the message call request templates for infants.
- templates such as "(name info), there is a message for (target user)", and "(name info), are you busy now or not?" are prepared.
- an appropriate template at the point of time is selected from the templates prepared as the message call request templates for adults.
- Fig. 13 is a diagram illustrating an example of word conversion synonym table 110h.
- the word conversion synonym table 110h is provided.
- the word conversion synonym table 110h is referred to when converting a template in the conveyance confirmation utterance generation template table 110e into a template for infants, for example. For example, words such as “respondable” or “response” in the templates in the conveyance confirmation utterance generation template table 110e are converted into "Can you answer?" by reference to the word conversion synonym table 110h.
- the word conversion synonym table 110h information for converting a message into words easily understandable by the person whom the system talks to is written in the word conversion synonym table 110h. For example, in a case where elderly people are not familiar with a name of "AXB” that is a female idol group, and when conveying information regarding "AXB” to an elderly person, the message is converted into a message in which "AXB" is replaced with “female idol group” and is conveyed to the elderly person.
- word conversion synonym table 110h information for converting an uncommon word into a common word is written in the word conversion synonym table 110h. For example, in a case where a name of "gramophone” is not commonly used, and when conveying information regarding "gramophone", the message is converted into a message in which "gramophone” is replaced with “record player” and is conveyed.
- word conversion synonym table 110h information for converting generally unfavorable (inappropriate) words into wording that can be relatively appropriate is also written in the word conversion synonym table 110h. For example, when conveying information of "dead”, the message is converted into a message in which "dead” is replaced with “deceased” and is conveyed.
- information for converting a language into a foreign language may also be written in the word conversion synonym table 110h.
- Fig. 14 is a diagram illustrating an example of a message template table 110i.
- the message template table 110i is a table that is referred to when conveying message content to the messenger 33.
- a template such as "There is a contact from (sender user) to (target user) at (time info) and (sender user) says “(output info)”.
- This template has been selected, for example, a message such as "There is a contact from Father to Mother at 19:30 and he says "I'll come home late today”.
- would you convey this message?” is created and is conveyed to the messenger 33.
- a template such as "Would you notify that please come here because there is an urgent business” to (target user)?" is described.
- this template has been selected, for example, a message such as "Would you notify that please come here because there is an urgent business to Mother?” is created and is conveyed to the messenger 33.
- This template is selected, for example, in a case where the message becomes long and the burden on the messenger 33 becomes heavy when using the message template for normal use, in a case where the transmission content is not suitable for the messenger 33 (such as confidential information), in a case where the concentration of recipients 32 is low and it is better to get the recipient 32 close to the system, or the like.
- a template for confirmation in another device a template such as "Would you notify that an email has been sent because there is an urgent business to (target user)?" is described.
- this template has been selected, for example, a message such as "Would you notify that an email has been sent because there is an urgent business to Mother?” is created and is conveyed to the messenger 33.
- This template is also selected, for example, in the case where the message becomes long and the burden on the messenger 33 becomes heavy when using the message template for normal use, in a case where it is determined that it is better to convey the message by text or image to the recipient 32, or the like.
- a template such as "Would you convey that "(output info)" because of an urgent matter?" is described.
- a message such as "Would you convey "contact school immediately” because of an urgent matter?" is created, for example, and is conveyed to the messenger 33.
- This template is selected in a case where, for example, it is determined that the transmission content needs to be given in notification urgently, for example, in a case where a word such as "immediately” or "important" is included in the message. Furthermore, in a case where the template in case of urgency has been selected, the message may be output with a larger volume than normal time.
- a template for feedback a template such as "Would you directly contact because "(output info)" has not been able to be conveyed?" is described.
- a message such as "Would you directly contact because "I'll be late today” has not been able to be conveyed?” is created, for example, and is conveyed to the messenger 33.
- This template is selected when notifying (feeding back) the sender 31 that the message desired to be conveyed to the recipient 32 has not reached the recipient 32, and further, conveyance of the message has not been able to be requested to the messenger 33.
- the conveyance of the message has not been able to be requested to the messenger 33 includes, for example, when the conveyance of the message itself has not been able to be requested because the messenger 33 has not been found, when the messenger 33 has been found but the messenger 33 has refused the conveyance of the message (there has been no reaction even through a request message has been sent), or in a case where the messenger 33 has not correctly conveyed the message to the recipient 32 (for example, the system requested the messenger 33 to go and get the recipient 32 in front of the system but the recipient 32 has not come in front of the system).
- the message template for asking for coming in front of the system and the template for confirmation in another device in the message template table 110i are selected in the case where the message becomes long and the burden on the messenger 33 becomes heavy when using the message template for normal use.
- criteria illustrated in Fig. 15 may be provided.
- Fig. 15 is a diagram illustrating an example of a character amount threshold value table 110j.
- the character amount threshold value table 110j is a table in which a threshold value of the number of characters of a message is described according to the age of the user identified as the messenger 33.
- the number of characters in this message may be of the entire message or may be the number of characters in a portion corresponding to "(output info)", for example.
- the message template for asking for coming in front of the system is selected.
- the message template for asking for coming in front of the system is selected.
- the message template for asking for coming in front of the system is selected.
- the message template for asking for coming in front of the system is selected.
- the message template for asking for coming in front of the system is selected. Furthermore, in a case of age 59 or younger, in a case where the message includes 15 characters or more, the message template for asking for coming in front of the system is selected. Furthermore, in a case of age 60 or older, in a case where the message includes 10 characters or more, the message template for asking for coming in front of the system is selected.
- the template can made selectable depending on the number of characters in the message.
- the message to be conveyed by the messenger 33 may be compressed (the number of characters may be reduced). For example, processing of compressing the message such as "Would you convey that Mother's favorite singer A will appear on an Internet TV program PPP from 19:30 today?" into a message such as "Would you convey that the singer A will appear on 19:30 today” may be included.
- Such message compression is performed using summarizing techniques or by thinning out vocabulary.
- the character amount threshold value table 110j may be referred to and a template may be selected for the message with a reduced number of characters as a result of such compression, or the character amount threshold value table 110j may be referred to and a template may be selected for the message before compression.
- compression may be performed to make the number of characters equal or smaller than the threshold value.
- the template for normal use is selected.
- the compression processing for the message is executed, and the message is converted into a message with 25 characters or less.
- the template for asking for coming in front of the system may be selected.
- the information processing unit 12 detects a situation as illustrated in Fig. 16 , for example.
- the situation at the time T0 is a situation where a burner 201 is in a range managed by the information processing unit 12 and the mother (in this case, the user who becomes the recipient 32) is cooking at the burner 201.
- the situation at the time T0 is a situation where a the television receiver 202 is in the range managed by the information processing unit 12, and a child (in this case, the user who becomes the messenger 33) is in the vicinity of the television receiver 202.
- the information processing unit 12 creates the detection information database 110b as illustrated in Fig. 7 (adds or updates data).
- the user attribute information database 110a illustrated in Fig. 6 has been created.
- the information processing unit 12 conveys, for example, information obtained from the external service providing unit 21 ( Fig. 1 ) to the user (recipient 32). As illustrated in Fig. 17 , at the time T1, the information processing unit 12 utters the message such as "Mother's favorite singer A will appear on an Internet TV program PPP from 19:30 today" to the recipient 32. This utterance is output by, for example, Text To Speech.
- the mode may be switched at the number of times other than 5 times.
- the processing transitions to a mode of outputting a conveyance confirmation message.
- the information processing unit 12 outputs a message such as "Mother, please raise your hand in a case where you can immediately cope with the system soon".
- the information processing unit 12 refers to the conveyance confirmation utterance generation template table 110e illustrated in Fig. 10 , and selects an appropriate conveyance confirmation utterance template. This case illustrates a case where the template with the template ID of "M-1-1" has been selected.
- the volume and pitch for outputting the message may be adjusted in consideration of properties of the user, such as the user's age and good or bad ear, a situation having steady noise such as noise from a vacuum cleaner or a washing machine, and a situation of surroundings having sudden noise such as a child's voice or a sound of a pot.
- processing of making the volume for outputting the message large in the situation where the recipient 32 has a property of bad ear or a steady noise is detected, or shortening the time to determine whether or not the recipient 32 has been responded as a result of outputting the message may be executed.
- processing of outputting the message at timing when the sudden noise is small in the situation where the sudden noise is detected, or elongating the time to determine whether or not the recipient 32 has responded as a result of outputting the message may be executed.
- the information processing unit 12 While outputting such a message, the information processing unit 12 also detects the reaction (response) of the recipient 32.
- the detection information database 110b ( Fig. 7 ) is updated with the detection result.
- Fig. 19 illustrates an example (a part) of the detection information database 110b updated at the time T2.
- the direction of the head is changed from the south direction (the detection information database 110b ( Fig. 7 ) at the time T1) to the west direction, and thus information of "west” is written in information of the direction of the head, and the volume changed from 0 to 40 at the utterance of the message is written.
- the message uttered by the recipient 32 is detected by the information processing unit 12 and is written in the column of "voice recognition result" in the detection information database 110b.
- the voice recognition result of "YYY, take a bath soon" written in the column of "voice recognition result” in the detection information database 110b is semantically analyzed, so that this message is analyzed to have the meaning of "command”, and the result is written in the column of "semantic analysis result” in the detection information database 110b.
- the information processing unit 12 Since the information processing unit 12 performs such detection to determine whether or not a reaction indicating that the message has been received from the recipient 32 has been obtained, the information processing unit 12 outputs a determination result indicating that there is no reaction (response) from the recipient 32 by voice in the case where the result of "command" has been output as the "semantic analysis result".
- information of "No” is written in the column of "gaze detection” in the detection information database 110b.
- the recipient 32 faces the direction of the child, and the gaze is also in the direction of the child and is not in the direction of the information processing unit 12 (system). Therefore, the information of "No” is written in the column of "gaze detection”.
- the information processing unit 12 Since the information processing unit 12 performs such detection to determine whether or not a reaction indicating that the message has been received from the recipient 32 has been obtained, the information processing unit 12 outputs a determination result indicating that there is no reaction (response) from the recipient 32 by gaze in the case where the result of "No" has been output as the "gaze detection".
- the information processing unit 12 Since the information processing unit 12 performs such detection to determine whether or not the reaction indicating that the message has been received from the recipient 32 has been obtained, the information processing unit 12 determines that there is a reaction by a gesture from the recipient 32 when detecting the gesture of "raised a hand" as the “gesture detection". However, in a case of detecting a gesture other than the gesture of "raised a hand", the information processing unit 12 outputs a determination result indicating that there is no reaction from the recipient 32 by gaze.
- the information processing unit 12 determines whether or not there has been a response from the recipient 32 by determining whether or not the information written in the detection information database 110b satisfies the criteria described in the response to conveyance confirmation utterance determination table 110f.
- the search information database 110b as illustrated in Fig. 19 is created at the time T3. Therefore, it is determined that there has been no response from the recipient 32. Then, the information processing unit 12 transitions to the message mode.
- the information processing unit 12 searches for a user other than the recipient 32 and existing in a range where the information processing unit 12 can output a message. Such a search is performed at the time T3 and the time T4 ( Fig. 3 ). Then, in a case where a user who can become the messenger 33 has been found, processing of requesting the messenger 33 to convey the message is started.
- determination is performed using an attribute of the users as a determination material. For example, a user with the highest age, a user who responded to a message request in the past, a male user, a female user, or a user with a threshold value or more using a predetermined threshold value can be set as the messenger 33.
- the message call request generation template table 110g illustrated in Fig. 12 the message call request templates for infants and for adults are prepared.
- whether the messenger 33 is an infant or an adult is determined.
- the description will be continued using the case where the messenger 33 is an infant as an example.
- the template such as "(name info)", can you come here?" is selected from the message call request generation template table 110g ( Fig. 12 ), and a message such as "YYY, can you come here?" is output.
- the name of the messenger 33 in this case, the name of the child, a name registered in advance such as a nickname enters the portion of "YYY" in the "YYY".
- the information processing unit 12 (system) outputs the message such as "YYY, can you come here?" to the messenger 33.
- the processing at and after the time T5 is basically the similar to the processing performed for the recipient 32 from the time T1 to time T2. That is, first, whether or not the messenger 33 has reacted as a result of outputting such a message is determined.
- the conveyance confirmation message is output to the messenger 33, as in the time T2.
- the conveyance confirmation message may be output after the message such as "YYY, can you come here?" has been output to the messenger 33, for example.
- the conveyance confirmation message may be output regardless of whether or not there has been a response from the messenger 33.
- the message such as "YYY, can you answer to the system soon? Please raise your hand" is output from the information processing unit 12 to the messenger 33 by Text To Speech.
- This message is output by selecting a template by reference to the conveyance confirmation utterance generation template table 110e and converting the wording of the selected template into wording for infants by reference to the word conversion synonym table 110h.
- the messenger 33 has responded by raising a hand or uttering "yes" at time T5-3, as illustrated in Fig. 22 .
- the information processing unit 12 detects the action or the utterance of the messenger 33. In this case, the information processing unit 12 detects that the messenger 33 has noticed the message from the information processing unit 12, and has faced the direction of the information processing unit 12 and has nodded.
- the messenger 33 faces the direction of the information processing unit 12 (system), and thus the gaze is also detected. "YES” is written in the column of "gaze detection” in the detection information database 110b.
- the messenger 33 has performed the action of raising a hand and thus this action is detected as a gesture.
- the "raising a hand” is written in the column of "gesture detection” in the detection information database 110b.
- the messenger 33 has responded to the message from these detection results.
- the request type in the message mode is selected (time T6 ( Fig. 4 )).
- the message template table 110i ( Fig. 14 ) is referred to when the request type is selected. Furthermore, as described with reference to Figs. 14 and 15 , for example, the threshold value of the number of characters in a message is provided according to the age of the messenger 33, and the template from the message template table 110i is selected according to whether or not a message has the number of characters equal or larger than the threshold value.
- the messenger 33 is a child (age of 5 according to the user attribute information database 110a in Fig. 6 ). Therefore, the threshold value of the number of characters in the message is determined to be 10 characters or less according to the character amount threshold value table 110j in Fig. 15 .
- the message desired to be conveyed to the recipient 32 is the message of "Mother's favorite singer A will appear on an Internet TV program PPP from 19:30 today", and in a case of asking the messenger 33 to convey the message to the recipient 32, the character amount is 10 characters or more. Therefore, in this case, an output for asking for coming in front of the system is selected as a result of referring to the character amount threshold value table 110j.
- the template such as "Would you notify that please come here because there is an urgent business" to (target user)?", which is the template for asking for coming in front of the system, is selected from the message template table 110i.
- the word conversion synonym table 110h ( Fig. 13 ) is referred to and the message is converted into a message for infants.
- a message such as "Would you tell your Mother that please come here because there is a little rush" is generated and is output to the messenger 33.
- Such a message is output from the information processing unit 12, and the messenger 33 understands the content of the message and starts an action. For example, as illustrated in Fig. 25 , at time T7-2, the messenger 33 moves toward the recipient 32 (mother).
- the messenger 33 when the messenger 33 approaches the mother, he/she utters (conveys the message) to the mother (recipient 32) that "the system has an urgent business". As a result of the messenger 33 conveying the message from the information processing unit 12 (system) to the recipient 32, the recipient 32 utters "ok” and approaches the system at time T7-3 ( Fig. 26 ).
- the information processing unit 12 detects the approach and the utterance content of the recipient 32, and updates the detection information database 110b on the basis of the detection result.
- the information processing unit 12 In a case where it can be determined that the recipient 32 is focusing on the information processing unit 12 in this way, the information processing unit 12 outputs a message such as "There is something to be conveyed to Mother" to the recipient 32 at time T8-2, as illustrated in Fig. 29 .
- the message of "Mother's favorite singer A will appear on an Internet TV program PPP from 19:30 today” is output at time T8-3 ( Fig. 30 ).
- This message is a message desired to be conveyed to the recipient 32 and is a message having not been conveyed to the recipient 32 at the time T1 ( Fig. 17 ).
- the recipient 32 When the message originally desired to be conveyed is conveyed from the information processing unit 12 to the recipient 32 in this manner, the recipient 32 shows a reaction of uttering words such as "Really. Thank you” in the example illustrated in Fig. 30 .
- the information processing unit 12 detects this reaction, whereby the information in the detection information database 110b is rewritten as illustrated in Fig. 31 .
- Information such as "really, thank you” is written in the column of "voice recognition result” and information of "appreciation” is written in the column of "semantic analysis result” in the detection information database 110b illustrated in Fig. 31 .
- the information processing unit 12 can detect that the message has been successively conveyed.
- the messenger 33 is searched for, and the message desired to be conveyed to the recipient 32 can be finally conveyed to the recipient 32 from the messenger 33.
- a message (information) desired to be conveyed can be more reliably conveyed to a user whom the message is desired to be conveyed to.
- step S11 content (message) desired to be conveyed is output to a target user.
- a target user For example, when an email is received from the sender 31 or when information in accordance with preference of the user is acquired from the external service providing unit 21, the content is output to the target user (recipient 32).
- This processing is processing executed at above-described time T1.
- step S12 whether or not there is a reaction from the target user (recipient 32) is determined. In a case where it is determined that there is a reaction from the target user, the processing proceeds to step S24, the content desired to be conveyed is output, and the processing is terminated.
- step S24 can be omitted because the content desired to be conveyed is output to the target user in step S11. Furthermore, a message for drawing attention from the target user may be output in step S11, and the content desired to be conveyed may be output in step S24. Furthermore, the same message may be output in steps S11 and S24, in other words, the same message may be output twice.
- step S12 determines whether there is no reaction from the target user.
- step S13 a conveyance confirmation utterance is performed. This processing is processing executed at above-described time T2.
- step S14 Whether or not having obtained a reaction from the target user as a result of the conveyance confirmation utterance in step S13 is determined in step S14.
- step S14 whether or not there is a reaction from the target user (recipient 32) is determined. In a case where it is determined that there is a reaction from the target user, the processing proceeds to step S24, the content desired to be conveyed is output, and the processing is terminated.
- step S15 the processing is moved onto the message mode.
- the message mode is a mode for searching for the messenger 33 and asking the messenger 33 to convey a message, and is processing executed at the above-described time T3 to T7.
- step S16 a search for the messenger 33 is performed. This processing is processing executed at above-described time T3. As a result of the search, whether or not the messenger 33 has been found is determined in step S17.
- step S17 in a case where it is determined that the messenger 33 has been found, the processing proceeds to step S18.
- step S18 a conveyance confirmation utterance is performed for the messenger 33.
- step S19 whether or not there is a reaction from the messenger 33 is determined.
- step S19 in a case where it is determined that there is a reaction from the messenger 33, the processing proceeds to step S20.
- step S20 a message for requesting the messenger 33 to convey a message is created and is output to the messenger 33. This processing is processing executed at above-described time T6 and T7.
- step S21 whether or not the message is a message for getting the recipient 32 is determined.
- the message for getting the recipient 32 is a message such as "Would you get "mother”?", for example.
- a message other than the message for getting the recipient 32 is a message including a message desired to be conveyed to the recipient 32 itself, and is, for example, a message such as "Would you notify that an email informing the father's coming home late has arrived".
- the message is determined not to be the message for getting the recipient 32 in step S21, in other words, in a case where the message asked for conveyance to the messenger 33 is determined to be the message including a message desired to be conveyed to the recipient 32, the message originally desired to be conveyed is conveyed from the messenger 33 to the recipient 32, and thus the processing is terminated.
- step S11 processing in step S11 and subsequent steps is repeated.
- the message is conveyed from the messenger 33 to the recipient 32, and as a result, the recipient 32 comes to the information processing unit 12, and whether or not it becomes a message conveyable state is determined.
- step S19 the processing proceeds to step S22.
- step S22 whether or not the sender 31 is a system is determined.
- the information desired to be conveyed to the user may be the information regarding the content of an email from the sender 31 (predetermined user) or may be the information obtained by the system (information processing unit 12) from the external service providing unit 21.
- the sender 31 is a predetermined user, and information created by the user is desired to be conveyed to the recipient 32, and there is a case where the sender 31 is the system, and information created by the system is desired to be conveyed to the recipient 32.
- step S23 feedback is given to the user who is the sender 31. In this case, it is a state where the sender 31 cannot convey the information to the recipient 32 whom the information desired to be conveyed to and the messenger 33 has not been able to be found. Therefore, processing for notifying the sender 31 that the information has not been conveyed is executed.
- the message template table 110i ( Fig. 14 ) is referred to, a feedback template is selected, and feedback is provided to the sender 31.
- the template such as ""(output info)" has not been able to be conveyed, so would you directly make a contact?” is selected, and for example, a message such as "the email informing coming home late has not been able to be conveyed, so would you directly make a contact?" is created.
- the message created in this manner is transmitted to the sender 31, as an email, for example.
- the sender 31 is also treated as the messenger 33, and a request for taking an action to enable the recipient 32 to receive the information is output.
- the feedback is provided to the sender 31, whereby the sender 31 who has transmitted the information becomes able to take an action for enabling the recipient 32 to receive the information.
- information can be more reliably conveyed to a user whom the information is desired to be conveyed to corresponding to various states.
- a message instructing (requesting) the messenger 33 to take the terminal to the recipient 32 may be output to the messenger 33.
- the messenger 33 takes the terminal (system) to the recipient 32 according to the instruction, and the recipient 32 can receive presentation of the information from the brought terminal.
- the email may be output to the portable terminal (smartphone or the like) of the messenger 33.
- a message such as "Take this smartphone to Mother and show this message to her" is output, for example, and the messenger 33 understands such an email and takes an action, whereby the information is presented to the recipient 32.
- the processing transitions to the message mode.
- the processing may be started from the message mode, for the predetermined recipient 32.
- the processing may transition to the message mode without outputting the message a plurality of times, and the message may be conveyed by the messenger 33.
- the processing for the recipient 32 to which the information is less easily conveyed can be reduced, and the time to convey the information can be shortened.
- setting not to transition to the message mode may be able to be made by the user, for example, by the sender 31.
- the sender 31 does not want the messenger 33 to know information depending on the content.
- a mechanism to set the processing not to transition to the message mode may be provided.
- the above-described series of processing can be executed by hardware or software.
- a program that configures the software is installed in a computer.
- the computer includes a computer incorporated in dedicated hardware, and a general-purpose personal computer and the like capable of executing various functions by installing various programs, for example.
- Fig. 33 is a block diagram illustrating a configuration example of hardware of a computer that executes the above-described series of processing by a program.
- a central processing unit (CPU) 1001 a read only memory (ROM) 1002, and a random access memory (RAM) 1003 are mutually connected by a bus 1004.
- an input/output interface 1005 is connected to the bus 1004.
- An input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a drive 1010 are connected to the input/output interface 1005.
- the input unit 1006 includes a keyboard, a mouse, a microphone, and the like.
- the output unit 1007 includes a display, a speaker, and the like.
- the storage unit 1008 includes a hard disk, a nonvolatile memory, and the like.
- the communication unit 1009 includes a network interface, and the like.
- the drive 1010 drives a removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
- the CPU 100 for example, loads a program stored in the storage unit 1008 into the RAM 1003 and executes the program via the input/output interface 1005 and the bus 1004, whereby the above-described series of processing is performed.
- the program to be executed by the computer (CPU 1001) can be recorded on the removable medium 1011 as a package medium and the like, for example, and provided. Furthermore, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcast.
- the removable medium 1011 is attached to the drive 1010, whereby the program can be installed in the storage unit 1008 via the input/output interface 1005. Furthermore, the program can be received by the communication unit 1009 via a wired or wireless transmission medium and installed in the storage unit 1008. Other than the above method, the program can be installed in the ROM 1002 or the storage unit 1008 in advance.
- the program executed by the computer may be a program processed in chronological order according to the order described in the present specification or may be a program executed in parallel or at necessary timing such as when a call is made.
- system refers to an entire apparatus configured by a plurality of apparatuses.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Multimedia (AREA)
- Acoustics & Sound (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- User Interface Of Digital Computer (AREA)
- Information Transfer Between Computers (AREA)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2017067652 | 2017-03-30 | ||
PCT/JP2018/010379 WO2018180571A1 (fr) | 2017-03-30 | 2018-03-16 | Dispositif de traitement d'informations, procédé et programme de traitement d'informations |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3605316A1 true EP3605316A1 (fr) | 2020-02-05 |
EP3605316A4 EP3605316A4 (fr) | 2020-04-01 |
Family
ID=63675688
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18776223.2A Ceased EP3605316A4 (fr) | 2017-03-30 | 2018-03-16 | Dispositif de traitement d'informations, procédé et programme de traitement d'informations |
Country Status (4)
Country | Link |
---|---|
US (1) | US11289079B2 (fr) |
EP (1) | EP3605316A4 (fr) |
JP (1) | JP7095684B2 (fr) |
WO (1) | WO2018180571A1 (fr) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108600911B (zh) * | 2018-03-30 | 2021-05-18 | 联想(北京)有限公司 | 一种输出方法及电子设备 |
JP7131077B2 (ja) * | 2018-05-24 | 2022-09-06 | カシオ計算機株式会社 | 会話装置、ロボット、会話装置制御方法及びプログラム |
JP2020112940A (ja) * | 2019-01-09 | 2020-07-27 | 本田技研工業株式会社 | コンテンツ出力装置及びコンテンツ出力方法 |
Family Cites Families (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH09218770A (ja) | 1996-02-14 | 1997-08-19 | Toshiba Corp | 対話処理装置および対話処理方法 |
JP2000148607A (ja) * | 1998-11-05 | 2000-05-30 | Casio Comput Co Ltd | 電子メール装置、及び電子メールプログラムが記録された記録媒体 |
US7069207B2 (en) * | 2001-01-26 | 2006-06-27 | Microsoft Corporation | Linguistically intelligent text compression |
JP2002251681A (ja) * | 2001-02-21 | 2002-09-06 | Saibuaasu:Kk | 動作検知装置、動作検知システム、異常動作通知システム、ゲームシステム、所定動作の通知方法およびセンタ装置 |
JP2003122992A (ja) | 2001-10-11 | 2003-04-25 | Nippon Telegr & Teleph Corp <Ntt> | 情報とメディアの選択システム、及び情報とメディアの選択方法 |
JP4049244B2 (ja) | 2002-01-10 | 2008-02-20 | 富士フイルム株式会社 | 伝言装置 |
JP2004220314A (ja) * | 2003-01-15 | 2004-08-05 | Recruit Co Ltd | 情報提供支援システム |
US7457879B2 (en) * | 2003-04-01 | 2008-11-25 | Microsoft Corporation | Notification platform architecture |
KR100636985B1 (ko) * | 2005-05-25 | 2006-10-19 | 엘지전자 주식회사 | 이동통신단말기의 sms 메시지 압축방법 및 그이동통신단말기 |
US7720572B2 (en) | 2005-09-30 | 2010-05-18 | Irobot Corporation | Companion robot for personal interaction |
US20100287050A1 (en) * | 2009-05-07 | 2010-11-11 | Chacha Search Inc. | Method and system for personally targeted search messages |
JP5283196B2 (ja) | 2010-09-28 | 2013-09-04 | Necアクセステクニカ株式会社 | 安否確認システム、安否確認方法 |
US20120203843A1 (en) * | 2011-02-03 | 2012-08-09 | International Business Machines Corporation | Contacting an unavailable user through a proxy using instant messaging |
US8698621B2 (en) * | 2011-11-22 | 2014-04-15 | Verizon Patent And Licensing Inc. | Method and system for providing notifications of a mobile device in motion to determine call treatment |
JP5821713B2 (ja) * | 2012-03-09 | 2015-11-24 | 富士通株式会社 | メール管理プログラム、メール管理装置、及びメール管理方法 |
JP6231205B2 (ja) | 2014-06-11 | 2017-11-15 | シャープ株式会社 | 出力メッセージ決定装置、および出力メッセージ決定装置の制御方法 |
US10721594B2 (en) * | 2014-06-26 | 2020-07-21 | Microsoft Technology Licensing, Llc | Location-based audio messaging |
WO2016157658A1 (fr) * | 2015-03-31 | 2016-10-06 | ソニー株式会社 | Dispositif de traitement d'informations, procédé de commande et programme |
-
2018
- 2018-03-16 EP EP18776223.2A patent/EP3605316A4/fr not_active Ceased
- 2018-03-16 JP JP2019509269A patent/JP7095684B2/ja active Active
- 2018-03-16 WO PCT/JP2018/010379 patent/WO2018180571A1/fr unknown
- 2018-03-16 US US16/489,810 patent/US11289079B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
EP3605316A4 (fr) | 2020-04-01 |
WO2018180571A1 (fr) | 2018-10-04 |
US11289079B2 (en) | 2022-03-29 |
JPWO2018180571A1 (ja) | 2020-02-06 |
US20200013402A1 (en) | 2020-01-09 |
JP7095684B2 (ja) | 2022-07-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN110741433B (zh) | 使用多个计算设备的对讲式通信 | |
US11657812B2 (en) | Message playback using a shared device | |
US11094316B2 (en) | Audio analytics for natural language processing | |
JP7341171B2 (ja) | 自動化アシスタントを呼び出すための動的および/またはコンテキスト固有のホットワード | |
US11289079B2 (en) | Information processing apparatus and information processing method | |
US10157614B1 (en) | Message playback using a shared device | |
KR20070026452A (ko) | 음성 인터랙티브 메시징을 위한 방법 및 장치 | |
US11659362B1 (en) | Managing communications sessions based on restrictions and permissions | |
US11172001B1 (en) | Announcement in a communications session | |
JP2023093714A (ja) | コンタクト制御プログラム、端末およびコンタクト制御方法 | |
KR20190024921A (ko) | 개인 일정 및 라이프 스타일을 반영한 인공지능의 컨텍 제어 | |
CN110324485A (zh) | 基于通信接收方偏好切换通信模式的设备、方法和系统 | |
CN110945473A (zh) | 信息处理装置和信息处理方法 | |
JP2019036174A (ja) | 制御装置、入出力装置、制御方法、および制御プログラム | |
US20190385633A1 (en) | Signal processing apparatus and method, and program | |
US11258746B1 (en) | Setting overrides | |
JP2016109784A (ja) | 情報処理装置、情報処理方法、対話システム、および制御プログラム | |
US11170754B2 (en) | Information processor, information processing method, and program | |
Sawhney | Contextual awareness, messaging and communication in nomadic audio environments | |
JP6855528B2 (ja) | 制御装置、入出力装置、制御方法、および制御プログラム | |
JP6776284B2 (ja) | 情報処理システム、情報処理方法、およびプログラム | |
JP6788620B2 (ja) | 情報処理システム、情報処理方法、およびプログラム | |
WO2015043529A1 (fr) | Procédé et système pour répondre à des informations d'application sociale |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20191021 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 20200304 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 15/22 20060101ALI20200227BHEP Ipc: G06F 3/16 20060101AFI20200227BHEP Ipc: G10L 13/02 20130101ALI20200227BHEP Ipc: G06F 3/01 20060101ALI20200227BHEP |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
RAP3 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: SONY GROUP CORPORATION |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20210903 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R003 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED |
|
18R | Application refused |
Effective date: 20221222 |