JP7163103B2 - Information processing device, information processing method and information processing program - Google Patents

Description

TECHNICAL FIELD Embodiments of the present invention relate to an information processing apparatus, an information processing method, and an information processing program.

2. Description of the Related Art In recent years, an information device has been developed that acquires a voice from a user, recognizes the acquired voice, and executes an operation corresponding to the acquired voice.

For example, Patent Literature 1 proposes an electronic device that is activated by a user's behavior without making the user feel uneasy.

JP 2017-4231 A

However, with the above-described conventional technology, it is not always possible to assist the user's actions by outputting information in anticipation of situations that may occur to the user in the future. For example, in the conventional technology described above, the user is notified that the speech recognition unit and the speech acquisition unit have been activated, triggered by the user's behavior with respect to the home appliance.

With the conventional technology described above, it is possible to prevent the user from feeling uneasy about whether or not the voice uttered by the user is correctly acquired and recognized. However, it is not always possible to help the user's actions by outputting information in anticipation of situations that may occur to the user in the future.

The present application has been made in view of the above, and provides an information processing device, an information processing method, and an information processing program that can assist the user's actions by outputting information that anticipates situations that may occur to the user in the future. for the purpose.

An information processing apparatus according to the present application includes an acquisition unit that acquires environmental sounds of a user's surrounding environment, and predetermined information for the user based on the user's context corresponding to the environmental sounds acquired by the acquisition unit. and a determination unit that determines the timing of outputting the .

According to one aspect of the embodiment, it is possible to assist the user's actions by outputting information that anticipates situations that may occur to the user in the future.

FIG. 1 is a diagram illustrating an example of information processing according to an embodiment; FIG. 2 is a diagram illustrating a configuration example of an information processing system according to the embodiment; FIG. 3 is a diagram illustrating a configuration example of an information processing apparatus according to the embodiment; 4 is a diagram illustrating an example of an environmental sound information storage unit according to the embodiment; FIG. 5 is a diagram illustrating an example of a context information storage unit according to the embodiment; FIG. 6 is a diagram illustrating an example of a target information storage unit according to the embodiment; FIG. FIG. 7 is a flowchart illustrating an information processing procedure according to the embodiment; FIG. 8 is a hardware configuration diagram showing an example of a computer that implements the functions of the information processing apparatus.

Hereinafter, modes for implementing an information processing apparatus, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described with reference to the drawings. The information processing apparatus, information processing method, and information processing program according to the present application are not limited to this embodiment. In addition, in the following embodiments, the same parts are denoted by the same reference numerals, and overlapping descriptions are omitted.

[1. information processing]
First, an example of information processing according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of information processing according to an embodiment; Information processing according to the embodiment is performed by the information processing apparatus 100 shown in FIG.

Prior to the description of FIG. 1, the information processing system according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram showing a configuration example of the information processing system 1 according to the embodiment. The information processing system 1 according to the embodiment includes a terminal device 10, an output device 30, and an information processing device 100, as shown in FIG. The terminal device 10, the output device 30, and the information processing device 100 are communicably connected via a network N by wire or wirelessly. The information processing system 1 shown in FIG. 1 may include a plurality of terminal devices 10, a plurality of output devices 30, and a plurality of information processing devices 100. FIG.

The terminal device 10 is an information processing device used by a user. The terminal device 10 is, for example, a smart phone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like.

The output device 30 is, for example, a stationary smart speaker placed indoors or the like. However, the output device 30 does not necessarily have to be a smart speaker as long as it has a function of realizing dialogue with a user, and may be, for example, a car navigation system, a mobile robot, or the like. However, in this embodiment, the output device 30 is assumed to be a smart speaker. Therefore, in the following embodiments, the output device 30 may be referred to as "smart speaker 30".

Here, the premise of information processing according to the embodiment will be described. For example, in daily life, the user may have a series of behaviors patterned such that after performing a certain behavior, the user performs a specific behavior. More specifically, a series of behaviors may be patterned such that a specific behavior is performed immediately after performing a certain behavior in a certain time period. To give an example, assume that a certain user UX wakes up around 7:00, opens the curtains every morning, and then issues a voice command to the smart speaker 30 such as "play the news." In such an example, the user UX has a pattern of a series of behaviors such as "open the curtains and then voice-operate the smart speaker 30 to play news" as a series of behaviors after waking up. I can say.

In addition to the action of opening the curtains, actions performed by the user as one of a series of actions such as the action of turning on the television, the action of starting the coffee maker, the action of boiling water, etc. Even users are diverse depending on the time zone. Nonetheless, in daily life, a pattern is formed for each user such that when this behavior is performed in a certain period of time, a specific behavior is performed immediately after that.

Here, in many cases, environmental sounds accompany the actions of the user. For example, when the curtain is opened, a peculiar (unique) environmental sound such as "shhh, shhh" is generated. The same is true for cases other than this example. When the user turns on the TV, environmental sounds such as "Today's weather is..." are generated as voices of TV moving images (there are also environmental sounds generated by operating the remote control of the TV). In addition, the action of starting the coffee maker produces an environmental sound unique to grinding coffee beans, such as "ga-ga-ga". In addition, the action of boiling water generates an environmental sound unique to operating a gas stove, such as "Gacha".

For this reason, in the above example of the user UX, if the environmental sound "shuh, shhh" is generated at 7 o'clock, for example, after a few seconds (or several minutes later), "the user UX is a smart speaker It is possible to foresee (predict) that the user will perform an action such as "manipulate voice to broadcast news to 30". This is based on a series of patterned behaviors of the user UX, and after the environmental sound "shhhhhhh" at 7 o'clock, the behavior of "the user UX voice-manipulates the smart speaker 30 to play news", that is, It is based on obtaining a trend of the user UX context corresponding to environmental sounds.

In such a case, after the user UX opens the curtain at around 7:00, if the news can be dynamically delivered before the smart speaker 30 receives the voice operation from the user UX, the user UX will not be bothered to perform the voice operation. This is convenient because it eliminates the need to perform

Based on the above premise, the information processing apparatus 100 according to the embodiment analyzes the user's context corresponding to the environmental sound, more specifically, the tendency of the user's context corresponding to the environmental sound, and analyzes the analysis result determines the timing of wakeup (providing information) based on

In summary, the information processing apparatus 100 according to the embodiment acquires environmental sounds of the user's surrounding environment, and provides predetermined information (an example of wake-up) to the user based on the user's context corresponding to the acquired environmental sounds. ) is output. Specifically, the information processing apparatus 100 acquires, as the environmental sound of the user's surrounding environment, the environmental sound that occurred before the situation indicated by the user's context occurred. For example, the information processing apparatus 100 acquires environmental sounds that occurred within a predetermined period of time before the situation indicated by the user's context occurred.

Then, the information processing apparatus 100 learns the relationship between the environmental sounds of the user's surrounding environment and the user's context. Then, the information processing apparatus 100 determines the timing of outputting the predetermined information to the user based on the context having the relationship indicating the temporal tendency with respect to the environmental sound. The relationship that indicates a temporal tendency with respect to the environmental sound is, for example, a tendency to “manipulate the smart speaker 30 by voice to broadcast news” a few seconds after the environmental sound “Shut, shut” that opens the curtain is generated. There is. An example of information processing according to the embodiment will be described below.

Before starting a detailed description, first, an outline of FIG. 1 will be described. The home HP1 shown in FIG. 1 has four users, user U1, user U2, user U3, and user U4 (users U1 to U4). Although the relationship between users U1 to U4 is not limited, it is assumed here that they are family members. In the example of FIG. 1, user U1 is in room R1 (bedroom). User U2 is in room R2 (dining room). User U3 is in room R3 (living room). User U4 is in room R4 (kitchen).

Also, the smart speaker 30 is placed at a predetermined location in the room HP1, and is capable of acquiring the uttered voices of users in each room. In general, one smart speaker is often installed in each room, but in this embodiment, for convenience of explanation, the smart speaker 30 is assumed to be able to acquire and cover the sound of all four rooms. do. In addition, when the smart speaker 30 is placed in each of the four rooms, the information detected by the smart speaker 30 is acquired from one smart speaker 30, or the information detected by the smart speaker 30 is acquired from each of the four smart speakers 30. The only difference is whether information detected by the speaker 30 is acquired, and there is no difference in the information processing performed by the information processing apparatus 100 .

First, the smart speaker 30 detects environmental sounds generated in each room and the user's context (step S1). In the room R1, "at 7:00 on July 1, 2018, the user U1 opens the curtain", and four minutes later, "at 7:04 on July 1, 2018, the user U1 opens the smart speaker 30 to broadcast the news." A voice instruction is being given to "." In this case, the smart speaker 30 detects the ambient sound of opening the curtains at 7:00 on July 1, 2018 as the environmental sound indicating the surrounding environment of the user U1. In addition, the smart speaker 30 detects, as the context of the user U1, the context "at 7:04 on July 1, 2018, the user U1 performs an action such as issuing a voice instruction to broadcast news".

The smart speaker 30 then transmits the detected environmental sound and context data (environmental sound data and context data) to the information processing device 100 (step S2). In the example of FIG. 1, the smart speaker 30 transmits to the information processing apparatus 100 environmental sound data DA11-1 detected in the room R1 and context data DA11-2 detected in the room R1. In room R1, of course, environmental sounds and contexts other than the environmental sounds and contexts described above can occur. Therefore, the smart speaker 30 transmits these data to the information processing device 100 each time it detects environmental sounds and context in the room R1.

In room R2, "user U2 started the coffee maker at 7:30 on July 1, 2018", and three minutes later, "user U2 started radio shopping at 7:33 on July 1, 2018". An example is shown in which a voice instruction is given to the smart speaker 30 to play. In such a case, the smart speaker 30 detects the environmental sound of the coffee maker operating at 7:30 on July 1, 2018 as the environmental sound indicating the surrounding environment of the user U2. In addition, the smart speaker 30 detects, as the context of the user U2, the context of "at 7:33 on July 1, 2018, the user U2 performs an action such as giving a voice instruction to play radio shopping".

Again, the smart speaker 30 transmits the detected environmental sound and context data (environmental sound data and context data) to the information processing device 100 (step S2). In the example of FIG. 1, the smart speaker 30 transmits to the information processing apparatus 100 environmental sound data DA21-1 detected in the room R2 and context data DA21-2 detected in the room R2. In room R2, of course, environmental sounds and contexts other than the environmental sounds and contexts described above can occur. Therefore, the smart speaker 30 transmits these data to the information processing device 100 every time it detects environmental sounds and context in the room R2.

Further, in room R3, "at 18:00 on July 1, 2018, user U3 turned on the weather forecast on the television", four minutes later, "at 18:04 on July 1, 2018, user U3 turned on the K zone." , the smart speaker 30 is instructed by voice to deliver traffic information of ". In such a case, the smart speaker 30 selects the ambient sound indicating the surrounding environment of the user U3 as "environmental sound of the TV broadcasting the weather forecast program at 18:00 on July 1, 2018" "tomorrow's weather will be... ” is detected. In addition, the smart speaker 30 detects the context of the user U3 as "At 18:04 on July 1, 2018, the user U3 takes an action such as issuing a voice instruction to broadcast traffic information for Section K". .

Again, the smart speaker 30 transmits the detected environmental sound and context data (environmental sound data and context data) to the information processing device 100 (step S2). In the example of FIG. 1, the smart speaker 30 transmits to the information processing apparatus 100 environmental sound data DA31-1 detected in the room R3 and context data DA31-2 detected in the room R2. In room R3, of course, environmental sounds and contexts other than the environmental sounds and contexts described above can occur. Therefore, the smart speaker 30 transmits these data to the information processing device 100 every time it detects environmental sounds and context in the room R3.

In room R4, "at 8:00 on July 1, 2018, the user U4 operates the gas stove", and five minutes later, "at 8:05 on July 1, 2018, the user U4 plays a cooking program." An example is given to the smart speaker 30 by voice. In this case, the smart speaker 30 detects the environmental sound of the gas stove being operated at 8:00 on July 1, 2018 as the environmental sound indicating the surrounding environment of the user U4. In addition, the smart speaker 30 detects, as the context of the user U4, the context "At 8:05 on July 1, 2018, the user U4 performs an action such as giving a voice instruction to play a cooking program".

Again, the smart speaker 30 transmits the detected environmental sound and context data (environmental sound data and context data) to the information processing device 100 (step S2). In the example of FIG. 1, smart speaker 30 transmits environmental sound data DA41-1 detected in room R4 and context data DA41-2 detected in room R4 to information processing apparatus 100. In FIG. In room R4, of course, environmental sounds and contexts other than the environmental sounds and contexts described above can occur. Therefore, smart speaker 30 transmits these data to information processing device 100 each time it detects environmental sounds and context in room R4.

The information processing device 100 stores the environmental sound data acquired from the smart speaker 30 in the environmental sound information storage unit 121 . In the example of FIG. 1, the environmental sound information storage unit 121 has items such as "user ID", "date and time", and "environmental sound data". “User ID” indicates identification information for identifying the user or the user's terminal device 10 . “Date and time” indicates the date and time when the environmental sound was detected by the smart speaker 30 . The "date and time" may include "day of the week", and in such a case, the "date and time" can be interpreted as calendar information. “Environmental sound data” indicates environmental sound data detected by the smart speaker 30 .

In the example of the room R1 shown in FIG. 1, the smart speaker 30 detects the environmental sound of opening the curtains at 7:00 on July 1, 2018 for the user U1. The smart speaker 30 then transmits environmental sound data DA11-1 indicating the detection result to the information processing apparatus 100. FIG. Therefore, the information processing apparatus 100 associates the user ID “U1”, the date and time “July 1, 2018, 7:00”, and the environmental sound data DA11-1 indicating the environmental sound “shuh, shuh”. Stored in the environmental sound information storage unit 121 . Since the rooms R2, R3, and R4 are the same as those described above, they are omitted.

The information processing device 100 also stores context data acquired from the smart speaker 30 in the context information storage unit 122 . In the example of FIG. 1, the context information storage unit 122 has items such as "user ID", "date and time", and "context data". “User ID” indicates identification information for identifying the user or the user's terminal device 10 . “Date and time” indicates the date and time when the user's context was detected by the smart speaker 30 . The "date and time" may include "day of the week", and in such a case, the "date and time" can be interpreted as calendar information. “Context data” refers to contextual data sensed by the smart speaker 30 .

In the example of the room R1 shown in FIG. 1, the smart speaker 30 detects the context of the user U1, "At 7:04 on July 1, 2018, the user U1 performs an action such as issuing a voice instruction to broadcast news". is doing. The smart speaker 30 then transmits context data DA11-2 indicating data of the detection result to the information processing apparatus 100. FIG. Therefore, the information processing apparatus 100 associates the user ID "U1", the date and time "July 1, 2018, 7:04", and the context data DA11-2 indicating the context "voice instruction to broadcast news". and stores it in the context information storage unit 122 . Since the rooms R2, R3, and R4 are the same as those described above, they are omitted.

Next, the information processing apparatus 100 acquires the environmental sound that occurred before the situation indicated by the context occurred (step S3). For example, the information processing apparatus 100 acquires environmental sounds generated before the situation indicated by the context occurs for each user. For example, the information processing apparatus 100 acquires environmental sounds that occurred within a predetermined period of time before the situation indicated by the context occurred. For example, the information processing apparatus 100 compares the environmental sound information storage unit 121 and the context information storage unit 122, and compares the environmental sound (environmental sound) generated within a predetermined time before the situation indicated by the context for each user. sound data). For example, the information processing device 100 acquires environmental sound data from the environmental sound information storage unit 121 .

In the example of FIG. 1 , the information processing apparatus 100 generates an environment sound environment for each “user” for each “time period” of each “date” that occurred within a predetermined period of time before the situation indicated by the context occurred. Get sound data. In this embodiment, the information processing apparatus 100 acquires environmental sound data of environmental sounds that occurred within five minutes before the situation indicated by the context occurred. This time condition does not necessarily have to be within 5 minutes.

The situation indicated by the context is the user's action situation indicated by the context. Therefore, for example, the situation indicated by the context data DA11-2 can be rephrased as a situation in which the user U1 performs an action of "at 7:04 on July 1, 2018, the user U1 issues a voice instruction to broadcast news." can be done. In addition, for example, the situation indicated by the context data DA21-2 can be rephrased as a situation in which the user U2 performs a voice instruction to play radio shopping at 7:33 on July 1, 2018. be able to. Further, for example, regarding the situation indicated by the context data DA31-2, there was a situation in which the user U3 took the action of "at 18:04 on July 1, 2018, the user U3 issues a voice instruction to broadcast the traffic information of Section K." , can be rephrased. In addition, for example, the situation indicated by the context data DA41-2 can be rephrased as a situation in which the user U4 performs a voice instruction to play a cooking program at 8:05 on July 1, 2018. be able to.

In the example of room R1, when comparing the environmental sound information storage unit 121 and the context information storage unit 122, the situation indicated by the context data DA11-2 for the user U1 at "7:00 on July 1, 2018" It can be seen that the environmental sound (curtain sound) indicated by the environmental sound data DA11-1 is generated within five minutes before the occurrence of . In addition, it can be said that the context (user behavior) indicated by the context data DA11-2 corresponds to the environmental sound indicated by the environmental sound data DA11-1. Therefore, in this example, the information processing apparatus 100 acquires the environmental sound data DA11-1 from the environmental sound information storage unit 121 in time with "7:00 on July 1, 2018". Further, the information processing apparatus 100 acquires the context data DA11-2 from the context information storage unit 122 at the time of "7:00 on July 1, 2018". The information processing apparatus 100 then stores the acquired data in the target information storage unit 123 . Similarly, the information processing apparatus 100 acquires data for each time zone for another date and stores the data in the target information storage unit 123 .

In addition, in the example of room R2, for user U2, at “7:00 on July 1, 2018”, environmental sound data DA21-1 is generated within three minutes before the situation indicated by context data DA21-2 occurs. It can be seen that the environmental sound indicated by (the sound of grinding coffee beans) is being generated. Also, from this, it can be said that the context indicated by the context data DA21-2 corresponds to the environmental sound indicated by the environmental sound data DA21-1. Therefore, in this example, the information processing apparatus 100 acquires the environmental sound data DA21-1 from the environmental sound information storage unit 121 in time with "7:00 on July 1, 2018". Further, the information processing apparatus 100 acquires the context data DA21-2 from the context information storage unit 122 at the time of "7:00 on July 1, 2018". The information processing apparatus 100 then stores the acquired data in the target information storage unit 123 .

In addition, in the example of room R3, for user U3, at “18:00 on July 1, 2018”, environmental sound data DA31-1 is generated within four minutes before the situation indicated by context data DA31-2 occurs. It can be seen that the environmental sound (TV sound) indicated by is occurring. In addition, it can be said that the context indicated by the context data DA31-2 corresponds to the environmental sound indicated by the environmental sound data DA31-1. Therefore, in this example, the information processing apparatus 100 acquires the environmental sound data DA31-1 from the environmental sound information storage unit 121 at "18:00 on July 1, 2018". Further, the information processing apparatus 100 acquires the context data DA31-2 from the context information storage unit 122 in time with "18:00 on July 1, 2018". The information processing apparatus 100 then stores the acquired data in the target information storage unit 123 .

In addition, in the example of room R4, for user U4, at “8:00 on July 1, 2018”, environmental sound data DA41-1 is generated within five minutes before the situation indicated by context data D41-2 occurs. It can be seen that the environmental sound (gas stove sound) indicated by is occurring. Also, from this, it can be said that the context indicated by the context data DA41-2 corresponds to the environmental sound indicated by the environmental sound data DA41-1. Therefore, in this example, the information processing apparatus 100 acquires the environmental sound data DA41-2 from the environmental sound information storage unit 121 in time with "July 1, 2018 between 8:00". Further, the information processing apparatus 100 acquires the context data DA41-2 from the context information storage unit 122 at the time of "July 1, 2018 between 8:00". The information processing apparatus 100 then stores the acquired data in the target information storage unit 123 .

The target information storage unit 123 stores information used for analysis processing, which will be described later. Here, in the example of FIG. 1, the target information storage unit 123 has items such as "user ID", "date", "time zone", "context data", and "environmental sound data". “User ID” indicates identification information for identifying the user or the user's terminal device 10 . "Date" indicates the date when the corresponding "context data" and "environmental sound data" were detected. "Time period" indicates the time period in which the corresponding "context data" and "environmental sound data" were detected. That is, the information processing apparatus 100 associates the data acquired from the context information storage unit 122 and the environmental sound information storage unit 121 with the "user ID", the "date", and the "time period", and stores the data in the target information storage unit 123. do.

Next, the information processing apparatus 100 analyzes (learns) the relationship between the environmental sound and the context using the environmental sound (environmental sound data) and the context (context data) stored in the target information storage unit 123 . ) is performed (step S4). The information processing apparatus 100 can perform, for example, correlation analysis (unsupervised learning) as such analysis processing. For example, the information processing apparatus 100 performs correlation analysis for each user using environmental sound data and context data corresponding to the user.

Taking the user U1 as an example, the information processing apparatus 100 converts, for example, the context data corresponding to the item “time zone” among the items stored in the target information storage unit 123 and the environmental sound data into variables (features). information), perform a correlation analysis. As a result, the information processing apparatus 100 can capture temporal trends such as what kind of context the user U1 tends to follow after what kind of environmental sound (within 5 minutes) in a certain time period. can be done. Note that the feature information here is feature information extracted from the environmental sound, and is feature information indicating the feature of the user.

In the example of FIG. 1, the information processing apparatus 100 obtains an analysis result indicating that there is the following correlation between the environmental sounds and the contexts for the users U1 to U4. Specifically, the information processing apparatus 100 outputs information such that "when an environmental sound indicating a curtain sound is generated around 7:00, the user U1 will issue a voice prompting the smart speaker 30 to play news within five minutes after the curtain sound." Suppose we obtain a correlation between the environmental sound and the context, such as "there is a tendency to indicate". In addition, the information processing apparatus 100 outputs "If an environmental sound indicating the sound of a coffee maker is generated at around 7:00, the user U2 will issue a voice requesting radio shopping to the smart speaker 30 within 5 minutes after the sound of the coffee maker." Suppose we obtain a correlation between the environmental sound and the context, such as "there is a tendency to indicate".

Further, the information processing apparatus 100 instructs the user U3 to play the traffic information of Section K to the smart speaker 30 within 5 minutes after the television sound when an environmental sound indicating the television sound is generated around 18:00. Suppose we obtain a correlation between the environmental sound and the context, such as "there is a tendency to give voice instructions". Further, the information processing apparatus 100 states that "when an environmental sound indicating the sound of a gas stove is generated around 8:00, the user U4 issues a voice instruction to the smart speaker 30 to play a cooking program within five minutes after the sound of the gas stove. Suppose we obtain a correlation between the environmental sound and the context, such as "there is a tendency".

Then, the information processing apparatus 100 determines the timing of outputting predetermined information to the user based on the analysis result in step S4 (step S5). For example, the information processing device 100 determines the timing to output predetermined information to the user's smart speaker 30 . For example, the information processing apparatus 100 determines the timing of outputting predetermined information to the user based on the context of the user's context that has a relationship indicating a temporal tendency to the environmental sound. Temporal trends are the correlations described above.

In the example of FIG. 1, the information processing apparatus 100 determines the output timing based on the time interval between the occurrence time of the context correlated with the environmental sound and the occurrence time of this environmental sound. In the example of room R1, the information processing apparatus 100 obtains an analysis result that there is a correlation between the curtain sound at around 7:00 and the context of user U1 at around 7:00 (audio instruction to broadcast news). , within this correlation, the context occurs within 5 minutes from the occurrence of the curtain sound. That is, in this example, the time interval between the time indicated by the context and the time when the environmental sound was generated is "5 minutes or less".

Therefore, when the curtain sound is detected around 7:00 in the room R1, the information processing apparatus 100 transmits the time within 5 minutes from the time when the curtain sound is detected from the smart speaker 30 to the user U1. This is determined as the timing for outputting news. For example, assume that the smart speaker 30 detects the curtain sound at "July 2, 2018, 7:05:00". In this case, the information processing apparatus 100 determines “July 2, 2018, 7:05:10” as the timing for outputting the news from the smart speaker 30 to the user U1.

Next, in the example of room R2, the information processing device 100 analyzes that there is a correlation between the sound of the coffee maker between 7:00 and the context of the user U2 between 7:00 (voice instruction for radio shopping). We have obtained results, and in this correlation context occurs within 3 minutes of the coffee maker sound. That is, in this example, the time interval between the time indicated by the context and the time when the environmental sound was generated is "3 minutes or less".

Therefore, when the sound of the coffee maker is detected around 7:00 in the room R2, the information processing apparatus 100 transmits the time within 3 minutes from the time when the sound of the coffee maker is detected to the user U2 through the smart speaker. 30 is determined as the timing for outputting radio shopping. For example, assume that the sound of a coffee maker is detected by the smart speaker 30 at “July 2, 2018, 7:06:00”. In this case, the information processing apparatus 100 determines “July 2, 2018, 7:06:10” as the timing for outputting news from the smart speaker 30 to the user U2.

Next, in the example of room R3, the information processing device 100 determines that there is a correlation between the TV sound between 18:00 and the context of the user U3 between 18:00 (voice instruction to play the traffic information of Section K). , and in this correlation, the context occurs within 4 minutes from the TV sound generation. That is, in this example, the time interval between the time indicated by the context and the time when the environmental sound is generated is "four minutes or less".

Therefore, when the TV sound is detected in the room R3 around 18:00, the information processing device 100 transmits the time within 4 minutes from the time when the TV sound is detected from the smart speaker 30 to the user U3. This is determined as the timing for outputting traffic information. For example, assume that the smart speaker 30 detects TV sound at “18:07:00 on July 2, 2018”. In this case, the information processing apparatus 100 determines “July 2, 2018, 18:07:10” as the timing for outputting the news from the smart speaker 30 to the user U3.

Next, in the example of room R4, the information processing apparatus 100 analyzes that there is a correlation between the sound of the gas stove at around 8:00 and the context of user U4 at around 8:00 (voice instruction to play a cooking program). In this correlation, the context occurs within 5 minutes after the sound of the gas stove. That is, in this example, the time interval between the time indicated by the context and the time when the environmental sound was generated is "5 minutes or less".

Therefore, when the TV sound is detected around 8:00 in the room R4, the information processing device 100 transmits the time within 5 minutes from the time when the TV sound is detected from the smart speaker 30 to the user U4. This is determined as the timing for outputting the cooking program. For example, assume that the smart speaker 30 detects the sound of a gas stove at “8:08:00 on July 2, 2018”. In this case, the information processing apparatus 100 determines “July 2, 2018, 8:08:10” as the timing for outputting news from the smart speaker 30 to the user U4.

Now, as explained so far, the information processing apparatus 100 according to the embodiment acquires the environmental sounds of the user's surrounding environment, and determines the relationship (correlation) with respect to the acquired environmental sounds based on the temporal tendency. The timing for outputting information to the user is determined based on a certain context. Thereby, for example, when a specific environmental sound is generated in a specific time period, the information processing apparatus 100 predicts that the behavior (context) of the user after that (for example, several seconds or several minutes later) will be as follows. Therefore, it is possible to perform preemptive actions so that the user does not have to perform the action before the user performs the action. Therefore, the information processing apparatus 100 can help the user's actions.

In the example of FIG. 1, the information processing apparatus 100 acquires environmental sound data of environmental sounds that occurred within five minutes before the situation indicated by the context occurred. However, the information processing apparatus 100 may acquire the environmental sound that occurred immediately before the situation indicated by the context occurred (for example, within 10 seconds). This is because, for example, in the case of user U1, when there is a pattern of behavior, it is often instructed to broadcast the news immediately (for example, within 10 seconds) in conjunction with the behavior of opening the curtain. This is because it is conceivable. In such a case, if the tendency can be acquired as described above, the information processing apparatus 100 can provide information on the time immediately after the detection (for example, 3 seconds) when the curtain sound is detected on a certain day. Decide on timing.

[2. Configuration of Information Processing Device]
Next, the information processing apparatus 100 according to the embodiment will be described using FIG. FIG. 3 is a diagram illustrating a configuration example of the information processing apparatus 100 according to the embodiment. As shown in FIG. 3, the information processing apparatus 100 has a communication section 110, a storage section 120, and a control section . For example, the information processing device 100 is a server device that performs the information processing described with reference to FIG.

(Regarding communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. The communication unit 110 is connected to the network N by wire or wirelessly, and transmits/receives information to/from the output device 30, for example.

(Regarding storage unit 120)
The storage unit 120 is realized by, for example, a RAM (Random Access Memory), a semiconductor memory device such as a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 has an environmental sound information storage unit 121 , a context information storage unit 122 and a target information storage unit 123 .

(Regarding the environmental sound information storage unit 121)
The environmental sound information storage unit 121 stores information about environmental sounds detected by the smart speaker 30 . Here, FIG. 4 shows an example of the environmental sound information storage unit 121 according to the embodiment. In the example of FIG. 4, the environmental sound information storage unit 121 has items such as "user ID", "date and time", and "environmental sound data". Since the environmental sound information storage unit 121 has already been described in FIG. 1, detailed description thereof will be omitted here. In addition to the items shown in FIG. 4 , the environmental sound information storage unit 121 stores the “day of the week” when the environmental sound was detected by the smart speaker 30, the “weather” when the environmental sound was detected by the smart speaker 30, the smart The user's “physical condition” when the environmental sound is detected by the speaker 30 may be included.

For example, the information processing device 100 can acquire “weather” information from a predetermined external server device when the smart speaker 30 detects the environmental sound. Further, for example, when the user wears a wearable terminal equipped with various sensors, the information processing apparatus 100 receives the user's "physical condition" information from the wearable terminal when the environmental sound is detected by the smart speaker 30. can be obtained. In addition, when the user speaks about his or her physical condition before and after the smart speaker 30 detects the environmental sound, the information processing apparatus 100 acquires the physical condition indicated by the speech information as the current physical condition of the user. You may

(Regarding the context information storage unit 122)
The context information storage unit 122 stores information about the user's context sensed by the smart speaker 30 . Here, FIG. 5 shows an example of the context information storage unit 122 according to the embodiment. In the example of FIG. 5, the context information storage unit 122 has items such as "user ID", "date and time", and "environmental sound data". Since the context information storage unit 122 has already been described in FIG. 1, detailed description thereof will be omitted here. In addition to the items shown in FIG. 5, the context information storage unit 122 stores the “day of the week” when the environmental sound was detected by the smart speaker 30, the “weather” when the environmental sound was detected by the smart speaker 30, the smart speaker The "physical condition" of the user when the environmental sound was detected by 30 may be included.

(Regarding the target information storage unit 123)
The target information storage unit 123 stores information used for machine learning (correlation analysis). For example, the target information storage unit 123 associates and stores the environmental sound data acquired from the environmental sound information storage unit 121 and the context data acquired from the context information storage unit 122 . Also, the environmental sound data and the context data that are associated here have relevance in a series of actions of the user, for example, "Immediately after opening the curtain, give a voice instruction."

Here, FIG. 6 shows an example of the target information storage unit 123 according to the embodiment. In the example of FIG. 6, the target information storage unit 123 has items such as "user ID", "date", "time zone", "context data", and "environmental sound data". Since the target information storage unit 123 has already been described in FIG. 1, detailed description thereof will be omitted here. Note that the target information storage unit 123 stores, in addition to the items shown in FIG. The "physical condition" of the user when the environmental sound was detected by 30 may be included.

In FIG. 1 , the information processing apparatus 100 analyzes (learns) the relationship between the environmental sound and the context for each “time period” for each user. showed an example that can capture trends in "time zones", such as what kind of context users tend to have.

However, since the target information storage unit 123 has the item "day of the week", the information processing apparatus 100 can grasp the tendency in the "time zone" for each "weather". In addition, the information processing apparatus 100 can grasp the trend in the "time period" for each "weather" because the target information storage unit 123 has the item "weather". In addition, the information processing apparatus 100 can grasp the tendency in the "time period" for each "physical condition" because the target information storage unit 123 has the item "physical condition".

Returning to FIG. 3, the control unit 130 executes various programs stored in a storage device inside the determination device 100 using a RAM as a work area by a CPU (Central Processing Unit), MPU (Micro Processing Unit), or the like. It is realized by Also, the control unit 130 is implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 3, the control unit 130 has a receiving unit 131, an acquiring unit 132, an analyzing unit 133, and a determining unit 134, and implements or executes information processing functions and actions described below. . Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 3, and may be another configuration as long as it performs information processing described later. Moreover, the connection relationship between the processing units of the control unit 130 is not limited to the connection relationship shown in FIG. 3, and may be another connection relationship.

(Regarding the receiving unit 131)
The receiving unit 131 receives information transmitted from the smart speaker 30 . For example, the receiving unit 131 receives (acquires) environmental sound data of environmental sounds detected by the smart speaker 30 and context data of contexts detected by the smart speaker 30 from the smart speaker 30 .

(Regarding the acquisition unit 132)
The acquisition unit 132 acquires environmental sounds of the user's surrounding environment. For example, the acquisition unit 132 acquires environmental sounds of the user's surrounding environment for each user. For example, the acquisition unit 132 acquires the environmental sounds in the living space of the user as the environmental sounds of the user's surrounding environment. In addition, the acquisition unit 132 acquires, as the environmental sound of the user's surrounding environment, the environmental sound that occurred before the situation indicated by the user's context occurred. In addition, the acquisition unit 132 acquires environmental sounds that occurred within a predetermined period of time before the situation indicated by the user's context occurred, as environmental sounds that occurred before the situation indicated by the user's context. In addition, the acquisition unit 132 acquires environmental sound data (environmental sound data) detected by the smart speaker 30 as the environmental sound.

In the example of FIG. 1, for the user U1, at “7:00 on July 1, 2018”, within 5 minutes before the situation indicated by the context data DA11-2 occurs, the environmental sound data DA11-1 indicates Environmental sound (curtain sound) is being generated. Therefore, in this example, the acquisition unit 132 acquires the environmental sound data DA11-1 from the environmental sound information storage unit 121 at the time of "7:00 on July 1, 2018". Further, the acquisition unit 132 acquires the context data DA11-2 from the context information storage unit 122 at the time of “7:00 on July 1, 2018”.

The information processing apparatus 100 then stores the acquired data in the target information storage unit 123 . For users U2 to U4 other than user U1, the acquisition unit 132 similarly acquires the environmental sound data, but the description is omitted since it is as described with reference to FIG.

(Regarding the analysis unit 133)
The analysis unit 132 analyzes (learns) the relationship between the environmental sounds acquired by the acquisition unit 132 and the user's context. For example, the analysis unit 132 analyzes the relationship between the feature information, which is feature information extracted from the environmental sound and indicates the features of the user, and the context of the user. Also, the analysis unit 133 can be rephrased as the learning unit 133 . For example, the analysis unit 132 analyzes the relationship between the environmental sound and the context using each item stored in the target information storage unit 123 as user feature information.

For example, the analysis unit 133 can perform correlation analysis (unsupervised learning). For example, the analysis unit 133 performs correlation analysis for each user using environmental sound data and context data corresponding to the user. Taking the user U1 as an example, the analysis unit 133 converts the context data corresponding to the item “time zone” among the items stored in the target information storage unit 123 and the environmental sound data into variables (feature data). information), perform a correlation analysis. As a result, the information processing apparatus 100 can capture temporal trends such as what kind of context the user U1 tends to follow after what kind of environmental sound (within 5 minutes) in a certain time period. can be done.

Note that the analyzing unit 132 stores the context data corresponding to the “day of the week”, the “weather”, and the “physical condition” and the environmental sound data as variables (feature information) in addition to the item “time period” of the target information storage unit 123 . ), a correlation analysis may be performed.

(About decision unit 134)
The determination unit 134 determines the timing of outputting predetermined information to the user based on the user's context corresponding to the environmental sound acquired by the acquisition unit 132 . For example, the determination unit 134 determines the timing of outputting predetermined information based on the user's context, which is a context indicating an operation on a predetermined output device (for example, the smart speaker 30). Specifically, the determination unit 134 determines the timing of outputting the predetermined information based on the context indicating the voice operation on the predetermined output device as the operation on the predetermined output device.

Also, for example, the determining unit 134 determines the timing of outputting the information indicated by the context, which is requested by the user in the context, as the predetermined information. In the example of FIG. 1, the user U1 requests the smart speaker 30 to output news. Such requests correspond to information the user is requesting in context.

Further, the determining unit 134 determines the timing of outputting predetermined information to the user based on the occurrence time of the environmental sound acquired by the acquiring unit 132 and the time indicated by the context corresponding to the environmental sound. decide. Specifically, the determination unit 134 determines the time between the occurrence time indicated by the context having a predetermined relationship (temporal tendency relationship) with the environmental sound based on the analysis result of the analysis unit 133 and the occurrence time of the environmental sound. , the timing for outputting predetermined information to the user is determined.

As an example, if an environmental sound having a predetermined relationship with the context is detected in the time period corresponding to the time of occurrence, the determination unit 134 determines whether the environmental sound is detected within a time interval from the time when the environmental sound was detected. is determined as the timing for outputting the predetermined information.

In the example of FIG. 1, the analysis unit 133 determines that there is a correlation (temporal tendency relationship) between the curtain sound at around 7:00 and the context of the user U1 at around 7:00 (audio instruction to broadcast news). I have the results of the analysis. Therefore, when the curtain sound is detected around 7:00 in the room R1, the determination unit 134 notifies the user U1 of the time within 5 minutes from the time when the curtain sound is detected from the smart speaker 30. is determined as the output timing. For example, assume that the smart speaker 30 detects the curtain sound at "July 2, 2018, 7:05:00". In this case, the determination unit 134 determines “July 2, 2018, 7:05:10” (10 seconds later) as the timing for outputting news from the smart speaker 30 to the user U1.

[3. Processing procedure]
Next, the procedure of information processing according to the embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating an information processing procedure according to the embodiment;

First, the receiving unit 131 determines whether or not environmental sound data indicating environmental sounds in the surrounding environment of the user using the smart speaker 30 and context data indicating the user's context have been received from the smart speaker 30 . (Step S101). If the receiving unit 131 determines that data has not been received (step S101; No), it waits until data is received.

On the other hand, when the receiving unit 131 determines that the data has been received (step S101; Yes), the acquiring unit 132 acquires the environment that occurred before the time when the user's context occurred (for example, 30 seconds ago). Acquire environmental sound data representing sound (step S102). The receiving unit 131 stores the received environmental sound data in the environmental sound information storage unit 121 each time it receives the environmental sound data. Also, the receiving unit 131 stores the received context data in the context information storage unit 122 every time it receives context data. Therefore, the acquisition unit 132 compares the data stored in these storage units to obtain the environmental sound data representing the environmental sound that occurred before (for example, 30 seconds) before the time when the user's context occurred. Acquired from the information storage unit 121 .

The acquisition unit 132 also acquires context data corresponding to the acquired environmental sound data at the same time. For example, the acquisition unit 132 acquires such context data from the context information storage unit 122 . Then, the acquisition unit 132 stores the combination of the acquired environmental sound data and the context data in the target information storage unit 123 in association with the time zone.

Next, the analysis unit 133 analyzes (learns) the relationship between the environmental sound and the context using the environmental sound acquired by the acquisition unit 132 and the context of the user (step S103). For example, the analysis unit 133 analyzes the relationship between the environmental sound and the context for each user using the environmental sound data and the context data corresponding to the user. For example, the analysis unit 133 uses the context data corresponding to the item “time zone” among the items stored in the target information storage unit 123 and the environmental sound data as variables (feature information) for each time. Correlation analysis is performed on the bands.

Next, the determination unit 134 determines the output timing for outputting the information requested by the user in the context to the smart speaker 30 based on the analysis result by the analysis unit 133 (step S104). For example, the determination unit 134 determines the time between the occurrence time indicated by the context having a predetermined relationship (temporal tendency relationship) with the environmental sound based on the analysis result of the analysis unit 133 and the occurrence time of the environmental sound. Based on the interval, determine the output timing. For example, when this environmental sound is detected on a certain day, the determination unit 134 determines a time earlier than the time at which it is thought that the behavior (context) that is considered to be a pattern of the user after that is shifted to this context. is determined as the output timing of the information requested in .

Further, the determination unit 134 controls the output of the smart speaker 30 so that the smart speaker 30 outputs information at the determined timing. The smart speaker 30 outputs information according to output control by the determination unit 134 .

[4. Modification]
The information processing apparatus 100 according to the above embodiment may be implemented in various different forms other than the above embodiment. Therefore, other embodiments of the information processing apparatus 100 will be described below.

[4-1. Regarding relationship analysis (1)]
In the above embodiment, the analysis unit 133 performs correlation analysis as an example of analyzing the relationship between the environmental sound acquired by the acquisition unit 132 and the user's context. However, the analysis unit 133 may generate a model indicating the tendency of the context with respect to the environmental sound by learning the relationship between the environmental sound acquired by the acquisition unit 132 and the user's context. More specifically, the analysis unit 133 learns the relationship between the environmental sound acquired by the acquisition unit 132 and the context of the user, thereby inputting the detected environmental sound and the occurrence time of the environmental sound. Then, a model may be generated that determines (outputs) the timing of outputting context-based information correlated with the environmental sound. Accordingly, the information processing apparatus 100 can grasp what kind of information should be output at what timing when a certain environmental sound is detected.

In such a case, in the example of FIG. 1, the combination of the “date and time” and the “context data” stored in the context information storage unit 122 becomes the objective variable of the model generated by the analysis unit 133 . Also, the combination of the "date and time" and the "environmental sound data" stored in the environmental sound information storage unit 121 becomes an explanatory variable. In addition, "day of the week", "weather", "physical condition", etc. may be used as explanatory variables (feature information).

Note that the learning method for the model generated by the analysis unit 133 is not limited to the example below, and various known machine learning methods may be employed.

For example, the analysis unit 133 determines that the individual feature information extracted from the environmental sound data indicating the curtain sound is, for example, "user U1 should broadcast news to the smart speaker 30 within 5 minutes after the curtain sound is generated." It calculates what kind of weight is given to the event "to give a voice instruction". As a result, the analysis unit 133 determines how much the individual feature information contributes to the phenomenon that "the user U1 issues a voice instruction to the smart speaker 30 to play news within five minutes after the curtain sound is generated". You can get information about what to do. Then, the analysis unit 133 uses the calculated information to generate a model indicating a tendency.

Also, the determination unit 134 determines the output timing using the model generated by the analysis unit 133 . For example, assume that the smart speaker 30 detects the curtain sound at "July 2, 2018, 7:05:00". In this case, the determining unit 134 inputs “July 2, 2018, 7:05:00” and the environmental sound data indicating the curtain sound to the generated model. For example, if the output result at this time is "to output news at 7:05:10 on July 2, 2018", the determining unit 134 outputs "at 7:05 on July 2, 2018". 10 seconds” as the output timing.

[4-2. Regarding relationship analysis (2)]
In the above embodiment, the analysis unit 133 performs correlation analysis as an example of analyzing the relationship between the environmental sound acquired by the acquisition unit 132 and the user's context. However, the analysis unit 133 learns the relationship between the environmental sound acquired by the acquisition unit 132 and the context of the user, and thus the analysis unit 133 learns the user's may generate models that show trends in the context of Such a model can be rephrased as a model that determines (outputs) the timing of outputting information based on the context correlated with the environmental sound when the detected environmental sound and the occurrence time of the environmental sound are input. Accordingly, the information processing apparatus 100 can grasp what kind of information should be output at what timing when a certain environmental sound is detected.

In addition, the analysis unit 133 generates a model for each user by learning the relationship between the environmental sound acquired by the acquisition unit 132 and acquired for each user and the context of the user. . On the other hand, the analysis unit 133 learns the relationship between the environmental sounds acquired by the acquisition unit 132 for a plurality of predetermined users and the contexts of the plurality of predetermined users. , may generate a model corresponding to the predetermined plurality of users. In such a case, the analysis unit 133 can generate a model using the contexts of a plurality of predetermined users as an average context obtained by averaging the contexts of a plurality of predetermined users.

Further, the determining unit 134 may determine the timing of outputting the predetermined information to the output target user based on the model of the other user who is the user other than the output target user who outputs the predetermined information. . Here, when the analysis unit 134 generates a model of an output target user (for example, user U1), it is necessary to accumulate context information for a predetermined period of time for user U1. One of the backgrounds is that a highly accurate model may not be generated unless a sufficient amount of context information is accumulated. Therefore, if the analysis unit 133 has not generated a model corresponding to the user U1 because the context information of the user U1 is insufficient, the determination unit 134 determines that the model has already been generated based on a sufficient number of context information, for example. Based on the generated model of another user (for example, user U2), the timing of outputting predetermined information to user U1 is determined.

Note that the determination unit 134 provides predetermined information to the user U1 based on the model of another user (for example, the user U2) only when the model corresponding to the user U1 is not generated by the analysis unit 133. It is not necessary to decide when to output.

[4-3. About environmental sounds]
In the above embodiment, the environmental sounds to be used for information processing by the information processing apparatus 100 are the environmental sounds of the surrounding environment of the user. An example of output mechanical sound is shown. However, the environmental sounds of the user's surrounding environment need not be limited to these, and may be, for example, human speech. For example, conversational noise (chattering noise) due to a plurality of people having a conversation can be mentioned. For example, in room R1 where home HP1 is located, parties are frequently held around 20:00, and specific music (BGM) is always played from the smart speaker according to voice instructions from user U1.

In such a case, the information processing apparatus 100 can obtain from the learning result the tendency of the user U1 to play specific BGM when conversation noise is detected between 20:00 and 20:00. Then, as described above, for example, when conversation noise is detected around 20:00 at the next party, the information processing device 100 detects noise from the smart speaker 30 prior to the operation by the user U1. BGM can be output.

[4-4. Standalone format]
In the above embodiment, the information processing device 100 cooperates with the smart speaker 30 to perform information processing according to the embodiment. However, the information processing described as being performed by the information processing device 100 may be performed independently (standalone) on the smart speaker 30 side. In such a case, the information processing apparatus 100 is not included in the information processing system 1 according to the embodiment, and the smart speaker 30, including the information processing apparatus 100, communicates the detected environmental sound to other external devices. or send any context data. Also, in that case, the smart speaker 30 does not need to have the receiving unit 131 described with reference to FIG. 3 .

[4-5. About timing]
In the above embodiment, an example in which the determination unit 134 causes the smart speaker 30 to output information (audio output) has been described. However, the device to which the determining unit 134 outputs information is not limited to the smart speaker 30 . For example, the determination unit 134 may determine the timing of outputting (displaying) content from the terminal device 10 of the user.

For example, assume that user U1 wakes up at 7:00 every morning, picks up the terminal device 10, activates the terminal device 10, and then browses the weather forecast site on the terminal device 10. Environmental sounds are generated when the terminal device 10 is picked up or when the terminal device 10 is activated. Then, the information processing apparatus 100 may obtain a learning result that, for example, there is a relationship indicating a tendency between the environmental sound and the context (browsing the weather forecast site with the terminal device 10).

Accordingly, the information processing device 100 can dynamically display the weather forecast site immediately after the user U1 wakes up and activates the terminal device 10, for example. Alternatively, if the user U1 can specify which region the user U1 tends to check the weather on the weather forecast site, the information processing apparatus 100 may push the weather information for that region.

[5. Hardware configuration]
Also, the smart speaker 30 and the information processing device 100 according to the above embodiment are implemented by a computer 1000 configured as shown in FIG. 8, for example. The information processing apparatus 100 will be described below as an example. FIG. 8 is a hardware configuration diagram showing an example of a computer 1000 that implements the functions of the information processing apparatus 100. As shown in FIG. Computer 1000 has CPU 1100 , RAM 1200 , ROM 1300 , HDD 1400 , communication interface (I/F) 1500 , input/output interface (I/F) 1600 and media interface (I/F) 1700 .

The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400 and controls each section. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program depending on the hardware of the computer 1000, and the like.

HDD 1400 stores programs executed by CPU 1100 and data used by these programs. Communication interface 1500 receives data from other devices via communication network 50 and sends the data to CPU 1100 , and transmits data generated by CPU 1100 to other devices via communication network 50 .

The CPU 1100 controls output devices such as displays and printers, and input devices such as keyboards and mice, through an input/output interface 1600 . CPU 1100 acquires data from an input device via input/output interface 1600 . CPU 1100 also outputs the generated data to an output device via input/output interface 1600 .

Media interface 1700 reads programs or data stored in recording medium 1800 and provides them to CPU 1100 via RAM 1200 . CPU 1100 loads such a program from recording medium 1800 onto RAM 1200 via media interface 1700, and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. etc.

For example, when the computer 1000 functions as the information processing apparatus 100 according to the embodiment, the CPU 1100 of the computer 1000 implements the functions of the control unit 130 by executing programs loaded on the RAM 1200 . In addition, data in storage unit 120 is stored in HDD 1400 . CPU 1100 of computer 1000 reads these programs from recording medium 1800 and executes them, but as another example, these programs may be obtained from another device via communication network 50 .

Also, for example, when the computer 1000 functions as the smart speaker 30 according to the embodiment, the CPU 1100 of the computer 1000 implements the functions of the control unit 13 by executing programs loaded on the RAM 1200 .

[6. others〕
Of the processes described in each of the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all of the processes described as being performed manually Alternatively, some can be done automatically by known methods. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified.

Also, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the one shown in the figure, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

As described above, the embodiments of the present application have been described in detail based on several drawings, but these are examples, and various modifications and It is possible to carry out the invention in other forms with modifications.

Also, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the acquisition unit can be read as acquisition means or an acquisition circuit.

1 information processing system 10 terminal device 30 output device 100 information processing device 120 storage unit 121 environmental sound information storage unit 122 context information storage unit 123 target information storage unit 130 control unit 131 reception unit 132 acquisition unit 133 analysis unit 134 determination unit

Claims (16)

an acquisition unit that acquires environmental sounds of the user's surrounding environment;
an analysis unit that analyzes the relationship between the environmental sound acquired by the acquisition unit and the context of the user corresponding to the environmental sound;
Predetermined information is provided to the user based on the context of the user, which is indicated by the analysis result of the analysis unit and which has a relationship indicating a temporal tendency with respect to the environmental sound. and a determination unit that determines the output timing ,
The determination unit determines a timing to output information requested by a user in a context having a relationship indicating a temporal tendency to the environmental sound as the predetermined information.
An information processing device characterized by: The information processing apparatus according to claim 1, wherein the acquisition unit acquires environmental sounds in the user's living space as the environmental sounds of the user's surrounding environment. The information processing according to claim 1 or 2, wherein the acquisition unit acquires, as the environmental sound of the user's surrounding environment, an environmental sound that occurred before a situation indicated by the user's context occurred. Device. The acquisition unit acquires, as the environmental sound generated before the situation indicated by the user's context, an environmental sound that occurred within a predetermined period of time before the situation indicated by the user's context occurred. The information processing apparatus according to claim 3. The determination unit determines the timing of outputting the predetermined information based on a context indicating an operation on a predetermined output device as a context having a relationship indicating a temporal tendency with respect to the environmental sound. The information processing apparatus according to any one of claims 1 to 4. 6. The determining unit determines the timing of outputting the predetermined information based on a context indicating a voice operation on the predetermined output device as the operation on the predetermined output device. information processing equipment. The determination unit identifies, from the analysis results of the analysis unit, contexts of the user that have a relationship indicating a temporal tendency to the environmental sound, and based on the identified context, 7. The information processing apparatus according to any one of claims 1 to 6, wherein timing for outputting the predetermined information is determined. The method according to any one of claims 1 to 7, wherein the analysis unit analyzes the relationship between the feature information extracted from the environmental sound and indicating the features of the user and the context of the user. The information processing device according to any one of the above. The analysis unit learns the relationship between the environmental sound acquired by the acquisition unit and the user's context, thereby indicating the tendency of the user's context within a predetermined time period after the environmental sound is generated. generate the model,
9. The method according to any one of claims 1 to 8 , wherein the determining unit determines timing for outputting the predetermined information to the user based on the model generated by the analyzing unit. information processing equipment. The analysis unit generates the model for each user,
The determination unit determines the timing of outputting the predetermined information to the output destination user based on a model of another user who is a user other than the output destination user among the models for each user. 10. The information processing apparatus according to claim 9 , characterized by: The analysis unit generates the environmental sound and the The information processing apparatus according to any one of claims 1 to 10 , wherein the relationship with the user's context is analyzed . The determining unit, based on the time interval between the time of occurrence indicated by a context having a relationship indicating a temporal tendency with respect to the environmental sound and the time of occurrence of the environmental sound, to the user The information processing apparatus according to any one of claims 1 to 11 , wherein the timing for outputting predetermined information is determined. When an environmental sound having a relationship indicating a temporal tendency with the context is detected in a time zone corresponding to the time of occurrence, the determination unit determines whether the environmental sound is detected within the time interval from the time when the environmental sound was detected. 13. The information processing apparatus according to claim 12 , wherein the time of is determined as the timing for outputting the predetermined information. The determination unit determines, for each user, the timing of outputting the predetermined information based on the context of each user corresponding to the environmental sound acquired by the acquisition unit. 14. The information processing device according to any one of 13 . An information processing method executed by an information processing device,
an acquisition step of acquiring environmental sounds of the user's surrounding environment;
an analyzing step of analyzing the relationship between the environmental sound acquired by the acquiring step and the context of the user corresponding to the environmental sound;
Predetermined information is provided to the user based on the context of the user, which is the context indicated by the analysis result of the analysis step and which has a relationship indicating a temporal tendency to the environmental sound. and a determining step of determining the output timing ,
The determining step determines timing for outputting, as the predetermined information, information requested by the user in a context having a relationship indicating a temporal tendency to the environmental sound.
An information processing method characterized by: an acquisition procedure for acquiring environmental sounds of a user's surrounding environment;
an analysis procedure for analyzing the relationship between the environmental sound acquired by the acquisition procedure and the context of the user corresponding to the environmental sound;
Predetermined information is provided to the user based on the context of the user, which is indicated by the analysis result of the analysis procedure and which has a relationship indicating a temporal tendency with respect to the environmental sound. causing a computer to execute a determination procedure for determining the output timing and
The determination procedure determines the timing of outputting the information requested by the user in a context having a relationship indicating a temporal tendency to the environmental sound as the predetermined information.
An information processing program characterized by:

Images