JP2018097474A - Information service system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information service system capable of making a proposal at an appropriate timing by learning a proposal result.SOLUTION: A center 12 of an information service system 10 has: a proposal acquisition section 38 acquiring a proposal to a user; and a timing learning section 35 which identifies a state containing a user location and a user status in the location as feature quantity, and when a proposal is made, grants a reward to the state when the proposal is made according to an acceptance result of the proposal and manages the state as learning information. Further, an agent ECU 11 connected to the center 12 has a timing determination section 20 which, referring to a state specified by the center 12 and strengthened learning information which is obtained from the center 12 and which has a state identical to the specified state, determines whether the specified state corresponds to a timing suitable to proposal, and when it is determined that the specified state corresponds to the suitable timing, makes a proposal to the user.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information providing system that learns a user acceptance result for a proposal and makes a proposal to the user based on the learning result.

  Conventionally, as this type of information providing system, for example, a navigation device described in Patent Document 1 is known. This navigation device proposes to a user a facility located within a predetermined suggested distance from a planned travel route of a vehicle on which the device is mounted as a stop-off point. The suggested distance is set for each genre or environment of the facility to be proposed, and is corrected based on the degree of acceptance of the proposal by the user in the past. For example, when the number of times of acceptance with respect to the number of proposals is 80% or more, the proposal distance is corrected to become longer. When the number of times of acceptance with respect to the number of proposals is less than 60%, the proposal distance is corrected to be shortened. From the next proposal, the proposed distance corrected in this way is used.

Japanese Patent Laying-Open No. 2006-121879

  However, the navigation device is based on the assumption that the user has a plan to travel in the vehicle. On the other hand, among users, there is a user who has no driving plan but has a potential demand for driving. If the drive content can be proposed to such a user, highly useful content can be proposed.

  However, if the proposal is frequently made when the user is not on the board, the proposal may be made even when the user has no desire for driving, such as when the user does not want to go out. Thus, if a drive proposal is made at a timing that conflicts with the user's request, a less useful proposal will be made.

  The present invention has been made in view of such a situation, and an object thereof is to provide an information providing system capable of making a proposal at an appropriate timing by learning a proposal result.

Hereinafter, means for solving the above-described problems and the effects thereof will be described.
An information providing system that solves the above problems includes a proposal acquisition unit that acquires a proposal for a user, a state specifying unit that specifies a location including the user's whereabouts and a user state at the location as a feature amount, and a proposal. When the proposal is accepted, a proposal result learning unit which gives a reward to the state when the proposal is made and uses it as learning information is newly specified by the state specifying unit. Determining whether the specified state corresponds to a timing suitable for the proposal, with reference to the state and the learning information in the same state as the specified state learned by the proposal result learning unit, A proposal timing determination unit that makes a proposal to the user when it is determined that the identified state corresponds to a timing suitable for the proposal.

  According to the said structure, the acceptance result of the proposal performed in the past is learned with the state containing a user's whereabouts and a user state as a feature-value. Then, based on the learning result, it is determined whether or not the timing is suitable for the proposal, and the proposal is made when the timing is suitable for the proposal. For this reason, a highly useful proposal can be made to the user at a timing at which the proposal is easily accepted.

The block diagram which shows schematic structure about one Embodiment of an information provision system. The conceptual diagram of the learning table in the embodiment. The flowchart which shows the procedure of the state specification in the embodiment. The flowchart which shows the procedure of reinforcement learning in the embodiment. The flowchart which shows the procedure of the determination process of the proposal timing in the embodiment.

Hereinafter, an embodiment of the information providing system will be described.
The information providing system according to the present embodiment is an apparatus that can be brought into a vehicle, and includes an agent ECU (electronic control apparatus) that makes suggestions regarding driving to a user. The agent ECU performs reinforcement learning based on the acceptance result when the proposal is made. Reinforcement learning means that when an agent ECU selects an action based on the environment, the agent ECU is adapted to the environment through trial and error by giving the agent some kind of reward as the environment changes based on the selected action. It is a learning method that goes. In the present embodiment, the agent ECU forms a state space that is a set of a plurality of states by “state” including the user's whereabouts, the user's state (user state) at the whereabouts, and the like. In addition, whether or not to accept a user proposal in each state corresponds to a reward in terms of reinforcement learning. The agent ECU identifies a state constituting the state space at a predetermined timing, compares the identified state with the result of reinforcement learning, and the identified state is a state where the possibility of accepting the proposal is high. When it is determined that the timing is suitable, a proposal is made through dialogue with the user.

  The configuration of the information providing system 10 will be described with reference to FIG. The information providing system 10 includes an agent ECU 11 and a timing learning unit 35 of the center 12 connected to the agent ECU 11. In this embodiment, the agent ECU 11 is mounted on a portable information terminal that can be brought into the vehicle. The agent ECU 11 is connected to the center 12 via the communication unit 15. The center 12 acquires drive information according to the attributes and preferences of the user who uses the portable information terminal. When the center 12 acquires the drive information, the center 12 transmits the drive information to the agent ECU 11. The drive information is information including a destination, a waypoint, and the like, and is provided both when the user is in the car and at home. As the drive information, the agent ECU 11 provides, for example, a user who is in the vehicle heading to the destination or home to a place that is estimated to be of interest to the user and can be reached by the vehicle. Moreover, agent ECU11 provides the destination which can be headed with a vehicle which is a place presumed to be a user's interest with respect to the user at home as drive information.

  The configuration of the agent ECU 11 will be described. The agent ECU 11 includes an image recognition unit 16 that acquires an image from the camera 50 and performs image recognition. The image recognition unit 16 acquires image information from the camera 50 provided in the portable information terminal. The image acquired by the camera 50 is an image of the user or the surrounding environment of the user. Or you may make it the image recognition part 16 acquire image information from the vehicle-mounted camera which images the inside of a vehicle, and the camera which images the inside of a house. The image recognition unit 16 performs image processing on the acquired image, recognizes an object or the like, and outputs the image recognition result to the user state estimation unit 17 and the user behavior estimation unit 18.

  In addition to acquiring the image recognition result from the image recognition unit 16, the user state estimation unit 17 acquires vehicle information and operation information from the in-vehicle device 54 via wireless communication or the like, and acquires from the home appliance 55 via wireless communication or the like. Operation information of the home appliance 55 is acquired. The in-vehicle device 54 is, for example, a navigation system or an operation device for an air conditioning system. The home appliance 55 is a television, an audio system, or the like. When the user state estimation unit 17 acquires the image recognition result, the vehicle information, the operation information of the in-vehicle device 54, and the operation information of the home appliance 55, the user state estimation unit 17 integrates the information and transmits it to the center 12 as user state estimation information. Moreover, the user state estimation part 17 receives the learning result of the user state with respect to user state estimation information from the center 12, and determines a user state.

  Specifically, the user state estimation unit 17 determines whether the user's whereabouts are “inside the vehicle”, “home”, or any other place based on the learning result of the user state acquired from the center 12. In addition, when the user state estimation unit 17 determines that the user's whereabouts are in the vehicle, the user state estimation unit 17 determines the traveling area of the vehicle and the surrounding situation of the vehicle based on the learning result of the user state. The travel area may be, for example, a road type such as “highway” or “general road”, and indicates whether the area is near the home (living area) or far away from the home (outside the living area). It may be information. The surrounding situation of the vehicle is traffic information such as “congestion”, “under road regulation”, “other than that”, and the like. Further, when the user state estimation unit 17 determines that the user's whereabouts are in the vehicle, based on the learning result of the user state or the vehicle information, for example, “forward”, “reverse”, “stop”, “getting on and off”, etc. The vehicle state is determined. “Forward”, “reverse”, “stop” and “getting on and off” can be acquired from a control device connected to the in-vehicle network such as a navigation system. Further, when the user state estimating unit 17 determines that the user's whereabouts are at home, the user's home such as “standing”, “sitting”, “walking”, “sleeping”, etc., based on the learning result of the user state. The state at is determined.

  Similar to the user state estimation unit 17, the user behavior estimation unit 18 acquires vehicle information and operation information from the in-vehicle device 54 via wireless communication or the like in addition to acquiring the image recognition result from the image recognition unit 16. Operation information of the home appliance 55 is acquired from the device 55 via wireless communication or the like. When acquiring the image recognition result, the operation information of the in-vehicle device 54, and the operation information of the home appliance 55, the user behavior estimation unit 18 integrates the information and transmits the information to the center 12 as user behavior estimation information. In addition, the user behavior estimation unit 18 receives the learning result of the user behavior with respect to the user behavior estimation information from the center 12 and determines the user behavior.

  Specifically, when it is determined that the user's whereabouts are in the vehicle, the user behavior estimating unit 18, for example, “during device operation”, “during conversation”, based on the learning result of the user behavior acquired from the center 12. The user's behavior in the vehicle such as “sleeping” is determined. On the other hand, when it is determined that the user's whereabouts are at home, the user behavior estimating unit 18 is based on the learning result of the user behavior acquired from the center 12, for example, “drinking”, “watching audio”, “busy” The user's behavior at home such as “is estimated.

  Further, the agent ECU 11 includes a timing determination unit 20. The timing determination unit 20 acquires the user state from the user state estimation unit 17 and the user behavior from the user behavior estimation unit 18 at a predetermined timing such as when the agent ECU 11 receives drive information. The agent ECU 11 transmits the acquired user state and user behavior to the center 12. In addition, the timing determination unit 20 receives the result of the reinforcement learning of the timing determination situation from the center 12, and based on the result of the reinforcement learning, the state S (i) specified by the user state and the user action proposes the drive. It is determined whether or not the timing is suitable. Then, when the state determination unit 20 determines that the state S (i) is suitable for the drive timing, the timing determination unit 20 requests the dialogue control unit 25 to output drive information. The dialogue control unit 25 outputs the drive information received from the center 12 to the voice synthesis unit 26. The voice synthesizing unit 26 converts the content of the drive information into voice, and outputs the drive information as voice through the speaker 52 provided in the portable information terminal.

  The dialogue control unit 25 determines whether or not the user can accept the proposal based on a dialogue with the user. Voice generated by the user's utterance is converted into a signal through the microphone 51 of the portable information terminal and input to the voice recognition unit 27. The voice recognition unit 27 analyzes the input voice signal and performs voice recognition to determine whether or not the proposal has been accepted. Then, proposal result information indicating acceptance / rejection is transmitted to the center 12.

  Next, the configuration of the center 12 will be described. The center 12 includes a communication unit 30 that communicates with the agent ECU 11, a user state learning unit 31, and a user behavior learning unit 32. The user state learning unit 31 learns the user state estimation information transmitted by the agent ECU 11, information obtained through dialogue with the user, and the like, and records the learning result in the learning result storage unit 33. For example, when the user state is specified, the user state learning unit 31 learns whether or not the user state specified through the dialogue with the user matches the actual state. Furthermore, when the user state learning unit 31 receives the user state estimation information from the agent ECU 11, the user state learning unit 31 assigns a user state having a high similarity to the user state estimation information based on the learning result stored in the learning result storage unit 33. Send to.

  The user behavior learning unit 32 learns the user behavior estimation information transmitted by the agent ECU 11, information obtained through dialogue with the user, and the like, and records the learning result in the learning result storage unit 33. Further, when the user behavior learning unit 32 receives the user behavior estimation information from the agent ECU 11, the user behavior learning unit 32 indicates a user state having a high similarity to the user behavior estimation information based on the learning result stored in the learning result storage unit 33. Send to.

  The center 12 includes a timing learning unit 35. The timing learning unit 35 stores the proposal result information transmitted from the agent ECU 11 in the proposal history storage unit 36 as proposal history information. The timing learning unit 35 performs reinforcement learning based on the proposal history information of one state, and records the reinforcement learning result of the state in the learning result storage unit 33 in association with the state.

  Further, the center 12 includes a proposal acquisition unit 38. The proposal acquisition unit 38 acquires drive information that can be proposed to the user based on the user's attributes (age, gender, address, whereabouts ...), the situation (weather, time zone ...), the user's preferences, and the like.

Next, reinforcement learning of proposal timing will be described. The feature quantity X (j) of the state S (i) for determining the proposal timing is defined as follows, for example.
-X1: User's whereabouts: in the car, at home-X2: Details of whereabouts of the vehicle ... Driver's seat, passenger's seat, rear seat-X3: Details of whereabouts at home ... Living, dining, kitchen, bedroom, bathroom, toilet * X4: Vehicle's location Travel area: Expressway, general road (living area), general road (out of living area)
-X5: Vehicle surroundings ... Traffic jams, road regulation, other states-X6: User status in vehicles ... Advancing, retreating, stopping, getting on / off-X7: User status at home ... standing posture, Sitting posture, walking, lying down X8: User behavior in the vehicle ... operating the device, talking, sleeping ... X9: User behavior at home ... drinking, watching audio, working, calling, talking, During sleep and smartphone operation As shown in FIG. 2, the reinforcement learning information 100 as learning information is recorded in the learning result storage unit 33. When the timing learning unit 35 acquires the proposal result information from the agent ECU 11, the timing learning unit 35 records the proposal result information in the proposal history storage unit 36 as the proposal history information. Further, the state S (i) determined by the user state (X1 to X7) specified by the user state learning unit 31 corresponding to the proposal result information and the user behavior (X8, X9) specified by the user behavior learning unit 32 Is identified. The state S (i) may be already learned or may be newly obtained.

  The timing learning unit 35 calculates a reward R (Si) for the specified state S (i) based on the newly obtained proposal result information. For example, when a proposal based on drive information is made in the state S (i), R (Si) is increased as a reward when accepted (for example, “1”), and a reward R when not accepted (( Si) is lowered (for example, “0”).

  Then, the timing learning unit 35 calculates the state value function V (Si) based on the calculated reward R (Si). The timing learning unit 35 sets an average of the reward R (Si) ′ calculated in the past and the newly calculated reward R (Si) as the state value function V (Si) of the state S (i). For example, when the specified state S (i) has not been learned in the past, the calculated reward R (Si) becomes the state value function V (Si). On the other hand, when the reward R (Si) has already been given to the state S (i) and the reward R (Si) ′ is newly calculated, the phases of the rewards R (Si) and R (Si) ′ The arithmetic mean is the state value function V (Si). In addition, since it is only necessary to reflect the past reward R (Si) in the state value function V (Si), the average value other than the arithmetic mean such as the median of the rewards R (Si) and R (Si) ′ and the geometric mean It may be “average”.

  In the states S (1) to S (5) shown in FIG. 2, a proposal has been made only once in the past. For example, since the proposal was not accepted in the state S (1) (“failure”), the state value function V (Si) is a low value such as “0”. Further, since the proposal is accepted in the state S (2) (“success”), the state value function V (Si) is a high value such as “1”.

  When the proposed drive information exists, the state S (i) transitions to S (3), S (4), S (5) shown in FIG. 2 and is in the state S (5). If a proposal is made and the proposal is accepted, a high reward is given to all of S (3), S (4), and S (5). As a result, the state value function V (Si) of each state S (3) to S (5) becomes high.

  In the state S (6) shown in FIG. 2, the averaged reward R (Si) is a state value function V (Si). When the number of proposals in the same state S (i) is large, the reward R (Si) is averaged even if the reward R (Si) changes to “0”, “1”, etc. depending on the proposal timing. It is possible to prevent the value of the state value function V (Si) from being excessively large or small.

  A state S (i) having a high state value function V (Si) is a state in which it is estimated that the user can easily accept the proposal, and is suitable as a timing for making the proposal. The state S (i) where the state value function V (Si) is low is a state that is estimated to be difficult for the user to accept the proposal, and is not suitable as a timing for making the proposal. Based on the reinforcement learning result, the agent ECU 11 determines whether or not the specified state S (i) is suitable as a timing for making a proposal based on drive information.

Next, processing procedures by the center 12 and the agent ECU 11 will be described with reference to FIGS.
First, with reference to FIG. 3, the identification process of the state S (i) by the center 12 will be described. The timing learning unit 35 estimates the user's whereabouts based on the user state estimation information and user behavior estimation information acquired from the agent ECU 11 and the learning information recorded in the learning result storage unit 33 (step S10). Moreover, the timing learning part 35 estimates whether a user is in a vehicle based on the estimation result of a whereabouts (step S11). When the timing learning unit 35 estimates that the user is in the vehicle (step S11: YES), the timing learning unit 35 identifies the travel area based on the vehicle information acquired from the agent ECU 11 (step S12), and identifies the situation of the travel area (step S13). ). Moreover, the timing learning part 35 estimates the state of a vehicle (step S14), and estimates a user's action in a vehicle (step S15).

  On the other hand, when the timing learning unit 35 estimates that the user's whereabouts are not in the vehicle (step S11: NO), the timing learning unit 35 estimates whether or not the user's whereabouts are at home (step S16). If the timing learning unit 35 estimates that the user's whereabouts are not at home (step S16: NO), the timing learning unit 35 transmits a notification that the state S (i) cannot be specified to the agent ECU 11, and ends the specifying process. When the timing learning unit 35 estimates that the user's whereabouts are at home (step S16: YES), the timing learning unit 35 estimates the state of the user at home (step S17) and estimates the user's behavior at home (step S18). ).

  Next, with reference to FIG. 4, the procedure of the reinforcement learning of the proposal timing by the center 12 is demonstrated. This process is performed when the center 12 receives a proposal acceptance result from the agent ECU 11. The timing learning unit 35 gives a reward R (Si) to the state S (i) based on the proposal result information transmitted from the agent ECU 11 (step S20). For example, when the timing learning unit 35 determines that the proposal is accepted, the timing learning unit 35 sets “1” as a reward R (Si), and when determining that the proposal is not accepted, the timing learning unit 35 sets the reward R (Si) as “0”.

  Next, the timing learning unit 35 updates the state value function V (Si) (step S21). The timing learning unit 35 calculates an average “mean (R (Si))” of the reward calculated for the state S (i) and the reward already given to the state S (i), and calculates the average. Let it be a new state value function V (Si). When the timing learning unit 35 calculates the state value function V (Si), the timing learning unit 35 records it as the reinforcement learning information 100 in the learning result storage unit 33.

  Next, with reference to FIG. 5, the process of determining the proposal timing by the agent ECU 11 will be described. The timing determination unit 20 of the agent ECU 11 determines whether there is a proposal for the drive (step S1). At this time, the agent ECU 11 determines whether or not drive information is received from the center 12. For example, the drive information includes a destination or waypoint, a route from the departure point to the destination, a required time, and the like.

  When the timing determination unit 20 of the agent ECU 11 determines that there is no proposal content (step S1: NO), it ends the proposal timing determination process, returns to step S1, and waits for provision of drive information from the center 12. On the other hand, when the timing determination unit 20 of the agent ECU 11 determines that there is a drive proposal content (step S1: YES), it estimates the state S (i) (step S2). Specifically, the timing determination unit 20 requests the user state estimation unit 17 and the user behavior estimation unit 18 to output estimation information. The user state estimation unit 17 and the user behavior estimation unit 18 obtain a recognition result from the image recognition unit 16 and obtain various information from the in-vehicle device 54 and the home appliance 55. The timing determination unit 20 acquires user state estimation information from the user state estimation unit 17, acquires user behavior estimation information from the user behavior estimation unit 18, and transmits it to the center 12.

  When the communication unit 30 of the center 12 acquires the user state estimation information and the user behavior estimation information from the agent ECU 11, the user state learning unit 31 and the user behavior learning unit 32 specify the state S (i). The state S (i) specified by the user state learning unit 31 and the user behavior learning unit 32 is output to the timing learning unit 35. The timing learning unit 35 searches the reinforcement learning information 100 for the specified state S (i). When the same state S (i) is extracted as a result of the search, the timing learning unit 35 transmits the state value function V (Si) of the state S (i) to the agent ECU 11 as a reinforcement learning result. On the other hand, when the same state S (i) cannot be extracted, the timing learning unit 35 transmits a search result indicating that there is no same state S (i) to the agent ECU 11 as a reinforcement learning result.

  The timing determination unit 20 of the agent ECU 11 determines whether or not the reinforcement learning result has been acquired from the center 12 (step S3). When the reinforcement learning result cannot be acquired from the center 12 (step S3: NO), for example, when the user's whereabouts cannot be specified, for example, the timing determination unit 20 makes a suggestion based on the proposal contents of the drive information (step S3). S5). Specifically, the timing determination unit 20 outputs a proposal request to the dialogue control unit 25. The dialogue control unit 25 outputs a voice based on the proposal content from the speaker 52 via the voice synthesis unit 26.

  On the other hand, when acquiring the reinforcement learning result (step S3: YES), the timing determination unit 20 determines whether or not the specified state S (i) is a timing suitable for the proposal based on the acquired reinforcement learning result. (Step S4). At this time, the timing determination unit 20 determines whether or not the state value function V (Si) that is the reinforcement learning result is equal to or greater than a predetermined value. If the state value function V (Si) is equal to or greater than a predetermined value, it is proposed that the state S (i) is at a timing suitable for the proposal based on the proposed contents of the drive information (step S5). On the other hand, if the state value function V (Si) is less than the predetermined value, the timing determination unit 20 does not propose drive information, ends the process, and returns to step S1. Further, when receiving a search result indicating that there is no same state S (i), the timing determination unit 20 considers that the timing is suitable for the proposal and makes a proposal based on the proposal content of the drive information ( Step S5).

  When the proposal is executed, the dialogue control unit 25 acquires the proposal result through dialogue with the user (step S6). The voice recognition unit 27 recognizes the user's utterance voice to the proposal and outputs the utterance content to the dialogue control unit 25. The dialogue control unit 25 determines whether the user has accepted the proposal based on the utterance content, and transmits the proposal result information to the center 12. The center 12 receives the proposal result information, and performs reinforcement learning of the proposal timing based on the received proposal result information (step S7, see FIG. 4).

  In this way, the agent ECU 11 does not make a proposal to the user every time the drive information is acquired, but reinforcement learning is performed for each user at a timing at which the user can easily accept the proposal, and suitable for the proposal based on the reinforcement learning result. Make a suggestion at the timing. For this reason, the user does not feel bothered compared to the case where the proposal is made each time the drive information is acquired. In addition, information that is highly useful to the user can be provided to a user who has a hope of driving by proposing a drive at a timing suitable for the proposal.

As described above, according to the present embodiment, the following effects can be obtained.
(1) In the above embodiment, the center 12 learns the results of accepting proposals made in the past together with the user's whereabouts, the user state at the whereabouts, and the state including the user behavior as feature quantities. Then, based on the reinforcement learning result acquired from the center 12, the agent ECU 11 determines whether or not the timing is suitable for the proposal, and the proposal is made when the timing is suitable for the proposal. For this reason, a highly useful proposal can be made to the user at a timing at which the proposal is easily accepted.

(Other embodiments)
In addition, the said embodiment can also be implemented with the following forms.
In the above embodiment, it is determined whether the proposal has been accepted through dialogue with the user via the dialogue control unit 25. Instead, it may be determined whether the proposal is accepted by turning on the touch panel display of the portable information terminal or the operation button when accepting the proposal.

  In the above-described embodiment, the timing learning unit 35 of the center 12 specifies the state determined by the feature amount of the user state and the feature amount of the user action. In addition to this, any one of the user state learning unit 31 and the user behavior learning unit 32 may specify a state defined by these feature amounts. Alternatively, the agent ECU 11 may learn the user state and the user behavior based on the image recognition result, and specify the state determined by the feature amount of the user state and the feature amount of the user behavior.

  In the above embodiment, the agent ECU 11 executes the proposal when the reinforcement learning result cannot be obtained from the center 12 (step S3: NO) (step S5). Instead, when the reinforcement learning result cannot be obtained from the center 12 (step S3: NO), the proposal may not be executed. In this aspect, for example, the proposal at a timing not based on the reinforcement learning result is repeated a predetermined number of times, and the reinforcement learning result obtained by learning the acceptance result of the proposal is accumulated.

  In the above embodiment, the proposal is made when the user's location is “in the car” or “home”. In addition to this, a proposal may be made when the user's whereabouts are other than “in the car” or “home”. For example, the proposal may be made when the user's whereabouts are “company” or “train” and a predetermined time zone.

  In the above embodiment, the timing learning unit 35 gives the reward when the proposal is accepted, for example, as “1” to the state S (i). In addition to this, the user's feelings for the proposal may be estimated through dialogue with the user, and the reward may be changed according to the user's feelings. For example, if the user's emotion to the proposal when the proposal is accepted is positive and includes “joy” or the like, the reward is increased, and the user's emotion to the proposal that has accepted the proposal is “joy”. The reward may be lowered if it does not include positive emotions such as.

In the above embodiment, the timing learning unit 35 that performs reinforcement learning is provided in the center 12, but the agent ECU 11 may perform reinforcement learning.
In the above embodiment, the agent ECU 11 proposes drive information. However, the agent ECU 11 may provide information (outing information) for traveling to a destination or waypoint by train, bicycle, or walking.

-In the above-mentioned embodiment, the state which constitutes state space was defined by user state and user action. Instead of this, the state may be determined only by the user state.
In the above embodiment, the timing learning unit 35 that performs reinforcement learning is provided in the center 12. Instead of this, the timing learning unit 35 may be provided in the agent ECU 11.

  In the above embodiment, the agent ECU 11 is provided in the portable information terminal, but instead, the agent ECU 11 may be provided in an in-vehicle device 54 such as a navigation system provided in the vehicle. In this case, the user state and user behavior in the vehicle and the proposal result are learned. Alternatively, the agent ECU 11 may be provided in the home appliance 55. In this case, the user state and user behavior at home and the proposal result are learned. Further, the center 12 may integrate the information from the agent ECU 11 provided in the in-vehicle device 54 and the information from the agent ECU 11 received by the home appliance 55. Alternatively, the agent ECU 11 may be provided in a device or system other than the portable information terminal, the in-vehicle device 54, and the home appliance 55.

  DESCRIPTION OF SYMBOLS 10 ... Information provision system, 11 ... Agent ECU, 12 ... Center, 15, 30 ... Communication part, 16 ... Image recognition part, 17 ... User state estimation part, 18 ... User action estimation part, 20 ... Timing determination part, 25 ... Dialog control unit, 26 ... voice synthesis unit, 27 ... voice recognition unit, 31 ... user state learning unit, 32 ... user behavior learning unit, 33 ... learning result storage unit, 35 ... timing learning unit, 36 ... suggestion history storage unit, 38 ... Proposal acquisition unit, 50 ... Camera, 51 ... Microphone, 52 ... Speaker, 54 ... In-vehicle device, 55 ... Home appliance, 100 ... Reinforcement learning information.

Claims (1)

A proposal acquisition unit for acquiring a proposal for the user;
A state identifying unit for identifying a state including the user's whereabouts and the user state at the whereabouts as a feature amount;
When a proposal is made, according to a result of acceptance of the proposal, a proposal result learning unit that gives a reward to the state when the proposal is made and sets it as learning information;
The specified state is suitable for the proposal with reference to the state newly specified by the state specifying unit and the learning information learned by the suggestion result learning unit and having a high degree of similarity with the specified state. A proposal timing determination unit that determines whether or not the timing corresponds to a timing and determines that the specified state corresponds to a timing suitable for the proposal;