JP2021018073A - Information providing device, information providing method, and program - Google Patents

Abstract

To provide an information providing device, information providing method, and a program for enabling an occupant to notice a predetermined event that occurs around a vehicle.SOLUTION: An information providing device includes: an event recognition section for recognizing that a first event included in a range unusual in an area around a first vehicle occurs on the basis of information acquired around the first vehicle; and a notification section for notifying an occupant of the first vehicle of information on the first event when the event recognition section recognizes that the first event occurs.SELECTED DRAWING: Figure 5

Description

The present invention relates to an information providing device, an information providing method, and a program.

Conventionally, it has been known to generate video data to which information about a celestial body specified based on the position, orientation and date and time of the own vehicle is added, and project the generated video data into the vehicle (for example, Patent Document 1). reference).

Japanese Unexamined Patent Publication No. 2008-064889

However, the conventional technology is related to astronomical observation from the inside of the vehicle, and the technology for occupants to observe other events from inside the vehicle has not been sufficiently studied.

The present invention has been made in consideration of such circumstances, and provides an information providing device, an information providing method, and a program capable of making a occupant aware of a predetermined event occurring in the vicinity of a vehicle. That is one of the purposes.

The information providing device, the information providing method, and the program according to the present invention have adopted the following configurations.
(1): In the information providing device according to one aspect of the present invention, a first event included in an unusual range occurs in the area around the first vehicle based on the information acquired in the vicinity of the first vehicle. When the event recognition unit that recognizes that the event is being performed and the event recognition unit recognizes that the first event has occurred, information about the first event is provided to the occupant of the first vehicle. It is provided with a notification unit for notifying.

(2): In the aspect of the above (1), the event recognition unit previously recognizes that an event occurring in the vicinity of the first vehicle is a daily event in the area around the first vehicle. When it corresponds to the registered daily event list, the event occurring is not recognized as the first event.

(3): In the aspect of (1) or (2) above, the event recognition unit is included in the daily range in the area around the first vehicle, but is in the daily range for the occupant of the first vehicle. Further recognizing that a second event not included in the above has occurred, the notification unit also recognizes that the second event has occurred by the event recognition unit. This is to notify the occupants of the vehicle of information regarding the second event.

(4): In any of the above aspects (1) to (3), the event recognition unit indicates that the event occurring in the vicinity of the recognized first vehicle is in the vicinity of the first vehicle. When it corresponds to the extraordinary event list registered in advance as an extraordinary event in the area, the occurring event is recognized as the first event.

(5): In the aspect of (4) above, the list editing unit for editing the extraordinary event list is further provided based on the instruction by the occupant of the first vehicle.

(6): In any of the above aspects (1) to (5), the event occurring in the vicinity of the first vehicle recognized by the event recognition unit is in the area around the first vehicle. If it does not correspond to the daily event list pre-registered as a daily event and does not correspond to the extraordinary event list pre-registered as a non-daily event in the area around the first vehicle, the first A confirmation unit for confirming the presence or absence of notification to the occupants of one vehicle is further provided, and the notification unit may notify the occupants of the first vehicle when instructed to notify the occupants of the first vehicle. It notifies information about events occurring around the first vehicle.

(7): In the aspect of (6) above, the learning unit further includes a learning unit that learns an event instructed to be notified by the occupant of the first vehicle for each occupant of the first vehicle.

(8): In any of the above aspects (1) to (7), the notification unit has the same event as the event notified to the occupant of the first vehicle within a predetermined period from the previous notification. If it occurs again, it will be excluded from the notification target.

(9): In any of the above aspects (1) to (8), the event recognition unit uses the in-vehicle detection device mounted on the first vehicle to obtain a detection result detected during traveling. Based on this, it recognizes an event occurring in the vicinity of the first vehicle.

(10): In any of the above aspects (1) to (9), the event recognition unit is installed outside the vehicle and the surrounding conditions detected by the in-vehicle detection device mounted on the first vehicle. It recognizes that the event has occurred based on at least one of the surrounding conditions detected by the vehicle exterior detection device.

(11): In any of the above aspects (1) to (10), when the event recognition unit recognizes that the event has occurred, the imaging image mounted on the first vehicle It further includes a data acquisition unit that acquires data obtained by imaging the event using the unit.

(12): In any of the above aspects (1) to (11), the notification unit notifies the occupants of the second vehicle traveling in the area where the event is occurring to that effect. It is a thing.

(13): In any of the above aspects (1) to (12), when the event recognition unit recognizes that the first event has occurred, the vehicle is driven slowly with respect to the first vehicle. It is further provided with an operation instruction unit for instructing.

(14): In the information providing method according to another aspect of the present invention, the computer is included in an area around the first vehicle in an unusual range based on the information acquired in the vicinity of the first vehicle. It is a method of recognizing that one event has occurred, and when it is recognized that the first event has occurred, notifying the occupants of the first vehicle of information about the first event. ..

(15): The program according to another aspect of the present invention is a first event included in an area around the first vehicle in an unusual range based on information acquired in the vicinity of the first vehicle by a computer. Is a program that recognizes that the first event has occurred, and when it is recognized that the first event has occurred, notifies the occupants of the first vehicle of information about the first event.

According to the aspects (1) to (15), it is possible to make the occupant aware of a predetermined event occurring in the vicinity of the vehicle.

The figure which shows an example of the information providing system 1 of this invention. The block diagram of the 1st vehicle 10A which concerns on embodiment. The functional block diagram of the 1st control unit 120 and the 2nd control unit 160. The figure which shows the structure of the in-vehicle notification device 300 and the device mounted on the 1st vehicle 10A. The figure which shows the configuration of the in-vehicle notification device 300 and a part of the configuration of the information providing device 500. The flowchart which shows an example of the process (the 1) executed by the information providing apparatus 500. The flowchart which shows an example of the process (the 2) executed by the information providing apparatus 500. The flowchart which shows an example of the process (the 3) executed by the information providing apparatus 500.

Hereinafter, the vehicle information providing device, the information providing method, and the embodiment of the program of the present invention will be described with reference to the drawings.

[overall structure]
FIG. 1 is a diagram showing an example of the information providing system 1 of the present invention. As shown in FIG. 1, the information providing system 1 includes, for example, a first vehicle 10A, a user terminal 70A, a second vehicle 10B, a user terminal 70B, an information providing device 500, an outside detection device 700C, and an outside vehicle. Includes a detector 700D. The first vehicle 10A, the user terminal 70A, the second vehicle 10B, the user terminal 70B, the information providing device 500, the vehicle outside detection device 700C, and the vehicle outside detection device 700D are connected via the network NW. .. The network NW includes, for example, the Internet, a WAN (Wide Area Network), a LAN (Local1 Area Network), a provider device, a wireless base station, and the like.

The first vehicle 10A and the second vehicle 10B are, for example, vehicles such as two wheels, three wheels, and four wheels, and the drive source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates by using the power generated by the generator connected to the internal combustion engine or the discharge power of the secondary battery or the fuel cell. In the following description, when the first vehicle 10A and the second vehicle 10B are not distinguished, they are referred to as vehicle 10. Although FIG. 1 shows two vehicles, the information providing system may include three or more vehicles.

Each vehicle 10 is an example of a moving body that recognizes the surrounding situation and notifies the information providing device 500 of information indicating the surrounding situation (hereinafter referred to as peripheral situation information). The surrounding situation information is, for example, information indicating that a rainbow appears around the vehicle 10 or a shooting star appears around the vehicle 10. For example, the first vehicle 10A transmits the peripheral situation information acquired in the surrounding area to the information providing device 500 via the network NW. The second vehicle 10B transmits the peripheral situation information acquired in the surrounding area to the information providing device 500 via the network NW. In addition, the vehicle 10 periodically generates information indicating its own current position (hereinafter referred to as current position information) and associates identification information (for example, vehicle ID) for identifying the vehicle 10 with the current position information. May be transmitted to the information providing device 500 via the network NW.

The user terminal 70A is a terminal owned by the user A who is an occupant of the first vehicle 10A. The user terminal 70B is a terminal owned by the user B who is an occupant of the second vehicle 10B. User terminals 70A and 70B include, for example, smartphones, tablet terminals, personal computers and the like.

The information providing device 500 generates notification information based on the surrounding situation information received from the first vehicle 10A, for example, and provides the generated notification information to the user A. The information providing device 500 provides the notification information to the user A by using, for example, the notification agent function. The notification agent function includes, for example, a function of recognizing the situation around the vehicle and notifying the occupant of the vehicle of the recognized content, and a request (command) included in the utterance of the occupant while interacting with the occupant of the vehicle. It includes a function to provide information based on. In addition, some of the notification agent functions may have a function of controlling devices in the vehicle (for example, devices related to driving control and vehicle body control).

Agent functions include, for example, a voice recognition function that recognizes the voice of an occupant (a function that converts voice into text), a natural language processing function (a function that understands the structure and meaning of text), a dialogue management function, and a network. It is realized by integratedly using a network search function or the like that searches for another device or a predetermined database owned by the own device. Some or all of these functions may be realized by AI (Artificial Intelligence) technology. Further, a part of the configuration for performing these functions (particularly, the voice recognition function and the natural language processing interpretation function) may be mounted on the information providing device 500. In the following description, it is premised that a part of the configuration is mounted on the information providing device 500, and the in-vehicle notification device 300 and the information providing device 500 cooperate to realize the agent system. Further, as the agent system, a service provider (service entity) in which the in-vehicle notification device 300 and the information providing device 500 cooperate to virtually appear is referred to as an agent.

An example of the process of notifying the occupants in the vehicle interior of the notification information by using the agent function will be described. For example, when a rainbow appears around the first vehicle 10A, the information providing device 500 generates notification information for notifying by voice or image that a rainbow appears around the first vehicle 10A. For example, the information providing device 500 notifies the user A by transmitting the notification information to the first vehicle 10A and outputting the appearance of the rainbow in the vehicle interior of the first vehicle 10A. Not limited to this, the information providing device 500 may notify the user A by transmitting the notification information to the user terminal 70A and outputting the rainbow from the user terminal 70A. Hereinafter, an example of transmitting the notification information to the first vehicle 10A and notifying the user A will be described.

Further, when the second vehicle 10B is also traveling in the area where the rainbow can be observed, the information providing device 500 may provide the generated notification information to the user B. Similar to the provision of the notification information to the user A, the notification information may be provided to the user B in a mode of outputting to the passenger compartment of the second vehicle 10B, or may be output from the user terminal 70B. Good.

The outside detection device 700C is fixed to the outside installation object 720C, and the outside detection device 700D is fixed to the outside installation object 720D. Each of the external detection devices 700C and 700D is an example of a detection device installed outside the vehicle 10, recognizes the peripheral situation of the externally installed object, and notifies the information providing device 500 of the peripheral situation information. For example, the vehicle exterior detection device 700C transmits peripheral status information indicating the peripheral status of the vehicle exterior installation object 720C to the information providing device 500 via the network NW. The vehicle exterior detection device 700D transmits peripheral condition information indicating the peripheral condition of the vehicle exterior installation object 720D to the information providing device 500 via the network NW. The information providing device 500 may generate notification information based on the surrounding situation information generated by the vehicle outside detection devices 700C and 700D, and provide the generated notification information to the user. In the following description, when the vehicle exterior detection devices 700C and 700D are not distinguished, they are referred to as the vehicle exterior detection device 700.

[Vehicle configuration]
FIG. 2 is a configuration diagram of the first vehicle 10A according to the embodiment. Since the second vehicle 10B also has the same configuration as the first vehicle 10A described below, the description of the configuration of the second vehicle 10B will be omitted.

The first vehicle 10A includes, for example, a camera 11, a radar device 12, a finder 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, and a navigation device 50. , MPU (Map Positioning Unit) 60, a driving operator 80, an automatic driving control device 100, a traveling driving force output device 200, a braking device 210, a steering device 220, and an in-vehicle notification device 300. These devices and devices are connected to each other by multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, and the like. The configuration shown in FIG. 2 is merely an example, and a part of the configuration may be omitted or another configuration may be added. The camera 11, the radar device 12, and the finder 14 are devices for detecting the situation around the vehicle 10, and are examples of the vehicle-mounted detection devices mounted on the vehicle 10.

The camera 11 is, for example, a digital camera that uses a solid-state image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 11 is attached to an arbitrary position on the first vehicle 10A. When photographing the front, the camera 11 is attached to the upper part of the front windshield, the back surface of the rearview mirror, and the like. The camera 11 periodically and repeatedly images the periphery of the first vehicle 10A, for example. The camera 11 may be a stereo camera. The camera 11 may be a drive recorder.

The radar device 12 radiates radio waves such as millimeter waves around the first vehicle 10A, and detects radio waves (reflected waves) reflected by the object to detect at least the position (distance and orientation) of the object. The radar device 12 is attached to an arbitrary position on the first vehicle 10A. The radar device 12 may detect the position and velocity of the object by the FM-CW (Frequency Modulated Continuous Wave) method.

The finder 14 is a LIDAR (Light Detection and Ranging). The finder 14 irradiates the periphery of the first vehicle 10A with light and measures the scattered light. The finder 14 detects the distance to the target based on the time from light emission to light reception. The light to be irradiated is, for example, a pulsed laser beam. The finder 14 is attached to an arbitrary position on the first vehicle 10A.

The object recognition device 16 performs sensor fusion processing on the detection results of a part or all of the camera 11, the radar device 12, and the finder 14, and recognizes the position, type, speed, and the like of the object. The object recognition device 16 outputs the recognition result to the automatic operation control device 100. The object recognition device 16 may output the detection results of the camera 11, the radar device 12, and the finder 14 to the automatic driving control device 100 as they are. The object recognition device 16 may be omitted.

The communication device 20 communicates with another vehicle existing in the vicinity of the first vehicle 10A by using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or the like. Communicates with various server devices via a wireless base station.

The HMI 30 presents various information to the occupants of the first vehicle 10A and accepts input operations by the occupants. The HMI 30 includes various display devices, speakers, buzzers, touch panels, switches, keys and the like.

The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the first vehicle 10A, an acceleration sensor that detects acceleration, a yaw rate sensor that detects an angular velocity around the vertical axis, an orientation sensor that detects the direction of the first vehicle 10A, and the like.

The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a routing unit 53. The navigation device 50 holds the first map information 54 in a storage device such as an HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 identifies the position of the first vehicle 10A based on the signal received from the GNSS satellite. The position of the first vehicle 10A may be specified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or wholly shared with the above-mentioned HMI 30. The route determination unit 53 is, for example, a route from the position of the first vehicle 10A (or an arbitrary position input) specified by the GNSS receiver 51 to the destination input by the occupant using the navigation HMI 52 (hereinafter,). , The route on the map) is determined with reference to the first map information 54. The first map information 54 is, for example, information in which a road shape is expressed by a link indicating a road and a node connected by the link. The first map information 54 may include road curvature, POI (Point Of Interest) information, and the like. The route on the map is output to MPU60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized by, for example, the function of a terminal device such as a smartphone or a tablet terminal owned by an occupant. The navigation device 50 may transmit the current position and the destination to the navigation server via the communication device 20 and acquire a route equivalent to the route on the map from the navigation server.

The MPU 60 includes, for example, a recommended lane determination unit 61, and holds the second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a plurality of blocks (for example, divides the route every 100 [m] with respect to the vehicle traveling direction), and refers to the second map information 62. Determine the recommended lane for each block. The recommended lane determination unit 61 determines which lane to drive from the left. When the route on the map has a branch point, the recommended lane determination unit 61 determines the recommended lane so that the first vehicle 10A can travel on a reasonable route to proceed to the branch destination.

The second map information 62 is more accurate map information than the first map information 54. The second map information 62 includes, for example, information on the center of the lane, information on the boundary of the lane, and the like. Further, the second map information 62 may include road information, traffic regulation information, address information (address / zip code), facility information, telephone number information, and the like. The second map information 62 may be updated at any time by the communication device 20 communicating with another device.

The driving controller 80 includes, for example, an accelerator pedal, a brake pedal, a shift lever, a steering wheel, a deformed steering wheel, a joystick, and other controls. A sensor for detecting the amount of operation or the presence or absence of operation is attached to the operation operator 80, and the detection result is the automatic operation control device 100, or the traveling driving force output device 200, the brake device 210, and the steering device. It is output to a part or all of 220.

The automatic operation control device 100 includes, for example, a first control unit 120 and a second control unit 160. Each of the first control unit 120 and the second control unit 160 is realized by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). In addition, some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). It may be realized by the part (including circuitry), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device including a non-transient storage medium) such as an HDD or a flash memory of the automatic operation control device 100, or is removable such as a DVD or a CD-ROM. It is stored in a storage medium, and may be installed in the HDD or flash memory of the automatic operation control device 100 by mounting the storage medium (non-transient storage medium) in the drive device.

The traveling driving force output device 200 outputs a traveling driving force (torque) for traveling the vehicle to the drive wheels. The traveling driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and the like, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to the information input from the second control unit 160 or the information input from the operation operator 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits flood pressure to the brake caliper, an electric motor that generates flood pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to the information input from the second control unit 160 or the information input from the operation operator 80 so that the brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include, as a backup, a mechanism for transmitting the oil pressure generated by the operation of the brake pedal included in the operation operator 80 to the cylinder via the master cylinder. The brake device 210 is not limited to the configuration described above, and is an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 to transmit the oil pressure of the master cylinder to the cylinder. May be good.

The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies a force to the rack and pinion mechanism to change the direction of the steering wheel. The steering ECU drives the electric motor according to the information input from the second control unit 160 or the information input from the operation controller 80, and changes the direction of the steering wheel.

FIG. 3 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130 and an action plan generation unit 140. The first control unit 120, for example, realizes a function by AI (Artificial Intelligence) and a function by a model given in advance in parallel. For example, the function of "recognizing an intersection" is executed in parallel with recognition of an intersection by deep learning or the like and recognition based on predetermined conditions (pattern matching signals, road markings, etc.), both of which are executed. It may be realized by scoring against and comprehensively evaluating. This ensures the reliability of autonomous driving.

The recognition unit 130 is based on the information input from the camera 11, the radar device 12, and the finder 14 via the object recognition device 16, the position, speed, acceleration, and other states of the objects around the first vehicle 10A. Recognize. The position of the object is recognized as, for example, a position on absolute coordinates with a representative point (center of gravity, center of drive axis, etc.) of the first vehicle 10A as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of an object may include acceleration or jerk of the object, or "behavioral state" (eg, whether or not the vehicle is changing lanes or is about to change lanes).

In principle, the action plan generation unit 140 travels in the recommended lane determined by the recommended lane determination unit 61, and the first vehicle 10A automatically (1st vehicle 10A) so as to be able to respond to the surrounding conditions of the first vehicle 10A. Generate a target trajectory to drive in the future (without relying on the driver's operation). The target trajectory includes, for example, a velocity element. For example, the target track is expressed as a series of points (track points) to be reached by the first vehicle 10A. The track point is a point to be reached by the first vehicle 10A for each predetermined mileage (for example, about several [m]) along the road, and separately, a predetermined sampling time (for example, 0 comma number [sec]]. A target velocity and a target acceleration for each degree) are generated as part of the target trajectory. Further, the track point may be a position to be reached by the first vehicle 10A at the sampling time for each predetermined sampling time. In this case, the information of the target velocity and the target acceleration is expressed by the interval of the orbital points.

The action plan generation unit 140 may set an event for automatic driving when generating a target trajectory. Autonomous driving events include constant speed driving events, low speed following driving events, lane change events, branching events, merging events, takeover events, and the like. The action plan generation unit 140 generates a target trajectory according to the activated event.

The second control unit 160 has a traveling driving force output device 200, a brake device 210, and a steering device 220 so that the first vehicle 10A passes the target trajectory generated by the action plan generation unit 140 at the scheduled time. To control.

The second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires the information of the target trajectory (orbit point) generated by the action plan generation unit 140 and stores it in a memory (not shown). The speed control unit 164 controls the traveling driving force output device 200 or the braking device 210 based on the speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the degree of bending of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized by, for example, a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road in front of the first vehicle 10A and feedback control based on the deviation from the target trajectory.

[In-vehicle notification device]
First, a part of the agent system on the in-vehicle notification device 300 side will be described. FIG. 4 is a diagram showing the configuration of the in-vehicle notification device 300 and the equipment mounted on the first vehicle 10A. For example, the microphone 410, the display / operation device 420, and the speaker 430 are further mounted on the first vehicle 10A. The in-vehicle notification device 300 and the user terminal 70A may be connected to each other by, for example, a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. As a result, when the user terminal 70A is brought into the vehicle interior, the user terminal 70A can output the notification information received from the information providing device 500 via the in-vehicle notification device 300.

The microphone 410 is a sound collecting unit that collects the sound emitted in the vehicle interior. There may be multiple microphones 410 to capture the utterances of multiple occupants of the vehicle. The display / operation device 420 is a device (or device group) capable of displaying an image and accepting an input operation. The display / operation device 420 includes, for example, a display device configured as a touch panel. The display / operation device 420 may further include a HUD (Head Up Display), a mechanical input device, and an output device. The speaker 430 includes, for example, a plurality of speakers (sound output units) arranged at different positions in the vehicle interior. The display / operation device 420 may be shared by the in-vehicle notification device 300 and the navigation device 50. Further, the in-vehicle notification device 300 may be constructed based on the navigation controller, and in that case, the navigation controller and the in-vehicle notification device 300 are integrally configured on the hardware.

The in-vehicle notification device 300 includes a management unit 310, a notification agent function unit 320, and an in-vehicle storage unit 330. The management unit 310 includes, for example, an acoustic processing unit 311, an instruction receiving unit 312, a display control unit 313, and a voice control unit 314. Each component of the in-vehicle notification device 300 is realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (including circuit part; circuitry), or realized by collaboration between software and hardware. May be good. The program may be stored in advance in a storage device such as an HDD or a flash memory (a storage device including a non-transient storage medium), or a removable storage medium (non-transient) such as a DVD or a CD-ROM. It is stored in a sex storage medium) and may be installed by attaching the storage medium to a drive device.

The management unit 310 functions by executing a program such as an OS (Operating System) or middleware. The in-vehicle storage unit 330 is realized by the above-mentioned various storage devices. For example, the information input from the object recognition device 16 is temporarily stored in the vehicle interior storage unit 330.

The notification agent function unit 320 causes an agent to appear in cooperation with the information providing device 500, and provides a notification agent function including information provision by voice. The notification agent function unit 320 may provide a notification agent function including a voice response in response to an utterance of a vehicle occupant. The notification agent function unit 320 may be given the authority to control the vehicle equipment.

The notification agent function unit 320 generates peripheral situation information based on the information input from the camera 11, the radar device 12, and the finder 14 via the object recognition device 16. Further, the notification agent function unit 320 may generate peripheral situation information based on the recognition result by the object recognition device 16. The notification agent function unit 320 transmits the generated peripheral status information to the information providing device 500 via the communication device 20. When the notification agent function unit 320 receives the notification information from the information providing device 500, the notification agent function unit 320 executes various processes for notifying the user A based on the received notification information.

Further, the notification agent function unit 320 temporarily stores the input information in the in-vehicle storage unit 330 when information is input from the camera 11 via the object recognition device 16, and when a certain time elapses, the in-vehicle storage unit 320 stores the input information. The information stored in 330 is erased. The information stored in the in-vehicle storage unit 330 includes image data or moving image data captured by the camera 11. The time when the image was taken is associated with the image data and the moving image data.

The notification agent function unit 320 recognizes the meaning of the voice from the voice (voice stream) on which the sound processing is performed. First, the notification agent function unit 320 detects a voice section based on the amplitude and zero intersection of the voice waveform in the voice stream. The notification agent function unit 320 may perform frame-by-frame speech recognition based on a mixture Gaussian mixture model (GMM) and section detection based on non-speech recognition. The notification agent function unit 320 recognizes a wakeup word or the like predetermined for each agent.

The sound processing unit 311 performs sound processing on the input sound so as to be in a state suitable for recognizing the utterance of the occupant. The notification agent function unit 320 recognizes, for example, a wakeup word or a request word from another occupant based on the voice stream processed by the sound processing unit 311. The wakeup word includes, for example, "start notification service". Request words include, for example, "tell me again", "speak louder", "where do you see it?", "Send an image of the notification to your cell phone" and so on.

The instruction receiving unit 312 receives an instruction from the occupant using the display / operating device 420. Not limited to this, the instruction receiving unit 312 may receive an instruction from the occupant by having a voice recognition function and recognizing the meaning of the voice based on the voice in the vehicle. The voice in the vehicle includes a sound input from the microphone 410, a voice processed by the sound processing unit 311 (voice stream), and the like.

The display control unit 313 causes the display of the display / operation device 420 to display an image or a moving image in response to an instruction from the notification agent function unit 320. The voice control unit 314 causes a part or all of the speakers included in the speaker 430 to output voice in response to an instruction from the notification agent function unit 320.

An example of notification when notification information is acquired by the notification agent function unit 320 will be described. For example, the notification agent function unit 320 instructs the display control unit 313 to display a text or an image based on the notification information on the display / operation device 420, and also outputs a voice based on the notification information from the speaker 430. Instruct the control unit 314. When notification to the user terminal 70A is set in advance by the user A, the notification agent function unit 320 transmits the notification information to the user terminal 70A via the communication device 20 to notify the user terminal 70A. Images and sounds based on the notification information may be output from the display unit and the speaker.

The notification agent function unit 320 may determine the mode of notifying the user A according to the notification information. For example, the notification agent function unit 320 emphasizes the output mode in the case of notification information having a high notification level as compared with the case of notification information having a low notification level. Emphasizing the output mode includes, for example, not only outputting sound from the speaker 430 but also causing the display / operation device 420 to output an image. By doing so, notification information with a high notification level (for example, a phenomenon with a considerably low probability of appearance such as a shooting star or a meteorite falling) and notification information with a low notification level (for example, a phenomenon such as a guerrilla rainstorm or the appearance of unusually shaped clouds) The occupants can be notified with more emphasis. Note that this output mode may be specified by the information providing device 500 for each notification information.

[Configuration of information providing device]
Next, a part of the agent system on the information providing device 500 side will be described. FIG. 5 is a diagram showing a configuration of the in-vehicle notification device 300 and a part of the configuration of the information providing device 500. Hereinafter, the operation of the in-vehicle notification device 300 and the like will be described together with the configuration of the information providing device 500. Here, the description of the physical communication from the in-vehicle notification device 300 to the network NW will be omitted.

The information providing device 500 includes, for example, a communication unit 510, a first processing unit 520, a second processing unit 530, a first storage unit 550, and a second storage unit 560. The communication unit 510 is a network interface such as a NIC (Network Interface Card). The components of the first processing unit 520 and the second processing unit 530 are realized by, for example, a hardware processor such as a CPU executing a program (software). Some or all of these components may be realized by hardware such as LSI, ASIC, FPGA, GPU (including circuit part; circuitry), or realized by collaboration between software and hardware. May be good. The program may be stored in advance in a storage device such as an HDD or a flash memory (a storage device including a non-transient storage medium), or a removable storage medium (non-transient) such as a DVD or a CD-ROM. It is stored in a sex storage medium) and may be installed by attaching the storage medium to a drive device.

The first processing unit 520 cooperates with the in-vehicle notification device 300 to execute each processing for realizing the agent system. The first processing unit 520 includes, for example, a voice recognition unit 521, a natural language processing unit 522, a dialogue management unit 523, and a response sentence generation unit 524.

The second processing unit 530 executes each process for generating notification information and transmitting it to the in-vehicle notification device 300. The second processing unit 530 includes, for example, an event recognition unit 531, a notification unit 532, a user confirmation unit 533, a list editing unit 534, a learning unit 535, a data acquisition unit 536, an operation instruction unit 537, and data. It includes a management unit 538 and another vehicle detection unit 539.

The first storage unit 550 and the second storage unit 560 are realized by the above-mentioned various storage devices. Data and programs such as a dictionary DB (database) 551 and a response rule DB 552 are stored in the first storage unit 550. This information is read out by the first processing unit 520. In the second storage unit 560, for example, data such as a daily event list 561, an extraordinary event list 562, a user-specific list 563, a user event identification model 564, a vehicle position information 565, and surrounding situation information 566 can be stored. The program is stored. These pieces of information are written by the second processing unit 530 and read out by the second processing unit 530.

First, the processing by the first processing unit 520 will be described. For example, in the in-vehicle notification device 300, the notification agent function unit 320 transmits a voice stream or a voice stream that has undergone processing such as compression or coding to the information providing device 500.

When the voice stream is acquired, the voice recognition unit 521 performs voice recognition and outputs textual character information, and the natural language processing unit 522 interprets the meaning of the character information while referring to the dictionary DB 551. The dictionary DB 551 is associated with abstract semantic information with respect to character information. The dictionary DB 551 may include list information of synonyms and synonyms. The processing of the voice recognition unit 521 and the processing of the natural language processing unit 522 are not clearly separated in stages, and the voice recognition unit 521 corrects the recognition result in response to the processing result of the natural language processing unit 522. It may be done by influencing each other.

For example, when the meanings such as "where do you see?" And "where?" Are recognized as the recognition result, the natural language processing unit 522 generates a command replaced with the standard character information "where". As a result, even if there is a character fluctuation in the voice of the request, it is possible to facilitate the dialogue according to the request. Further, the natural language processing unit 522 may recognize the meaning of the character information by using artificial intelligence processing such as machine learning processing using probability, or may generate a command based on the recognition result.

The dialogue management unit 523 determines the content of the utterance to the occupant of the vehicle based on the processing result (command) of the natural language processing unit 522 with reference to the response rule DB 552. The response rule DB 552 is information that defines an operation (answer, device control content, etc.) that the agent should perform in response to the command.

The response sentence generation unit 524 generates a response sentence so that the content of the utterance determined by the dialogue management unit 523 is transmitted to the occupants of the vehicle, and transmits the response sentence to the in-vehicle notification device 300 via the communication unit 510. The response sentence generation unit 524 may call the occupant's name or generate a response sentence that resembles the occupant's speech when the occupant is identified as a registered occupant in the personal profile. Good.

When the notification agent function unit 320 acquires the response sentence, the notification agent function unit 320 instructs the voice control unit 314 to perform voice synthesis and output the voice. Further, the notification agent function unit 320 may instruct the display control unit 313 to display the AG animation in accordance with the voice output, or instruct the display control unit 313 to display the response sentence in characters. May be good. In this way, the agent function in which the virtually appearing agent responds to the occupants of the vehicle is realized.

Next, the processing by the second processing unit 530 will be described. For example, in the in-vehicle notification device 300, the notification agent function unit 320 transmits the generated peripheral situation information to the information providing device 500.

The event recognition unit 531 receives peripheral situation information from the first vehicle 10A via the communication unit 510. For example, the event recognition unit 531 uses the external detection unit (for example, the camera 11, the radar device 12, the finder 14, etc.) mounted on the first vehicle 10A to detect the detection result during traveling, and obtains peripheral situation information. Receive as. The event recognition unit 531 may receive the detection result detected by the vehicle outside detection device 700 as peripheral situation information.

For example, the event recognition unit 531, based on the surrounding situation information of the first vehicle 10A which receives, events occurring in the vicinity of the first vehicle 10A (hereinafter, referred to as occurred event E _A) recognizes. The peripheral situation information of the first vehicle 10A includes, for example, information output from the camera 11, the radar device 12, and the finder 14 of the first vehicle 10A. Event recognition unit 531 receives the peripheral status information from the outside detector 700C, based on the surrounding conditions information received, may recognize the occurrence event E _C occurring in the vicinity of the outside detector 700C. In the following description, mainly, an example will be described that recognizes the occurrence event E _A. The development events E _C, the "first vehicle 10" or In other read as "outside detector 700C". Occurrence event E _C will be explained for the part different from the occurrence event E _A.

The event recognition unit 531 detects a predetermined image included in the image by, for example, analyzing the image captured by the camera 11. Event recognition unit 531 determines the occurrence event E _A based on a predetermined image detected. The event recognition unit 531 may detect a predetermined image by pattern recognition, or may detect a predetermined image included in the image based on the feature amount extracted from the image. Certain images include, for example, rainbows, shooting stars, meteorite falls, guerrilla rainstorms, unusually shaped clouds, monkeys, raccoon dogs, and the like.

Event recognition unit 531, in addition to the image analysis, based on the information output from the radar device 12 and a finder 14, and derives the like movement and speed of the predetermined image may recognize the occurrence event E _A ..

In addition, the event recognition unit 531 recognizes that an event (hereinafter referred to as a first event) included in an unusual range has occurred in the area around the first vehicle 10A. For example, if the recognized developmental events E _A corresponds to an event that is included in the extraordinary event list 562, event recognizer 531 recognizes that the first event has occurred. The extraordinary event list 562 is a list of events included in the non-daily range. In the extraordinary event list 562, for example, non-daily events in the corresponding area are registered in advance for each corresponding area. The extraordinary event list 562 and other recognition methods will be described later.

Event recognition unit 531, if recognized developmental events E _A does not correspond to an event that is included in the daily events list 561, the event recognition unit 531 does not recognize the occurrence event E _A as the first event. The daily event list 561 is a list of events included in the daily range. The daily event list 561 and other recognition methods will be described later.

When the event recognition unit 531 recognizes that the first event has occurred, the notification unit 532 indicates the event occurring in the vicinity of the first vehicle 10A to the user A who is a occupant of the first vehicle 10A. Notify information about. Notification unit 532 generates, for example, the notification information expressed by sound or image that recognized generated event E _A first event is occurring. The notification unit 532 transmits the generated notification information to the first vehicle 10A or the user terminal 70A via the communication unit 510.

Incidentally, if the first event based on the occurred event E _C by event recognition unit 531 is recognized to have occurred, the notification unit 532 derives the position occurred event E _C has occurred, derived based on the position where the occurred event E _C has occurred, to extract the vehicle 10 you can see the occurred event E _C from the passenger compartment. Notification unit 532, as information indicating the position where the occurred event E _C has occurred, deriving a latitude, longitude, vertical height. Notification unit 532, for example, based on the geographic information of an obstacle located on a straight line connecting height and buildings around the vehicle 10 and the position of the vehicle 10 to generate event E _C has occurred, The vehicle 10 that can be seen from the passenger compartment may be extracted. The geographic information may be stored in an external server or may be stored in the second storage unit 560. Notification unit 532 notifies the occurrence event _{E C} with respect to the occupant of the extracted vehicle 10. Hereinafter, for the occurred event E _C, a counterpart to notify the "occupant of the first vehicle 10" or In other read as "extracted vehicle occupant".

In addition, the event recognition unit 531 has an event (hereinafter referred to as a second event) that is included in the daily range in the area around the first vehicle 10A but is not included in the daily range for the occupants of the first vehicle. Recognize that you are doing. For example, recognized the occurred event E _A is corresponds to an event that is included in the daily events list 561, and, if applicable to events included in the user listwise 563, the event recognition unit 531, a second event occurs Recognize that.

When the event recognition unit 531 recognizes that the second event has occurred, the notification unit 532 indicates the event occurring in the vicinity of the first vehicle 10A to the user A who is a occupant of the first vehicle 10A. Notify information about. Notification unit 532 generates, for example, the notification information expressed by sound or image that recognized generated event E _A and the second event occurs. The notification unit 532 transmits the generated notification information to the first vehicle 10A or the user terminal 70A via the communication unit 510.

Also, the event recognition unit 531, if recognized developmental events E _A corresponds to an event that is included in the user listwise 563 determines that for the passenger in the first vehicle 10A, an event that is included in the range that is not routine. The user-by-user list 563 includes, for example, an event set as an extraordinary range by the user and an event set as a notification target by the user. The per-user list 563 may include, for example, events that overlap part of the extraordinary event list 562.

Also, the event recognition unit 531, the determination together with the recognized event occurrence E _A to determine whether true extraordinary event list 562, the recognized developmental events E _A is whether corresponds to everyday events list 561 You may.

User confirmation section 533 recognized the occurred event E _A does not correspond to the extraordinary event list 562, and, if recognized developmental events E _A does not correspond to a daily event list 561 is the occupant of the first vehicle 10A Confirm the presence or absence of notification to user A. For example, the user confirmation unit 533 outputs a voice "Do you want to be notified of an event recognized around the vehicle?" From the microphone 410 of the first vehicle 10A. Not limited to this, the user confirmation unit 533 may display a message to the same effect on the user terminal 70A of the user A or output a voice. The user confirmation unit 533 receives a response from the user to the confirmation to the user based on the voice stream received via the communication unit 510 and the instruction information from the user received by the display / operation device 420.

Notification unit 532, the confirmation by the user confirmation unit 533, if it notifies the event recognized around the vehicle is instructed by the user A, produced event E _A does not correspond to the extraordinary event list 562, and, daily even if that does not correspond to the event list 561, notifying that the occurred event E _a is occurring for the user a.

List editing unit 534, if it notifies the event recognized around the vehicle is instructed by the user A, the recognized generated event E _A, is added to the user for each list 563. Further, the list editing unit 534 may add or delete the event instructed by the user A to the list 563 for each user, even if the event is not the recognized event.

The list editing unit 534 edits the extraordinary event list 562 based on the instructions given by the occupants of the first vehicle 10A. For example, the list editorial unit 534 adds an event that the user wants to be notified to the extraordinary event list 562, even if the event is generally included in the daily range. The list editorial unit 534 may delete an event that the user does not want to be notified from the extraordinary event list 562, even if the event is generally included in a non-daily range.

The list editing unit 534 may edit the daily event list 561 based on the instructions given by the occupants of the first vehicle 10A. For example, the list editorial unit 534 adds an event that the user does not want to be notified to the daily event list 561 even if the event is generally included in a range other than daily. The list editorial unit 534 deletes an event that the user wants to be notified from the daily event list 561 even if the event is generally included in the daily range.

Learning unit 535, if it notifies the event recognized around the vehicle is instructed by the occupant of the first vehicle 10A, event recognized around the vehicle (i.e., generating an event E _A) based on the user event identification model 564 is learned for each occupant of the first vehicle 10A. For example, the learning unit 535, the information based on the generated event E _A as teacher data, to learn the user event identification model 564. Various known methods may be used for this learning process. The user event identification model 564 is a model that identifies for each user whether or not the input event is an event to be notified to the user. The user event identification model 564 may include a plurality of models prepared for each user.

Learning unit 535 does not match the event recognized developmental events E _A is included in the extraordinary event list 562, and, recognized developmental events E _A is a case that does not match the event included in the daily events list 561 , if it is notified of the occurrence event E _a is instructed by the user, based on the occurred event E _a, a user event identification model 564, it may be learned every passenger in the first vehicle 10A.

Event recognition unit 531, using the user event identification model 564, recognized the occurred event E _A may identify whether an event to notify the user A. For example, when the occurrence event E _A input to the user event identification model 564 is identified as an event to be notified to the user A, the notification unit 532 informs the user A that the occurrence event E _A has occurred. Notify.

Data acquisition unit 536 generates generation event data for generating an event E _A that is notified to the occupant by the notification unit 532. The generated event data, including image data or moving image data obtained by capturing the occurrence event E _A. For example, the data acquisition unit 536, if the event recognition unit 531 that is notified event (including the event to notify the first event and the user A) is recognized, it occurred event E _A based on the surrounding situation information generation Specify the time (hereinafter referred to as the occurrence time). The data acquisition unit 536 requests the in-vehicle notification device 300 via the communication unit 510 to transmit the image data and the moving image data captured in a certain period including the specified occurrence time. Upon receiving the request, the in-vehicle notification device 300 transmits image data or moving image data for a predetermined time including the occurrence time from the in-vehicle storage unit 330 to the information providing device 500. The data acquisition unit 536 generates event data based on the image data or the moving image data received from the in-vehicle notification device 300. The occurrence event data generated by the data acquisition unit 536 may be transmitted by the notification unit 532 to the occupant to whom the notification information has been notified. The notification unit 532, instead of the notification information, the event occurrence data by transmitting the first vehicle 10A or the user terminal 70A, the occurred event E _A may notify the occupant. For example, when the occupant is sleeping or the baby is on board, the comfort in the vehicle interior can be improved by displaying the image data without notifying the notification information by voice.

When the event recognition unit 531 recognizes that the notification event (including the first event and the event notified to the user A), the driving instruction unit 537 instructs the first vehicle 10A to drive slowly. At this time, the driving instruction unit 537 may determine the traveling speed, the traveling distance, and the like for slow driving, and instruct the first vehicle 10A. Upon receiving the instruction, the first vehicle 10A executes slow driving by automatic driving. This makes it easier for the occupant to check the events around the vehicle.

The data management unit 538 stores the information received from the vehicle 10 and the out-of-vehicle detection device 700 via the communication unit 510 in the second storage unit 560. For example, the data management unit 538 stores the information in which the current position information received from the vehicle 10 and the vehicle ID are associated with each other in the second storage unit 560 as a part of the vehicle position information 565. The data management unit 538 stores the peripheral situation information received from the vehicle 10 and the outside detection device 700 in the second storage unit 560 as a part of the peripheral situation information 566. The data management unit 538 may store the image data, the moving image data, and the generated occurrence event data received from the vehicle 10 or the vehicle outside detection device 700 in the second storage unit 560 as a part of the peripheral situation information 566.

The other vehicle detection unit 539 refers to the vehicle position information 565 and detects another vehicle that transmits the notification information. For example, when it is recognized that a notification event has occurred around the first vehicle 10A, another vehicle existing in a predetermined area including the first vehicle 10A is detected. Notification unit 532 notifies the other vehicle detected by another vehicle detection unit 539, the occurred event E _A in the vicinity of the other vehicles has occurred. Other vehicle detecting portion 539, similarly to the processing of the notification unit 532 described above, derives the position occurred event E _A has occurred, the derived occurred event E _A sees position occurring from the vehicle interior Extract other vehicles that can. By doing so, the recognized generated event E _A by the first vehicle 10A, can be also notifies to the other vehicle traveling on the periphery.

[Daily event list and extraordinary event list]
Hereinafter, an example of the daily event list 561 and the extraordinary event list 562 will be described.

The daily event list 561 includes, for example, an event in which it snows in a snowy country, an event in which monkeys appear in sunlight, an event in which a raccoon dog appears in the countryside, and the like.

The extraordinary event list 562 includes, for example, the first phenomenon (eg, rainbow, shooting star, UFO, meteorite fall, etc.) with a very low probability of occurrence regardless of region. This first phenomenon is an example of notification information having a higher notification level than the second and third phenomena described later.

The extraordinary event list 562 may include a second phenomenon (for example, guerrilla rainstorm) that has a higher probability of appearing than the first phenomenon but is rare regardless of the region. The unusual second phenomenon includes, for example, a phenomenon in which the appearance mode does not correspond to the normal mode (for example, the appearance of an unusually shaped cloud, the appearance of an unusually colored moon, etc.).

The extraordinary event list 562, for example, occurred event E _A appearance probability is low third phenomenon in the region that have occurred (for example, snow in the snow is not too much rain the region, such as the thunder out of season) may be included. This third phenomenon occurred event E _A frequency lower animals in areas occurs (e.g., raccoon like in the city) may include such phenomena appear.

This third phenomenon includes a phenomenon in which the elapsed time from the time of the previous appearance at the current position of the first vehicle 10A exceeds the threshold value (for example, rain for the first time in several months, snow for the first time in several months, and snow for the first time in several months). Thunder, etc.) may be included. The notification unit 532 excludes the same event as the event notified to the occupant from the notification target when it occurs again within a predetermined period from the previous notification. For example, when the event recognition unit 531 recognizes that the third phenomenon has occurred, the notification unit 532 informs the occupant when a predetermined time has elapsed from the previous date and time when the same phenomenon occurred. You may want to be notified. That is, if the predetermined time has not passed since the previous occurrence date and time, the notification unit 532 may not notify the occupant. Further, when the same phenomenon occurs in the same season of the same year, the notification unit 532 may notify the occupant only for the first time and not for the second and subsequent times.

The extraordinary event list 562 may include, for example, an event in which a celebrity is likely to appear (such as an event in which a celebrity appears or a shooting location is being carried out). Celebrities include, for example, famous people regardless of region (nationally well-known talents, members of parliament, etc.) and people famous in the area (mayor, city council members, local idols, etc.).

The extraordinary event list 562 may include, for example, an event in which a landscape different from the usual appearance of the town (for example, a lantern, a banner, a musical accompaniment, a town broadcast, etc.) appears. The extraordinary event list 562 may include events in which topical items (recently opened topical facilities, popular ramen shops, popular bakeries, etc.) appear. The extraordinary event list 562 may include a phenomenon in which seasonal events (for example, cherry blossoms, autumn leaves, etc.) that are rare in other seasons appear.

[Processing flow]
Next, an example of the flow of processing executed by the information providing device 500 of the embodiment will be described. Here, an example in which the first vehicle 10A transmits the peripheral situation information around the first vehicle 10A to the information providing device 500 will be described.

[Processing flow (1)]
FIG. 6 is a flowchart showing an example of the process (No. 1) executed by the information providing device 500. First, the event recognition unit 531 acquires the peripheral status information (step S101), based on the acquired peripheral status information, recognizes the occurred event _{E A} (step S103). Event recognition unit 531 collates the generated event _{E A} and extraordinary event list 562 (step S105). If recognized developmental events _{E A} matches the events included in the extraordinary event list 562 (step S107), the event recognition unit 531 recognizes that the first event has occurred (step S109). Then, the notification unit 532 notifies the user A, who is an occupant of the first vehicle 10A, of an event occurring in the vicinity of the first vehicle 10A (step S111). Then, the data acquisition unit 536 generates generation event data for generating an event _{E A} (step S113). The notification unit 532 may notify the generated event data to the user A who is an occupant of the first vehicle 10A.

In step S107, recognized developmental events _{E A} may not match the events included in the extraordinary event list 562, event recognizer 531 compares the generated event _{E A} and user listwise 563 (step S115). If recognized developmental events _{E A} matches the events included in the user listwise 563 (step S117), the process proceeds to step S111.

[Processing flow (2)]
FIG. 7 is a flowchart showing an example of the process (No. 2) executed by the information providing device 500. First, the event recognition unit 531 acquires the peripheral status information (step S201), based on the acquired peripheral status information, recognizes the occurred event _{E A} (step S203). Event recognition unit 531 collates the generated event _{E A} and everyday events list 561 (step S205). If recognized developmental events _{E A} does not match the event included in the daily events list 561 (step S207), the event recognition unit 531 recognizes that the first event has occurred (step S209). Then, the notification unit 532 notifies the user A who is an occupant of the first vehicle 10A of an event occurring in the vicinity of the first vehicle 10A (step S211). Then, the data acquisition unit 536 generates generation event data for generating an event _{E A} (step S213). The notification unit 532 may notify the generated event data to the user A who is an occupant of the first vehicle 10A.

In step S207, if the recognized developmental events _{E A} matches the events included in the daily events list 561, the event recognition unit 531 collates the event occurrence _{E A} and user listwise 563 (step S215). If recognized developmental events _{E A} matches the events included in the user listwise 563 (step S217), the process proceeds to step S211.

[Processing flow (3)]
FIG. 8 is a flowchart showing an example of the process (No. 3) executed by the information providing device 500. First, the event recognition unit 531 acquires the peripheral status information (step S301), based on the acquired peripheral status information, recognizes the occurred event _{E A} (step S303). Event recognition unit 531 collates the generated event _{E A} and extraordinary event list 562 (step S305). If recognized developmental events _{E A} matches the events included in the extraordinary event list 562 (step S307), the event recognition unit 531 recognizes that the first event has occurred (step S309). Then, the notification unit 532 notifies the user A, who is an occupant of the first vehicle 10A, of an event occurring in the vicinity of the first vehicle 10A (step S311). Then, the data acquisition unit 536 generates generation event data for generating an event _{E A} (step S313). The notification unit 532 may notify the generated event data to the user A who is an occupant of the first vehicle 10A.

In step S307, recognized developmental events _{E A} may not match the events included in the extraordinary event list 562, event recognizer 531 compares the generated event _{E A} and everyday events list 561 (step S315). When the recognized event E _A does not match the event included in the daily event list 561 (step S317), the user confirmation unit 533 confirms the presence / absence of notification to the user A who is the occupant of the first vehicle 10A (step S317). Step S319). Then, the user confirmation unit 533 determines whether or not the user A has instructed to notify the confirmed event (step S321), and if instructed to notify, the process proceeds to step S311. The determination result in step S321 may be used as teacher data for learning by the learning unit 535.

According to the information providing device 500 of the embodiment described above, a first event included in an unusual range occurs in the area around the first vehicle 10A based on the information acquired in the vicinity of the first vehicle 10A. It is provided with an event recognition unit 531 that recognizes that the event is occurring, and a notification unit 532 that notifies the occupants of the first vehicle 10A of information about the first event when it is recognized that the first event has occurred. As a result, it is possible to make the occupant aware of a predetermined event occurring in the vicinity of the vehicle.

The embodiment described above can be expressed as follows.
A storage device that stores programs and
With a hardware processor,
When the hardware processor executes a program stored in the storage device,
Based on the information acquired around the first vehicle, it is recognized that the first event included in the non-daily range is occurring in the area around the first vehicle.
When it is recognized that the first event has occurred, the occupants of the first vehicle are notified of the information regarding the first event.
An information provider that is configured to.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

For example, the configuration included in the information providing device 500 may be mounted on the vehicle 10. For example, when the vehicle 10 does not need to notify other vehicles (when notifying only its own vehicle), the vehicle 10 may notify the occupants of the vehicle 10 of the events recognized in the vicinity of the vehicle 10.

500 Information providing device 531 Event recognition unit 532 Notification unit 533 User confirmation unit 534 List editing unit 535 Learning unit 536 Data acquisition unit 537 Driving instruction unit 538 Data management unit 539 Other vehicle detection unit

Claims (15)

An event recognition unit that recognizes that a first event included in an unusual range has occurred in the area around the first vehicle based on the information acquired around the first vehicle.
When the event recognition unit recognizes that the first event has occurred, the notification unit that notifies the occupants of the first vehicle of information about the first event, and the notification unit.
An information providing device including. When the recognized event occurring in the vicinity of the first vehicle corresponds to a daily event list registered in advance as a daily event in the area around the first vehicle, the event recognition unit may use the event recognition unit. The event that is occurring is not recognized as the first event,
The information providing device according to claim 1. The event recognition unit further recognizes that a second event that is included in the daily range in the area around the first vehicle but is not included in the daily range for the occupants of the first vehicle is occurring. And
The notification unit notifies the occupants of the first vehicle of information regarding the second event even when the event recognition unit recognizes that the second event has occurred.
The information providing device according to claim 1 or 2. When the recognized event occurring in the vicinity of the first vehicle corresponds to an extraordinary event list registered in advance as an extraordinary event in the area around the first vehicle. Recognizes the occurring event as the first event,
The information providing device according to any one of claims 1 to 3. A list editing unit that edits the extraordinary event list based on the instructions given by the occupants of the first vehicle is further provided.
The information providing device according to claim 4. The event occurring in the vicinity of the first vehicle recognized by the event recognition unit does not correspond to the daily event list registered in advance as a daily event in the area around the first vehicle, and If it does not correspond to the extraordinary event list registered in advance as an unusual event in the area around the first vehicle, a confirmation unit for confirming the presence or absence of notification to the occupants of the first vehicle is further provided.
When instructed by the occupant of the first vehicle to notify, the notification unit notifies the occupant of the first vehicle of information on an event occurring in the vicinity of the first vehicle.
The information providing device according to any one of claims 1 to 5. A learning unit for learning an event instructed to be notified by the occupant of the first vehicle for each occupant of the first vehicle is further provided.
The information providing device according to claim 6. If the same event as the event notified to the occupant of the first vehicle occurs again within a predetermined period from the previous notification, the notification unit excludes it from the notification target.
The information providing device according to any one of claims 1 to 7. The event recognition unit recognizes an event occurring in the vicinity of the first vehicle based on the detection result detected during traveling by using the in-vehicle detection device mounted on the first vehicle.
The information providing device according to any one of claims 1 to 8. The event recognition unit is based on at least one of the peripheral situation detected by the in-vehicle detection device mounted on the first vehicle and the peripheral situation detected by the external detection device installed outside the vehicle. Recognizes that the event is occurring,
The information providing device according to any one of claims 1 to 9. When the event recognition unit recognizes that the event has occurred, a data acquisition unit that acquires data obtained by imaging the event using the image pickup unit mounted on the first vehicle is further added. Prepare, prepare
The information providing device according to any one of claims 1 to 10. The notification unit notifies the occupants of the second vehicle traveling in the area where the event is occurring to that effect.
The information providing device according to any one of claims 1 to 11. When the event recognition unit recognizes that the first event has occurred, it further includes a driving instruction unit that instructs the first vehicle to drive slowly.
The information providing device according to any one of claims 1 to 12. The computer
Based on the information acquired around the first vehicle, it is recognized that the first event included in the non-daily range is occurring in the area around the first vehicle.
When it is recognized that the first event has occurred, the occupants of the first vehicle are notified of the information regarding the first event.
Information provision method. On the computer
Based on the information acquired in the vicinity of the first vehicle, it is made to recognize that the first event included in an unusual range occurs in the area around the first vehicle.
When it is recognized that the first event has occurred, the occupants of the first vehicle are notified of the information regarding the first event.
program.

Images