JPWO2018127962A1 - Notification control apparatus and notification control method - Google Patents

Abstract

An object of the present invention is to provide a notification control device and a notification control method capable of performing appropriate notification to a driver. The notification control device according to the present invention obtains at least the peripheral information acquisition unit for acquiring the peripheral information indicating the peripheral situation of the host vehicle, and the peripheral information acquired by the peripheral information acquisition unit, the object or person existing around the host vehicle. A primary event recognition unit that recognizes a primary event that is a status, and a status that can occur after the primary event that is recognized by the primary event recognition unit, and the causal relationship with the status of the primary event A second event estimation unit that estimates a second event that is a situation according to a predetermined rule, and a notification control unit that performs control of reporting a second event estimated by the second event estimation unit; Prepare.

Description

  The present invention relates to a notification control device and a notification control method for notifying a driver of a situation around the host vehicle.

  2. Description of the Related Art Conventionally, there has been disclosed a technology for alerting a driver of a host vehicle by sensing an object or a person present around the host vehicle and emphasizing and displaying the detected object or person (for example, Patent Literature 1, 2, 3).

Unexamined-Japanese-Patent No. 11-115660 JP, 2008-62762, A JP, 2008-310376, A

  For example, when a ball is present in front of a traveling vehicle, it is conceivable that a child then pops out in front of the vehicle to pick up the ball. If the situation that may occur after such an actual situation is informed to the driver in addition to the actual situation, the driver can take an appropriate response in consideration of the situation that may occur in the future.

  In Patent Documents 1, 2, and 3, only the actual situation is notified to the driver, and the situation that may occur after the actual situation is not notified to the driver. Therefore, in Patent Documents 1, 2, and 3, it can not be said that the driver is appropriately notified.

  The present invention has been made to solve such a problem, and an object of the present invention is to provide a notification control apparatus and a notification control method capable of performing appropriate notification to a driver.

  In order to solve the above-described problems, the notification control device according to the present invention acquires at least the peripheral information acquisition unit that acquires peripheral information indicating the peripheral conditions of the host vehicle and the peripheral information acquired by the peripheral information acquisition unit. Primary event recognition unit that recognizes a primary event that is the status of an object or person present in the vicinity of the scene, and a status that can occur after the primary event recognized by the primary event recognition unit, The second event estimation unit estimates the second event which is a condition having a causal relationship with the situation of the primary event according to a predetermined rule, and broadcasts the second event estimated by the second event estimation unit And a notification control unit that performs control.

  In addition, the notification control method according to the present invention acquires at least peripheral information indicating the state around the host vehicle, and from the acquired peripheral information, a primary event that is a state of an object or person existing around the host vehicle A second event which is a situation that may occur after a recognized first event and is a situation having a causal relationship with the situation of the first event is estimated according to a predetermined rule, Control to notify secondary events.

  According to the present invention, the notification control device acquires an object existing in the periphery of the vehicle from the periphery information acquisition unit acquiring the periphery information indicating at least the situation around the vehicle and the periphery information acquired by the periphery information acquisition unit A primary event recognition unit that recognizes a primary event that is the condition of a person, and a situation that can occur after the primary event recognized by the primary event recognition unit, and the condition and causality of the primary event A secondary event estimation unit that estimates secondary events that are related situations according to a predetermined rule, and a notification control unit that performs control to notify of secondary events estimated by the secondary event estimation unit And the driver can be appropriately notified.

  In addition, the notification control method acquires peripheral information indicating at least the peripheral situation of the vehicle, and recognizes from the acquired peripheral information a primary event that is the condition of an object or a person present in the periphery of the vehicle, A secondary event that is a situation that may occur after a recognized primary event and that is a situation that has a causal relationship with the situation of the primary event, estimated by a predetermined rule, and estimated secondary event In order to perform control to notify the driver, it is possible to perform appropriate notification to the driver.

  The objects, features, aspects, and advantages of the present invention will be more apparent from the following detailed description and the accompanying drawings.

It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a figure which shows an example of the primary event according to Embodiment 1 of this invention, and a secondary event. It is a block diagram which shows an example of the hardware constitutions of the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a flowchart which shows an example of operation | movement of the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a figure which shows an example of the display in the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a figure which shows an example of the display in the alerting | reporting control apparatus by Embodiment 1 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 2 of this invention. It is a flowchart which shows an example of operation | movement of the alerting | reporting control apparatus by Embodiment 2 of this invention. It is a figure which shows an example of the display in the alerting | reporting control apparatus by Embodiment 2 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 3 of this invention. It is a flowchart which shows an example of operation | movement of the alerting | reporting control apparatus by Embodiment 3 of this invention. It is a figure for demonstrating the recognition range in the alerting | reporting control apparatus by Embodiment 3 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 4 of this invention. It is a flowchart which shows an example of operation | movement of the alerting | reporting control apparatus by Embodiment 4 of this invention. It is a flowchart which shows an example of operation | movement of the external server by Embodiment 4 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control apparatus by Embodiment 5 of this invention. It is a flowchart which shows an example of operation | movement of the alerting | reporting control apparatus by Embodiment 5 of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control system by embodiment of this invention. It is a block diagram which shows an example of a structure of the alerting | reporting control system by embodiment of this invention.

  Embodiments of the present invention will be described below based on the drawings.

Embodiment 1
<Configuration>
First, the configuration of the notification control device according to the first embodiment of the present invention will be described.

  FIG. 1 is a block diagram showing an example of the configuration of the notification control device 1 according to the first embodiment. In addition, in FIG. 1, the required minimum component which comprises the alerting | reporting control apparatus by this Embodiment is shown.

  As shown in FIG. 1, the notification control device 1 includes a peripheral information acquisition unit 2, a primary event recognition unit 3, a secondary event estimation unit 4, and a notification control unit 5. The surrounding information acquisition unit 2 acquires surrounding information indicating at least the situation around the host vehicle. The primary event recognition unit 3 recognizes, from the surrounding information acquired by the surrounding information acquisition unit 2, a primary event which is a state of an object or a person present in the vicinity of the host vehicle. The primary event recognized by the primary event recognition unit may be a situation in which the object or person is stationary or a situation in which the object or person is moving.

  The secondary event estimation unit 4 is a situation that can occur after the primary event recognized by the primary event recognition unit 3, and is a situation that has a causal relationship with the situation of the primary event Is estimated according to a predetermined rule. The notification control unit 5 performs control to notify the second event estimated by the second event estimation unit 4.

  Next, another configuration of the notification control device including the notification control device 1 shown in FIG. 1 will be described.

  FIG. 2 is a block diagram showing an example of the configuration of the notification control device 6 according to another configuration.

  As shown in FIG. 2, the notification control device 6 includes a surrounding information acquisition unit 2, a primary event recognition unit 3, a secondary event estimation unit 4, a notification control unit 5, and a host vehicle information acquisition unit 7. And a map information acquisition unit 8 and an alarm image generation unit 9. The peripheral information acquisition unit 2 is connected to the peripheral recognition device 10. The notification control unit 5 is connected to a HUD (Head Up Display) 11.

  The surrounding area recognition device 10 recognizes at least the surrounding situation of the own vehicle, and outputs the recognized surrounding situation of the own vehicle to the surrounding information acquisition unit 2 as surrounding information. The periphery recognition device 10 is installed in the host vehicle, and is configured of, for example, a camera, a millimeter wave radar, or a combination thereof. The periphery recognition device 10 is not limited to the camera and the millimeter wave radar, and may be any device that recognizes the surroundings of the host vehicle. The peripheral information acquisition unit 2 acquires peripheral information output from the peripheral recognition device 10.

  The host vehicle information acquisition unit 7 acquires host vehicle information, which is various information related to the host vehicle, via an in-vehicle network provided in the host vehicle. For example, the host vehicle information acquisition unit 7 acquires current position information of the host vehicle as host vehicle information from a GPS (Global Positioning System).

  The map information acquisition unit 8 is configured of, for example, a storage device such as a hard disk drive (HDD: Hard Disk Drive) or a semiconductor memory, and acquires and stores map information. The map information acquisition unit 8 may acquire map information from the outside. For example, the map information acquisition unit 8 may be acquired by downloading from an external server or the like via a communication network, or may be acquired by reading from a storage medium such as a memory.

  For example, when the periphery recognition device 10 is a camera, the primary event recognition unit 3 recognizes a primary event which is a state of an object or a person from an image which is peripheral information acquired by the peripheral information acquisition unit 2. Also, even if the primary event recognition unit 3 recognizes the primary event based on the current position information of the own vehicle acquired by the own vehicle information acquisition unit 7 and the map information acquired by the map information acquisition unit 8. Good. The primary event recognition unit 3 may recognize the primary event by arbitrarily combining the surrounding information, the vehicle information, and the map information.

  The secondary event estimation unit 4 estimates a secondary event from the primary event recognized by the primary event recognition unit 3 according to a predetermined rule. The predetermined rule is converted into data and held by the second event estimating unit 4, and is data in which the first event and the second event as shown in FIG. 3 are associated as an example.

  For example, if the primary event recognition unit 3 recognizes "ball out" as a primary event, the secondary event estimation unit 4 estimates "child out" as a secondary event. That is, when the ball comes out, the secondary event estimation unit 4 estimates that a child comes out to pick up the ball. In this case, the primary event recognition unit 3 can recognize the ball from an image photographed by a camera which is, for example, the periphery recognition device 10. Note that the primary event recognition unit 3 may recognize that the ball is stationary as a primary event without being limited to recognizing the movement of the ball as a primary event.

  If the first event recognition unit 3 recognizes the "stopped vehicle" stopping at the side of the road as the first event, the second event estimation unit 4 selects "the jumping out of a person or a motorcycle from behind the vehicle" Estimated as the next event. That is, when there is a stopped vehicle, the secondary event estimation unit 4 estimates that a person or a motorcycle comes out of the shade of the vehicle. In this case, the primary event recognition unit 3 can recognize a stopped vehicle, for example, from an image captured by a camera that is the periphery recognition device 10.

  When the primary event recognition unit 3 recognizes "crossing" as a primary event, the secondary event estimation unit 4 estimates "passing of a train" as a secondary event. That is, when there is a level crossing, the secondary event estimating unit 4 estimates that the train passes. In this case, the primary event recognition unit 3 can recognize a level crossing from an image captured by a camera which is, for example, the periphery recognition device 10. Alternatively, the primary event recognition unit 3 can recognize the level crossing based on the current position information of the own vehicle acquired by the own vehicle information acquisition unit 7 and the map information acquired by the map information acquisition unit 8.

  When the primary event recognition unit 3 recognizes "reduction of vehicle lane" as a primary event, the secondary event estimation unit 4 estimates "interrupted vehicle" as a secondary event. That is, when the vehicle lane of the road on which the host vehicle travels decreases, the secondary event estimation unit 4 estimates that another vehicle breaks into the lane in which the host vehicle is traveling. In this case, the primary event recognition unit 3 can recognize that the vehicle lane of the road on which the host vehicle travels is reduced, for example, from the image captured by the camera which is the periphery recognition device 10. Alternatively, based on the current position information of the host vehicle acquired by the host vehicle information acquisition unit 7 and the map information acquired by the map information acquisition unit 8, the first event recognition unit 3 is a vehicle on the road on which the host vehicle travels. It can be recognized that the lanes decrease. In addition, as a case where the vehicle lane of the road the own vehicle travels decreases, for example, the case where the vehicle lane decreases from two lanes to one lane can be mentioned. In addition, as the first event when the second event is set as the "interrupt vehicle", not only when the vehicle lane decreases, but also in the vehicle lane in which the own vehicle is traveling, such as a junction of a freeway A situation in which another vehicle is expected to interrupt may be set as the primary event.

  When the primary event recognition unit 3 recognizes "a person who has an object on the crossing overhead overhead road and has an object" as a primary event, the secondary event estimation unit 4 "drops an object" Estimate as a secondary event. The crossing overhead road is, for example, a footbridge. That is, if there is a person holding an object on the footbridge crossing the road on which the host vehicle is traveling, for example, the second event estimation unit 4 is the road on which the host vehicle is traveling. It is estimated to fall into In this case, the primary event recognition unit 3 recognizes from the image taken by the camera which is, for example, the periphery recognition device 10, that there is a person who has an object facing this on an intersecting overhead road. Can.

  The alarm image generation unit 9 generates a first event alarm image indicating the first event recognized by the first event recognition unit 3. In addition, the alarm image generation unit 9 generates a second event alarm image indicating the second event estimated by the second event estimation unit 4.

  The notification control unit 5 performs control to display the primary event warning image or the secondary event warning image generated by the warning image generation unit 9 on the HUD 11. The HUD 11 displays a primary event alert image or a secondary event alert image using AR (Augmented Reality) technology. The driver can see that the primary event warning image or the secondary event warning image is displayed superimposed on the view seen through the windshield of the host vehicle. The notification control unit 5 may perform control to simultaneously display the primary event warning image and the secondary event warning image on the HUD 11.

  FIG. 4 is a block diagram showing an example of the hardware configuration of the notification control device 6. The same applies to the notification control device 1.

  A surrounding information acquisition unit 2, a primary event recognition unit 3, a secondary event estimation unit 4, a notification control unit 5, a host vehicle information acquisition unit 7, a map information acquisition unit 8, and an alarm image generation unit in the notification control device 6 Each function of 9 is realized by a processing circuit. That is, the notification control device 6 acquires surrounding information, recognizes a first event, estimates a second event, controls notification, acquires vehicle information, acquires map information, and Processing circuitry is provided for generating the next event alert image and the second event alert image. The processing circuit is a processor 12 (also referred to as a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, or a DSP (Digital Signal Processor)) that executes a program stored in the memory 13.

  A surrounding information acquisition unit 2, a primary event recognition unit 3, a secondary event estimation unit 4, a notification control unit 5, a host vehicle information acquisition unit 7, a map information acquisition unit 8, and an alarm image generation unit in the notification control device 6 Each function of 9 is realized by software, firmware, or a combination of software and firmware. The software or firmware is described as a program and stored in the memory 13. The processing circuit implements the functions of the respective units by reading and executing the program stored in the memory 13. That is, the notification control device 6 obtains the peripheral information, the step of recognizing the first event, the step of estimating the second event, the step of controlling the notification, the step of obtaining the own vehicle information, the map information A memory 13 is provided for storing a program for obtaining, for generating the primary event alert image and for generating the secondary event alert image, as a result. In addition, these programs include a peripheral information acquisition unit 2, a primary event recognition unit 3, a secondary event estimation unit 4, a notification control unit 5, a host vehicle information acquisition unit 7, a map information acquisition unit 8, and an alarm image. It can be said that the procedure or method of the generation unit 9 is to be executed by a computer. Here, the memory means, for example, nonvolatile or volatile such as random access memory (RAM), read only memory (ROM), flash memory, erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), etc. Semiconductor memory, magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), etc., or any storage medium used in the future.

<Operation>
Next, the operation of the notification control device 6 will be described.

  FIG. 5 is a flowchart showing an example of the operation of the notification control device 6. In the following, the surrounding area recognition device 10 is a camera, and images the front of the host vehicle.

  In step S11, the peripheral information acquisition unit 2 acquires the image captured by the peripheral recognition device 10 as peripheral information. In step S12, the primary event recognition unit 3 recognizes the primary event from the image acquired by the peripheral information acquisition unit 2. Here, the primary event recognition unit 3 recognizes a ball existing in front of the host vehicle as a primary event.

  In step S13, the alarm image generation unit 9 generates a primary event alarm image indicating the primary event recognized by the primary event recognition unit 3. The notification control unit 5 performs control to display the primary event warning image generated by the warning image generation unit 9 on the HUD 11.

  FIG. 6 is a diagram showing an example of the case where the primary event alarm image is displayed on the HUD 11. In FIG. 6, the primary event warning image 14 is an image that emphasizes the ball itself. The primary event warning image 15 is an image that warns that there is an object that impedes traveling ahead of the host vehicle, and in the example of FIG. 6, an image that warns that a ball is present. It is. The primary event warning image 14 and the primary event warning image 15 are not limited to being displayed together as shown in FIG. 6, but either one may be displayed.

  In step S14, the secondary event estimation unit 4 estimates a secondary event from the primary event recognized by the primary event recognition unit 3. Specifically, the secondary event estimating unit 4 estimates a secondary event based on the data as shown in FIG. 3 that the secondary event estimating unit 4 has. Here, the secondary event estimation unit 4 estimates “jump out of child” as a secondary event.

  In step S15, the alarm image generation unit 9 generates a second event alarm image indicating the second event estimated by the second event estimation unit 4. The notification control unit 5 performs control to display the secondary event warning image generated by the warning image generation unit 9 on the HUD 11.

  FIG. 7 is a view showing an example of the case where the secondary event alarm image is displayed on the HUD 11. In FIG. 7, the secondary event alarm image 16 is an image in which a child jumps out in front of the host vehicle. The secondary event warning image 16 is displayed at a predetermined position. The position at which the secondary event alarm image 16 is displayed may be arbitrarily set by the user.

  From the above, according to the first embodiment, the notification control devices 1 and 6 not only receive the primary event that is the actual situation, but also the secondary event that is the situation that may occur after the primary event. Since the notification is made, appropriate notification can be given to the driver. Therefore, the driver can take appropriate action in consideration of the secondary event.

Second Embodiment
<Configuration>
First, the configuration of a notification control device according to a second embodiment of the present invention will be described.

  FIG. 8 is a block diagram showing an example of the configuration of the notification control device 17 according to the second embodiment.

  As shown in FIG. 8, the notification control device 17 is characterized by including a second order event position estimation unit 18. The other configuration and operation are the same as in the first embodiment, and thus detailed description will be omitted here.

  The second event position estimation unit 18 estimates the occurrence position of the second event estimated by the second event estimation unit 4 based on the position at which the first event recognition unit 3 recognizes the first event. . The alarm image generation unit 9 generates a second event alarm image corresponding to the occurrence position estimated by the second event position estimation unit 18.

  The function of the secondary event position estimation unit 18 in the notification control device 17 is realized by a processing circuit. That is, the secondary event position estimation unit 18 includes a processing circuit for estimating the occurrence position of the secondary event. The processing circuit is, for example, a processor 12 which executes a program stored in the memory 13 as shown in FIG.

  The function of the secondary event position estimation unit 18 in the notification control device 17 is realized by software, firmware, or a combination of software and firmware. The software or firmware is described as a program and stored in the memory 13. The processing circuit implements the function of the secondary event position estimation unit 18 by reading and executing the program stored in the memory 13. That is, the notification control device 17 includes the memory 13 for storing a program which results in the step of estimating the occurrence position of the secondary event. Further, it can be said that this program causes a computer to execute the procedure or method of the secondary event position estimation unit 18.

<Operation>
Next, the operation of the notification control device 17 will be described.

  FIG. 9 is a flowchart showing an example of the operation of the notification control device 17. In addition, since step S21-step S24 of FIG. 9 respond | correspond to step S11-step S14 of FIG. 5, description is abbreviate | omitted here. Below, step S25 and step S26 are demonstrated. The surrounding area recognition device 10 is a camera, and images the front of the host vehicle.

  In step S25, the secondary event position estimation unit 18 determines, in the image acquired by the peripheral information acquisition unit 2, a predetermined position near the position at which the primary event recognition unit 3 recognizes the primary event, Estimated as the occurrence position of the second event.

  In step S26, the alarm image generation unit 9 generates a second event alarm image corresponding to the occurrence position of the second event estimated by the second event position estimation unit 18. The notification control unit 5 performs control to display the secondary event warning image generated by the warning image generation unit 9 on the HUD 11.

  FIG. 10 is a diagram showing an example when the secondary event alarm image is displayed on the HUD 11. In FIG. 10, the secondary event alarm image 16 is displayed at the occurrence position estimated by the secondary event position estimation unit 18. In the example of FIG. 10, a situation in which a child follows a ball and jumps out ahead of the host vehicle is represented.

  From the above, according to the second embodiment, since the notification control device 17 displays the secondary event at the position where the possibility of the secondary event is high, the driver performs the secondary event. It can grasp more correctly than form 1.

Embodiment 3
<Configuration>
First, the configuration of a notification control device according to a third embodiment of the present invention will be described.

  FIG. 11 is a block diagram showing an example of the configuration of the notification control device 19 according to the third embodiment.

  As shown in FIG. 11, the notification control device 19 is characterized by including a recognition range restriction unit 20. The other configuration and operation are the same as in the second embodiment, and thus detailed description will be omitted here.

  The recognition range restriction unit 20 restricts the recognition range by the primary event recognition unit 3 to a predetermined range including the occurrence position of the second event estimated by the second event position estimation unit 18. The primary event recognition unit 3 recognizes the primary event within the range limited by the recognition range limiting unit 20. For example, when the primary event recognition unit 3 recognizes the primary event from the image, the primary event recognition unit 3 recognizes the primary event within the range of the image limited by the recognition range limiting unit 20. I do.

  The function of the recognition range limiting unit 20 in the notification control device 19 is realized by a processing circuit. That is, the recognition range limiting unit 20 limits the recognition range by the primary event recognition unit 3 to a predetermined range including the occurrence position of the secondary event estimated by the secondary event position estimation unit 18. The processing circuit of The processing circuit is, for example, a processor 12 which executes a program stored in the memory 13 as shown in FIG.

  The function of the recognition range limiting unit 20 in the notification control device 19 is realized by software, firmware, or a combination of software and firmware. The software or firmware is described as a program and stored in the memory 13. The processing circuit implements the function of the recognition range restriction unit 20 by reading and executing the program stored in the memory 13. That is, the notification control device 19 limits the recognition range by the primary event recognition unit 3 to a predetermined range including the occurrence position of the secondary event estimated by the secondary event position estimation unit 18. A memory 13 is provided for storing a program to be executed as a result. Further, it can be said that this program causes a computer to execute the procedure or method of the recognition range restriction unit 20.

<Operation>
Next, the operation of the notification control device 19 will be described.

  FIG. 12 is a flowchart showing an example of the operation of the notification control device 19. Since step S31 to step S36 in FIG. 12 correspond to step S21 to step S26 in FIG. 9, the description will be omitted here. Hereinafter, steps S37 and S38 will be described. The surrounding area recognition device 10 is a camera, and images the front of the host vehicle.

  In step S37, the notification control device 19 determines whether to limit the recognition range of the primary event recognition unit 3. The determination as to whether or not to limit the recognition range by the primary event recognition unit 3 may be made according to the resolution of the image acquired by the peripheral information acquisition unit 2. Specifically, when the resolution of the image is high, the processing load in the primary event recognition unit 3 becomes large, so it may be determined to limit the recognition range by the primary event recognition unit 3. If the resolution of the image is low, the processing load on the primary event recognition unit 3 is small, so it may be determined that the recognition range by the primary event recognition unit 3 is not limited. Note that the user may arbitrarily set the determination as to whether or not to limit the recognition range by the primary event recognition unit 3. When limiting the recognition range by the primary event recognition unit 3, the process proceeds to step S38. On the other hand, when the recognition range by the primary event recognition unit 3 is not limited, the process proceeds to step S31.

  In step S38, the recognition range restriction unit 20 restricts the recognition range by the first event recognition unit 3 to a predetermined range including the occurrence position of the second event estimated by the second event position estimation unit 18 Do. Specifically, the recognition range limiting unit 20 limits the recognition range by the primary event recognition unit 3 to, for example, the recognition range 21 as shown in FIG. The recognition range 21 shown in FIG. 13 is illustrated for the purpose of explanation, and is not displayed on the HUD 11 in practice.

  From the above, according to the third embodiment, in order to limit the recognition range by the first event recognition unit 3, it is possible to improve the efficiency of recognizing the first event by the first event recognition unit 3. It is possible to display secondary events quickly. Also, even if the resource for recognizing the primary event is limited by the primary event recognition unit 3, the primary event recognition unit 3 can smoothly recognize the primary event. it can.

  In the example of FIG. 12, the primary event recognition unit 3 recognizes the primary event within the range restricted by the recognition range limiting unit 20 for all the images acquired by the peripheral information acquisition unit 2. However, the present invention is not limited to this. For example, within the range restricted by the recognition range limiting unit 20, the primary event recognition unit 3 recognizes the first event with respect to all the images, and outside the range restricted by the recognition range limiting unit 20. Alternatively, the primary event recognition may be performed on the image acquired by the peripheral information acquisition unit 2 at predetermined intervals.

Fourth Preferred Embodiment
<Configuration>
First, the configuration of a notification control device according to the fourth embodiment of the present invention will be described.

  FIG. 14 is a block diagram showing an example of the configuration of the notification control device 22 according to the fourth embodiment.

  As shown in FIG. 14, the notification control device 22 is characterized by including a communication unit 23. The other configuration and operation are the same as in the third embodiment, and thus detailed description will be omitted here.

  The communication unit 23 is communicably connected to each of the image recognition learning server 24 and the secondary event estimation data server 25 provided outside the notification control device 22, and the image recognition learning server 24 and the secondary event Parameters and data to be described later are acquired from each of the estimated data servers 25.

  The image recognition and learning server 24 is configured to improve the recognition accuracy of the primary event by the primary event recognition unit 3 with respect to the parameters used when the primary event recognition unit 3 recognizes the primary event from the image. Machine learning. The secondary event estimation data server 25 accumulates data used by the secondary event estimation unit 4 to estimate a secondary event. The image recognition learning server 24 and the secondary event estimation data server 25 may be provided separately or integrally.

  The function of the communication unit 23 in the notification control device 22 is realized by a processing circuit. That is, the communication unit 23 includes a processing circuit for communicating with each of the image recognition learning server 24 and the secondary event estimation data server 25. The processing circuit is, for example, a processor 12 which executes a program stored in the memory 13 as shown in FIG.

  The function of the communication unit 23 in the notification control device 22 is realized by software, firmware, or a combination of software and firmware. The software or firmware is described as a program and stored in the memory 13. The processing circuit implements the function of the communication unit 23 by reading and executing the program stored in the memory 13. That is, the notification control device 22 includes the memory 13 for storing a program that results in the step of communicating with each of the image recognition learning server 24 and the secondary event estimation data server 25. Moreover, it can be said that this program causes a computer to execute the procedure or method of the communication unit 23.

<Operation>
Next, the operation of the notification control device 22 will be described.

  FIG. 15 is a flowchart illustrating an example of the operation of the notification control device 22. In addition, since step S41-step S48 of FIG. 15 respond | correspond to step S31-step S38 of FIG. 12, description is abbreviate | omitted here. In addition, in step S47 of FIG. 15, when not limiting the recognition range by the primary event recognition part 3, it transfers to step S49. Hereinafter, step S49 will be described. The surrounding area recognition device 10 is a camera, and images the front of the host vehicle.

  In step S49, the communication unit 23 acquires from the image recognition learning server 24 the parameters used when the primary event recognition unit 3 recognizes the primary event from the image. The parameters used by the primary event recognition unit 3 are updated to the parameters acquired by the communication unit 23. Further, the communication unit 23 acquires, from the secondary event estimation data server 25, data used when the secondary event estimation unit 4 estimates a secondary event. The data used by the second event estimation unit 4 is updated to the data acquired by the communication unit 23. The process of step S49 is not limited to the process performed after step S47 or step S48, but may be any timing.

  Next, operations of the image recognition learning server 24 and the secondary event estimation data server 25 will be described.

  FIG. 16 is a flowchart showing an example of the operation of the image recognition learning server 24 and the secondary event estimation data server 25.

  In step S51, the image recognition learning server 24 determines the recognition accuracy of the primary event by the primary event recognition unit 3 for the parameter used when the primary event recognition unit 3 recognizes the primary event from the image. Do machine learning to improve The machine-learned parameters are updated.

  In step S52, the secondary event estimation data server 25 accumulates data used when the secondary event estimation unit 4 estimates a secondary event. Specifically, the secondary event estimation data server 25 accumulates data in which the primary event and the secondary event are associated with each other as shown in FIG. 3, for example, and updates the data as necessary.

  In step S53, the image recognition learning server 24 transmits the parameter used when the primary event recognition unit 3 recognizes the primary event from the image, to the communication unit 23 of the notification control device 22. Further, the secondary event estimation data server 25 transmits data used when the secondary event estimation unit 4 estimates a secondary event to the communication unit 23 of the notification control device 22. Note that the image recognition learning server 24 and the secondary event estimation data server 25 may transmit the parameter and data simultaneously or may transmit separately. Further, the timing at which the image recognition learning server 24 and the secondary event estimation data server 25 transmit parameters and data may be a predetermined timing or an arbitrary timing set by the user.

  From the above, according to the fourth embodiment, the first event recognition unit 3 updates the parameters used when the first event recognition unit 3 recognizes the first event from the image. It is possible to improve the recognition accuracy of the primary event. In addition, by updating data used when the second event estimating unit 4 estimates the second event, it is possible to improve the accuracy with which the second event estimating unit 4 estimates the second event. .

  In addition, although the case where the alerting | reporting control apparatus 19 shown in FIG. 11 is further equipped with the communication part 23 was demonstrated in FIG. 14, it does not restrict to this. For example, the communication unit 23 may be provided in each of the notification control device 6 shown in FIG. 2 and the notification control device 17 shown in FIG.

The Fifth Preferred Embodiment
<Configuration>
First, the configuration of a notification control device according to the fifth embodiment of the present invention will be described.

  FIG. 17 is a block diagram showing an example of the configuration of the notification control device 26 according to the fifth embodiment.

  As shown in FIG. 17, the notification control device 26 is characterized by including a vehicle control unit 27. The other configuration and operation are the same as in the first embodiment, and thus detailed description will be omitted here.

  The vehicle control unit 27 controls the driving operation of the own vehicle based on the estimation result of the second event by the second event estimation unit 4. Specifically, the vehicle control unit 27 controls each operation of the accelerator, the brake, and the steering wheel of the host vehicle.

  The function of the vehicle control unit 27 in the notification control device 26 is realized by a processing circuit. That is, the vehicle control unit 27 includes a processing circuit for controlling the driving of the host vehicle based on the estimation result of the second event by the second event estimation unit 4. The processing circuit is, for example, a processor 12 which executes a program stored in the memory 13 as shown in FIG.

  The function of the vehicle control unit 27 in the notification control device 26 is realized by software, firmware, or a combination of software and firmware. The software or firmware is described as a program and stored in the memory 13. The processing circuit implements the function of the vehicle control unit 27 by reading and executing the program stored in the memory 13. That is, the vehicle control unit 27 stores a program that results in the step of controlling the driving of the host vehicle based on the estimation result of the second event by the second event estimation unit 4. A memory 13 is provided. Further, it can be said that this program causes a computer to execute the procedure or method of the vehicle control unit 27.

<Operation>
Next, the operation of the notification control device 26 will be described.

  FIG. 18 is a flowchart showing an example of the operation of the notification control device 26. In addition, since step S61-step S65 of FIG. 18 respond | correspond to step S11-step S15 of FIG. 5, description is abbreviate | omitted here. Hereinafter, step S66 will be described. The surrounding area recognition device 10 is a camera, and images the front of the host vehicle.

  In step S66, the vehicle control unit 27 controls the driving operation of the host vehicle based on the estimation result of the second event by the second event estimation unit 4. Specifically, the vehicle control unit 27 reduces the reaction force of the steering wheel in the direction of avoiding the second event, operates the steering wheel in the direction of avoiding the second event, applies the brake early, etc. Control the driving operation of the vehicle. For example, referring to FIG. 3, when the secondary event is the “interrupt vehicle”, the vehicle control unit 27 performs control to decelerate the host vehicle.

  The vehicle control unit 27 may control the driving operation of the own vehicle based on the recognition result by the primary event recognition unit 3. For example, referring to FIG. 3, when the primary event is “ball out”, the vehicle control unit 27 may perform control so as to handle the ball so as to avoid the ball.

  From the above, according to the fifth embodiment, not only the secondary event is displayed, but also the driving operation of the vehicle is controlled so as to avoid the secondary event. Therefore, the notification control device 26 can support the driving operation of the host vehicle as well as notifying the second event.

  In addition, although FIG. 17 demonstrated the case where the alerting | reporting control apparatus 6 shown in FIG. 2 was further equipped with the vehicle control part 27, it does not restrict it to this. For example, the vehicle control unit 27 may be provided in each of the notification control device 17 shown in FIG. 8, the notification control device 19 shown in FIG. 11, and the notification control device 22 shown in FIG.

  Although the first to fifth embodiments have described the case where the primary event recognition unit 3 recognizes a primary event, the present invention is not limited to this. For example, a primary event recognized by another vehicle may be acquired from the other vehicle by inter-vehicle communication. Moreover, you may acquire the primary event which infrastructure installations, such as a camera installed in the road, recognized from the said infrastructure installation by road-to-vehicle communication. In this case, in the host vehicle, the secondary event can be estimated using the primary event relating to the place where the surrounding area recognition device 10 provided in the host vehicle can not shoot.

  In the first to fifth embodiments, the second event estimated by the second event estimating unit 4 may be transmitted to another vehicle.

  In the first to fifth embodiments, the camera as the periphery recognition device 10 captures the front of the host vehicle, but the present invention is not limited to this. The camera which is the periphery recognition device 10 may capture an omnidirectional or arbitrary direction around the host vehicle.

  The notification control device described above is not only an on-vehicle navigation device, that is, a car navigation device, but also a navigation device or navigation constructed as a system by appropriately combining a PND (Portable Navigation Device) mountable on a vehicle and a server. The present invention can be applied to devices other than devices. In this case, each function or each component of the notification control device is distributively arranged in each function constructing the system.

  Specifically, as an example, the function of the notification control device can be arranged in the server. For example, as shown in FIG. 19, the vehicle side is equipped with the periphery recognition device 10 and the HUD 11, and the server 28 has the periphery information acquisition unit 2, the primary event recognition unit 3, the secondary event estimation unit 4, and the notification control unit 5. By providing the host vehicle information acquisition unit 7, the map information acquisition unit 8, and the alarm image generation unit 9, a notification control system can be constructed. The same applies to the notification control device 17 shown in FIG. 8, the notification control device 19 shown in FIG. 11, the notification control device 22 shown in FIG. 14, and the notification control device 26 shown in FIG.

  In addition, the function of the notification control device can be disposed in the server and the mobile communication terminal 30. For example, as shown in FIG. 20, the vehicle side is equipped with the periphery recognition device 10 and the HUD 11, and the server 29 acquires the periphery information acquisition unit 2, the first event recognition unit 3, the second event estimation unit 4, and the own vehicle information acquisition A notification control system can be constructed by providing the notification control unit 5 in the mobile communication terminal 30 including the unit 7, the map information acquisition unit 8, and the alarm image generation unit 9. The same applies to the notification control device 17 shown in FIG. 8, the notification control device 19 shown in FIG. 11, the notification control device 22 shown in FIG. 14, and the notification control device 26 shown in FIG.

  Even in the case of the above configuration, the same effect as the above embodiment can be obtained.

  Further, software (informing control method) for executing the operation in the above embodiment may be incorporated in, for example, a server or a mobile communication terminal.

  Specifically, as one example, the notification control method described above acquires the peripheral information indicating at least the peripheral situation of the host vehicle, and from the acquired peripheral information, the status of the object or person existing around the host vehicle A primary event is recognized, and a secondary event that is a situation that may occur after the recognized primary event and that has a causal relationship with the situation of the primary event is estimated by a predetermined rule. Control to notify the estimated secondary event.

  From the above, the same effect as the above embodiment can be obtained by incorporating the software for executing the operation in the above embodiment into the server or the mobile communication terminal and operating the software.

  In the present invention, within the scope of the invention, each embodiment can be freely combined, or each embodiment can be appropriately modified or omitted.

  Although the present invention has been described in detail, the above description is an exemplification in all aspects, and the present invention is not limited thereto. It is understood that countless variations not illustrated are conceivable without departing from the scope of the present invention.

  DESCRIPTION OF SYMBOLS 1 alerting | reporting control apparatus, 2 periphery information acquisition part, 3 primary event recognition part, 4 secondary event estimation part, 5 alerting | reporting control part, 6 alerting | reporting control apparatus, 7 self-vehicles information acquisition part, 8 map information acquisition part, 9 alarm image generation unit 10 peripheral recognition device 11 HUD 12 processor 13 memory 14 15 primary event alarm image 16 secondary event alarm image 17 notification control device 18 secondary event position estimation Unit 19 notification control device 20 recognition range limiting unit 21 recognition range 22 notification control device 23 communication unit 24 image recognition learning server 25 secondary event estimation data server 26 notification control device 27 vehicle control unit 28, 29 servers, 30 mobile communication terminals.

In order to solve the above-described problems, the notification control device according to the present invention acquires at least the peripheral information acquisition unit that acquires peripheral information indicating the peripheral conditions of the host vehicle, and the peripheral information acquired by the peripheral information acquisition unit. Primary event recognition unit that recognizes a primary event that is the status of an object or person present in the vicinity of the scene, and a status that can occur after the primary event recognized by the primary event recognition unit, The second event estimation unit estimates the second event which is a condition having a causal relationship with the situation of the primary event according to a predetermined rule, and broadcasts the second event estimated by the second event estimation unit The second event is an object or person present in the vicinity of the host vehicle that the primary event recognizer does not recognize as a primary event affects the travel of the host vehicle. Ru situation der on the.

In addition, the notification control method according to the present invention acquires at least peripheral information indicating the state around the host vehicle, and from the acquired peripheral information, a primary event that is a state of an object or person existing around the host vehicle A second event which is a situation that may occur after a recognized first event and is a situation having a causal relationship with the situation of the first event is estimated according to a predetermined rule, there line control for notifying the secondary event, secondary event, object or person is present around the subject vehicle not recognized as primary event that affects situation the traveling of the vehicle.

According to the present invention, the notification control device acquires an object existing in the periphery of the vehicle from the periphery information acquisition unit acquiring the periphery information indicating at least the situation around the vehicle and the periphery information acquired by the periphery information acquisition unit A primary event recognition unit that recognizes a primary event that is the condition of a person, and a situation that can occur after the primary event recognized by the primary event recognition unit, and the condition and causality of the primary event A secondary event estimation unit that estimates secondary events that are related situations according to a predetermined rule, and a notification control unit that performs control to notify of secondary events estimated by the secondary event estimation unit with the door, secondary event, first primary event recognition unit affects situations der because the traveling object or person of the vehicle present around the subject vehicle not recognized as primary event , It will be possible to give appropriate notification to the driver

In addition, the notification control method acquires peripheral information indicating at least the peripheral situation of the vehicle, and recognizes from the acquired peripheral information a primary event that is the condition of an object or a person present in the periphery of the vehicle, A secondary event that is a situation that may occur after a recognized primary event and that is a situation that has a causal relationship with the situation of the primary event, estimated by a predetermined rule, and estimated secondary event there line control for notifying, secondary event, since the object or person is present around the subject vehicle not recognized as primary event that affects situation the traveling of the vehicle to driver It is possible to perform appropriate notification.

Claims (8)

A peripheral information acquisition unit that acquires peripheral information indicating at least the situation around the host vehicle;
A first-order event recognition unit that recognizes a first-order event that is a state of an object or a person present around the host vehicle from the vicinity information acquired by the vicinity information acquisition unit;
It is a situation that may occur after the primary event recognized by the primary event recognition unit, and a secondary event having a causal relationship with the situation of the primary event is predetermined. A second event estimation unit that estimates by rules;
A notification control unit that performs control to notify the second event estimated by the second event estimation unit;
A notification control device comprising: It further comprises an alarm image generation unit that generates a second event alarm image indicating the second event estimated by the second event estimation unit,
The notification control device according to claim 1, wherein the notification control unit performs control to notify the second event warning image generated by the warning image generation unit. And a second event position estimation unit for estimating the occurrence position of the second event based on the position at which the first event recognition unit recognizes the first event.
The notification control device according to claim 2, wherein the alarm image generation unit generates the second event alarm image corresponding to the occurrence position estimated by the second event position estimation unit.   The system further comprises a recognition range restriction unit for restricting the recognition range by the first event recognition unit to a predetermined range including the occurrence position estimated by the second event position estimation unit. The notification control device according to Item 3.   The notification control device according to claim 1, wherein the parameter used when the primary event recognition unit recognizes the primary event is updatable.   The notification control device according to claim 1, wherein data used when the second event estimating unit estimates the second event is updatable.   The notification control device according to claim 1, further comprising: a vehicle control unit configured to control a driving operation of the host vehicle based on an estimation result of the second event by the second event estimation unit. . At least get peripheral information indicating the situation around the vehicle,
Recognizing a primary event that is a state of an object or person existing around the host vehicle from the acquired surrounding information;
According to a predetermined rule, a secondary event that is a situation that may occur after the recognized primary event and that has a causal relationship with the situation of the primary event is estimated;
A notification control method of performing control to notify the estimated second event.

Images