JP2019012046A - Notification device, notification system, and notification method - Google Patents

Abstract

To provide a notification device, a notification system, and a notification method capable of surely notifying an occupant of notification information.SOLUTION: A notification device comprises: a notification part; a visual line detection part; and a visual line movement prediction part. The notification part controls a plurality of types of output devices including a display device capable of outputting notification information, and causes an output device to output the notification information. The visual line detection part detects a visual line of an occupant. The visual line movement prediction part predicts a movement direction of the visual line on the basis of the detected visual line. When the display device does not exist in the predicted movement direction of the visual line, the notification part causes the output device different from the display device to output the notification information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a notification device, a notification system, and a notification method.

  2. Description of the Related Art Conventionally, a vehicle is equipped with a device for notifying passengers of various notification information such as emergency information such as warnings and road traffic information such as traffic jam information. In such a device, for example, notification information is displayed on a display device to notify a passenger (see, for example, Patent Document 1).

JP 2005-134971 A

  However, in the above-described conventional technology, for example, when the passenger does not visually recognize the display device, the notification information may not be recognized by the passenger. Thus, the prior art has room for further improvement in that the notification information is surely notified to the passenger.

  This invention is made in view of the above, Comprising: It aims at providing the notification apparatus, the notification system, and the notification method which can notify notification information reliably with respect to a passenger.

  In order to solve the above problems and achieve the object, the present invention includes a notification device including a notification unit, a line-of-sight detection unit, and a line-of-sight movement prediction unit. The notification unit controls each of a plurality of types of output devices including a display device capable of outputting notification information, and outputs the notification information via the output device. The line-of-sight detection unit detects the line of sight of the passenger. The line-of-sight movement prediction unit predicts the movement direction of the line of sight based on the line of sight detected by the line-of-sight detection unit. In addition, when the display device does not exist in the movement direction of the line of sight predicted by the line-of-sight movement prediction unit, the notification unit outputs the notification information via the output device different from the display device.

  According to the present invention, notification information can be reliably notified to the passenger.

FIG. 1 is a schematic explanatory diagram of a notification method according to the embodiment. FIG. 2 is a block diagram illustrating a configuration example of the notification system according to the embodiment. FIG. 3 is a diagram schematically showing the interior of the vehicle. FIG. 4 is a diagram for explaining the condition information. FIG. 5A is a diagram (part 1) for explaining priority information. FIG. 5B is a diagram (part 2) for explaining priority information. FIG. 6A is a flowchart (part 1) illustrating a processing procedure executed by the notification device. FIG. 6B is a flowchart (part 2) illustrating a processing procedure executed by the notification device. FIG. 7 is a diagram illustrating an example of guidance control. FIG. 8 is a flowchart illustrating a processing procedure executed by the notification device according to the modification.

  Hereinafter, embodiments of a notification device, a notification system, and a notification method disclosed in the present application will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by embodiment shown below.

<1. Overview of notification method>
Below, the outline | summary of the notification method which concerns on embodiment first is demonstrated with reference to FIG. FIG. 1 is a schematic explanatory diagram of a notification method according to the embodiment.

  As shown in FIG. 1, the notification method according to the embodiment is executed by a notification device 10 mounted on a vehicle 1 such as an automobile. Specifically, in addition to the notification device 10 described above, the output device 50 and the imaging unit 61 are mounted on the vehicle 1.

  The vehicle compartment 2 of the vehicle 1 is provided with a driver seat S1 and a passenger seat S2 in front of it, and rear seats S3, S4, and S5 are provided in the rear. Note that the positions and number of seats shown in FIG. 1 are merely examples and are not limited.

  Further, in the example shown in FIG. 1, it is assumed that the driver P1 is seated in the driver's seat S1 as a passenger, and here, a case where notification information is notified to the driver P1 who is a passenger is taken as an example. I will explain. The notification information includes, for example, emergency information and road traffic information, which will be described later.

  The output device 50 is a device that can output notification information. The output device 50 includes a plurality of types of output devices such as a display device 40 that displays and outputs notification information and a speaker 51 that outputs a sound indicating the notification information. The speaker 51 is an example of a sound output device.

  Specifically, for example, the output device 50 includes a display device 40 and a speaker 51. The display device 40 includes, for example, a meter display 41 and a center display 42.

  The meter display 41 and the center display 42 are attached to, for example, the instrument panel 3 and display various information including notification information. Specifically, the meter display 41 is attached to the instrument panel 3 on the back side of the handle 4 and in the vicinity of the front of the driver's seat S1. The center display 42 is attached near the center of the instrument panel 3 at a position corresponding to between the driver seat S1 and the passenger seat S2.

  The mounting positions of the meter display 41 and the center display 42 described above are merely examples and are not limited. Moreover, although the meter display 41 etc. were mentioned as an example of the display apparatus 40 in the above, it is not restricted to these. That is, as the display device 40, in addition to or instead of the meter display 41 or the like, for example, a head-up display that displays notification information on the windshield, a rear display for passengers in the rear seats S3, S4, and S5 (described later), etc. May be included.

  The speaker 51 is attached to an appropriate position of the vehicle compartment 2 such as a door or an instrument panel 3 (not shown), for example, and outputs sounds indicating various information including notification information. Note that the “sound” output from the speaker 51 may include various sounds such as a sound (sound) imitating a human voice and a warning sound such as a buzzer sound.

  The imaging unit 61 is provided at an appropriate position in the passenger compartment 2 and images a passenger such as the driver P1, for example. As will be described later, the imaging unit 61 is used to detect the passenger's line of sight, and thus captures an image including the face of the passenger, in other words, an image related to the passenger's line of sight, and sends the image information to the notification device 10. Output.

  The notification device 10 controls a plurality of types of output devices 50 and imaging units 61, respectively. The detailed configuration of the notification device 10 will be described later with reference to FIG.

  And in the notification apparatus 10, notification information is reliably notified with respect to a passenger by detecting a passenger's eyes | visual_axis and outputting notification information via the output device 50 based on the detected eyes | visual_axis. I was able to do it.

  Specifically, the notification device 10 according to the present embodiment detects the line of sight of the passenger (here, the driver P1) based on the image information output from the imaging unit 61 (step S1). In the example of FIG. 1, it is assumed that the driver P1 first visually recognizes the front of the vehicle 1, and the line of sight A1 of the driver P1 at that time is indicated by a solid arrow.

  In FIG. 1, for example, the line of sight A2 when the driver P1 is viewing the center display 42 on the left front is indicated by a one-dot chain line arrow, and the driver P1 is viewing the outside of the vehicle on the right front. The line of sight A3 is indicated by a two-dot chain line arrow.

  The notification device 10 continuously or intermittently detects the line of sight of the driver P1, and predicts the movement direction of the line of sight based on the detected line of sight (step S2).

  Then, the notification device 10 displays the notification information on the center display 42 when the predicted movement direction of the line of sight is, for example, the direction of movement from the line of sight A1 to the line of sight A2 (step S3). In other words, when the display device 40 (here, the center display 42) is present in the predicted movement direction of the line of sight, the notification device 10 displays notification information on the center display 42 and notifies the driver P1. Thus, the notification device 10 can notify the driver P1 of the notification information via the center display 42.

  On the other hand, when the predicted movement direction of the line of sight is, for example, the direction of movement from the line of sight A1 to the line of sight A3, the notification device 10 outputs a sound indicating the notification information from the speaker 51 (step S4). In other words, when the display device 40 such as the center display 42 does not exist in the predicted movement direction of the line of sight, the notification device 10 outputs a sound indicating the notification information via the speaker 51 to notify the driver P1. Do.

  As described above, in the notification device 10 according to the present embodiment, when the display device 40 does not exist in the predicted movement direction of the line of sight, the notification information is output from the output device 50 (here, the speaker 51) different from the display device 40. ) To notify the driver P1 who is a passenger.

  Thereby, the notification apparatus 10 can notify notification information reliably with respect to the passenger (for example, driver | operator P1) who has not visually recognized display apparatuses 40, such as the meter display 41 and the center display 42, for example.

  In the above description, the case where the passenger notified of the notification information is the driver P1 has been described as an example. However, the present invention is not limited to this, and the notification information is notified to other passengers seated in the passenger seat S2 or the like. Also good. The case where the notification information is notified to passengers other than the driver P1 will be described later.

<2. Configuration of Notification System with Notification Device>
Next, the configuration of the notification system 100 including the notification device 10 according to the embodiment will be described with reference to FIG. FIG. 2 is a block diagram illustrating a configuration example of the notification system 100 according to the embodiment. In FIG. 2, only components necessary for explaining the features of the present embodiment are represented by functional blocks, and descriptions of general components are omitted.

  In other words, each component illustrated in FIG. 2 is functionally conceptual and does not necessarily need to be physically configured as illustrated. For example, the specific form of distribution / integration of each functional block is not limited to the one shown in the figure, and all or a part thereof is functionally or physically distributed in arbitrary units according to various loads or usage conditions.・ It can be integrated and configured.

  As illustrated in FIG. 2, the notification system 100 according to the present embodiment includes a notification device 10, an output device 50, an imaging unit 61, an in-vehicle sensor group 62, and a communication unit 63. The output device 50 includes a display device 40, a speaker 51, a handle actuator 52, and a seat actuator 53.

  The display device 40 may include a rear display 43 in addition to the meter display 41 and the like described above. FIG. 3 is a diagram schematically showing the interior of the passenger compartment 2. As shown in FIG. 3, the rear display 43 is attached to a position where passengers P3, P4, and P5 seated in the rear seats S3, S4, and S5, such as the center of the ceiling of the passenger compartment 2, are easily visible. 3 shows an example in which the number of the rear displays 43 is one, the present invention is not limited to this, and the rear displays 43 may be attached one by one in front of the rear seats S3, S4, and S5.

  For example, a plurality of speakers 51 may be attached in the passenger compartment 2. Although FIG. 3 shows an example in which the speaker 51 is attached in the vicinity of the driver seat S1, the passenger seat S2, and the rear seats S3 and S4, the attachment position of the speaker 51 can be arbitrarily set.

  As shown in FIG. 3, the handle actuator 52 is an actuator that is built in, for example, the handle 4 and can vibrate the handle 4. The handle actuator 52 is an example of a handle vibration output device.

  The seat actuator 53 is an actuator that is incorporated in, for example, the driver seat S1, the passenger seat S2, and the rear seats S3, S4, and S5, and can vibrate each seat. The seat actuator 53 does not need to be built in all the seats, and may be built in an arbitrary seat such as only the driver's seat S1. The seat actuator 53 is an example of a seat vibration output device. For example, a vibration motor can be used as the handle actuator 52 and the seat actuator 53 described above, but is not limited thereto.

  As described above, the imaging unit 61 captures an image related to the line of sight of the passengers P1 to P5 and outputs the image information to the control unit 20 (described later) of the notification device 10. Note that an in-vehicle camera or the like can be used as the imaging unit 61. In the example illustrated in FIG. 3, the number of the imaging units 61 is one, but may be two or more.

  The in-vehicle sensor group 62 includes various sensors that detect the running state of the vehicle 1 (see FIG. 1), and outputs a signal indicating notification information obtained based on various sensor outputs. For example, the in-vehicle sensor group 62 includes a radar that measures the distance and direction to an object such as another vehicle based on the reflected wave obtained by radiating electromagnetic waves toward the periphery of the vehicle 1, The camera etc. which image objects, such as other vehicles which exist in the way (periphery), are included.

  The in-vehicle sensor group 62 notifies a signal indicating urgent notification information (that is, emergency information) such as an alarm for calling attention to the inter-vehicle distance when the distance to the preceding vehicle is equal to or less than a predetermined value. 10 to the control unit 20.

  The communication unit 63 includes, for example, a road-to-vehicle communication device that transmits and receives various types of information (for example, road traffic information such as traffic congestion information and construction information on a running road) to and from a communication device installed on a shoulder. A signal indicating notification information including the received road traffic information and the like is output to the control unit 20.

  Moreover, the communication part 63 detects the present position of the own vehicle based on the signal obtained by communicating with a GPS (Global Positioning System) satellite, for example. And the communication part 63 is connected to communication networks, such as the internet, acquires the periphery information (for example, sightseeing spot information in the periphery of a current position, popular restaurant information, etc.), and this periphery information is acquired. A signal indicating the included notification information is output to the control unit 20. Thus, the notification information notified to the passenger is, for example, emergency information, road traffic information, and surrounding information, but is not limited thereto.

  Continuing the description of FIG. 2, the notification device 10 includes a control unit 20 and a storage unit 30. The control unit 20 is a microcomputer that includes a line-of-sight detection unit 21, a line-of-sight movement prediction unit 22, a priority calculation unit 23, and a notification unit 24 and includes a CPU (Central Processing Unit) and the like.

  The storage unit 30 is a storage unit configured by a storage device such as a nonvolatile memory or a hard disk drive, and stores condition information 31 and priority information 32.

  In the present embodiment, as described later, the method for notifying the passenger is changed depending on the type of the notification information. The condition information 31 described above includes information on conditions for changing the notification method according to the type of notification information. FIG. 4 is a diagram for explaining the condition information 31.

  As shown in FIG. 4, in the condition information 31, “condition A” is associated when the notification information is emergency information. In the condition information 31, “condition B” is associated when the notification information is road traffic information, and “condition C” is associated when the notification information is peripheral information. Note that the control unit 20 of the notification device 10 notifies the passenger suitable for the type of the notification information based on the condition information 31, which will be described later.

  The priority information 32 includes information used for calculating the priority of the passenger described later. The priority indicates the degree to which the notification information is preferentially notified. For example, when there are a plurality of passengers, the priority is calculated for each of the plurality of passengers.

  Such priority may be calculated based on, for example, the boarding position of the passenger and the age of the passenger. 5A and 5B are diagrams illustrating the priority information 32. FIG. FIG. 5A shows priority information 32a related to the boarding position of the passenger, and FIG. 5B shows priority information 32b related to the age group of the passenger. In the present embodiment, description will be made using an example in which the priority is calculated for the passengers excluding the driver P1, but the present invention is not limited to this, and the priority is calculated for the passengers including the driver P1. May be.

  As shown in FIG. 5A, the priority information 32a relating to the boarding position is associated with information on the number of points used for calculating the priority for the boarding position. The priority information 32a relating to the boarding position is set so that the number of boarding positions estimated to be easily seated or often seated by the passenger who should be notified of the notification information. For example, the score for the passenger seat S2 is 4, the score for the rear seat S3 behind the driver seat S1 is 3, the score for the rear seat S4 behind the passenger seat S2 is 2, and the score for the center rear seat S5 is 1. (See FIG. 3).

  As shown in FIG. 5B, the priority information 32b regarding the age group is associated with information on the number of points used for calculating the priority for the age group. The priority information 32b regarding the age group is set so that the score of the age group of the passenger estimated to be notified of the notification information is increased. For example, the score is set to 3 when the age group is an adult, the score is 2 for an elderly person, the score is 1 for a child, and the score is 0 for a pet other than a baby or a person. Note that the points shown in FIGS. 5A and 5B are merely examples and are not limited.

  Returning to the description of FIG. 2, the line-of-sight detection unit 21 of the control unit 20 acquires the passenger's image captured by the imaging unit 61, performs image analysis processing on the image, and detects the passenger's line of sight. For example, the line-of-sight detection unit 21 compares the direction of the head of the passenger and the position of the eyes in the image of the passenger with a reference image stored in the storage unit 30 in advance, and where the passenger's line of sight is directed. May be detected.

  For example, the imaging unit 61 may be configured to include an infrared camera and an infrared LED, and the line-of-sight detection unit 21 may detect the line of sight of the passenger based on the image captured by the infrared camera. Specifically, the line-of-sight detection unit 21 positions the pupil in the infrared image obtained by imaging the passenger's face illuminated by the infrared LED with the infrared camera and the infrared illumination reflection image generated on the eyeball. Based on the relationship, the sight line of the passenger may be detected.

  Then, the line-of-sight detection unit 21 outputs information indicating the detected line of sight of the passenger to the line-of-sight movement prediction unit 22 and the notification unit 24. Note that the above-described method of detecting the sight line of the passenger is merely an example and is not limited.

  The line-of-sight movement prediction unit 22 predicts the movement direction of the line of sight based on the line of sight detected by the line-of-sight detection unit 21. For example, when the detected line of sight is moving, the line-of-sight movement prediction unit 22 calculates a movement amount of the line of sight per predetermined time, and predicts the movement direction of the line of sight based on the calculated movement amount of the line of sight. May be.

  The priority calculation unit 23 calculates the priority based on the priority information 32. Specifically, the priority calculation unit 23 calculates the priority based on the priority information 32a regarding the boarding position and the priority information 32b regarding the age group.

  For example, when the passenger is an adult seated in the passenger seat S2, the priority calculation unit 23 adds “4” associated with “passenger seat S2” in the priority information 32a to “4” in the priority information 32b. The product “12” obtained by multiplying the score “3” associated with “adult” is calculated as the priority. Note that, as described above, when there are a plurality of passengers, the priority calculation unit 23 calculates a priority for each of the plurality of passengers. Then, the priority calculation unit 23 outputs information indicating the calculated priority to the notification unit 24.

  The notification unit 24 described later determines whether or not there is a passenger other than the driver P1. For example, when the passenger is only a baby and the highest priority is 0, the passenger is the driver. The determination that only P1 is performed may be performed.

  The priority calculation unit 23 detects the boarding position of the passenger based on, for example, the output of a seating sensor (not shown) that detects the presence or absence of the passenger, the analysis result of the captured image by the imaging unit 61, and the like. Can do. Moreover, the priority calculation part 23 can estimate the age group of the passenger currently imaged, for example by analyzing the image captured by the imaging part 61.

  In the above description, the priority calculation unit 23 calculates the priority by multiplying the score of the priority information 32a by the score of the priority information 32b. However, the priority calculation unit 23 is not limited thereto, and may be obtained by adding two scores, for example. The sum may be calculated as the priority. Further, a predetermined mathematical expression for calculating the priority may be set in advance, and the priority calculation unit 23 may calculate the priority by substituting the score into the mathematical expression. Moreover, the priority calculation part 23 may calculate a priority using either the priority information 32a regarding a boarding position, or the priority information 32b regarding an age group.

  The notification unit 24 controls each of a plurality of types of output devices 50 and outputs notification information via the output device 50 to notify the passenger of the vehicle 1 (see FIG. 1).

  More specifically, the notification unit 24 notifies the passenger based on the condition information 31 using a notification method suitable for the type of notification information. For example, when the notification information is emergency information and satisfies the condition A, the notification unit 24 outputs the notification information via the display device 40 and the speaker 51 regardless of the detected line of sight of the passenger. Specifically, when the notification information is emergency information, the notification unit 24 displays the notification information on the display device 40 such as the meter display 41 or the center display 42 and outputs a sound indicating the notification information from the speaker 51. . Thereby, the notification part 24 can notify emergency information which is notification information to a passenger (for example, driver | operator P1) reliably.

  Further, for example, when the notification information is road traffic information and satisfies the condition B, the notification unit 24 displays the notification information on the display device 40 according to the detected line of sight of the passenger (for example, the driver P1). Or a sound indicating notification information is output from the speaker 51. Specifically, when the driver's line of sight of the driver P1 does not move and the display device 40 is viewed, the notification unit 24 displays the road traffic information on the display device 40 being viewed and displays the road traffic information. The sound shown is output from the speaker 51 near the driver's seat S1.

  Further, the notification unit 24 displays the road traffic information on the display device 40 and displays the road traffic information when the display device 40 is present in the predicted movement direction of the visual line when the driver P1 moves. The sound shown is output from the speaker 51 near the driver's seat S1.

  Further, the notification unit 24 visually recognizes the display device 40 when there is a movement of the line of sight of the driver P1 and the display device 40 does not exist in the predicted movement direction of the line of sight, or there is no movement of the line of sight of the driver P1. If not, a sound indicating road traffic information is output from the speaker 51 near the driver's seat S1.

  Thus, the notification unit 24 ensures that the road traffic information as notification information is given to the passenger (for example, the driver P1) even when the display device 40 is not present in the direction of movement of the driver's line of sight, for example. You can be notified.

  The road traffic information is mainly information that should be notified to the driver P1. Therefore, in the above, the sound indicating the road traffic information is output from the speaker 51 near the driver's seat S1, but the present invention is not limited to this. Instead, it may be output from a speaker 51 other than the speaker 51 near the driver's seat S1.

  For example, when the notification information is peripheral information and satisfies the condition C, the notification unit 24 notifies the passenger of the peripheral information based on the priority. Specifically, when there is a passenger other than the driver P1, the surrounding information is notified to the passengers other than the driver P1.

  For example, the notification unit 24 compares priorities calculated for passengers other than the driver P1, selects a passenger with a high priority, and notifies the selected passenger of the surrounding information. . Specifically, when the selected passenger is viewing the display device 40, the notification unit 24 displays the peripheral information on the viewing display device 40. In addition, when the selected occupant does not visually recognize the display device 40, the notification unit 24 outputs a sound indicating the peripheral information from the speaker 51 in the vicinity of the selected occupant's seat. Thereby, the notification part 24 can notify the surrounding information which is notification information reliably with respect to a passenger with high priority.

  Further, when the selected passenger is not waking up, that is, when sleeping, the notifying unit 24 notifies the surrounding information to the passenger with the next highest priority after the selected passenger. Also good.

  Note that the notification unit 24 can determine whether or not the passenger is awake based on, for example, the line of sight of the passenger. For example, the notification unit 24 determines that the occupant is awake when the sight line is detected by the sight line detection unit 21, while the occupant is not awake, that is, when the line of sight is not detected for a predetermined time. Can be determined.

  In addition, the notification unit 24 notifies the driver P1 of the peripheral information when the notification information is the peripheral information and the passenger is only the driver P1. In addition, the method of notifying surrounding information may be the same as the notification method of road traffic information, and is not limited to this, and may be a different notification method.

  As a different notification method, for example, the notification unit 24 has a movement of the line of sight of the driver P1, and when the display device 40 exists in the predicted movement direction of the line of sight, the display unit 40 displays the peripheral information, In the road traffic information, notification with the sound that has been performed may not be performed. This is because the surrounding information is less important than the road traffic information, and if the surrounding information is notified by sound in addition to the display of the display device 40, it may be annoying depending on the driver P1.

  In the above description, the notification unit 24 is an output device 50 that is different from the display device 40 in the case where there is a movement of the line of sight of the driver P1 and the display device 40 does not exist in the predicted movement direction of the line of sight. The output is made through the speaker 51, but the present invention is not limited to this.

  That is, the handle actuator 52 may be used as the output device 50 different from the display device 40. For example, when the display device 40 is not present in the predicted movement direction of the line of sight, the notification unit 24 controls the handle actuator 52 to vibrate the handle 4 so as to notify the passenger of the notification information. It may be.

  Thereby, the notification unit 24 can reliably notify the notification information to the passenger (for example, the driver P1) even when the display device 40 is not present in the direction of movement of the passenger's line of sight, for example. .

  Further, the seat actuator 53 may be used as the output device 50 different from the display device 40. For example, when the display device 40 does not exist in the predicted movement direction of the line of sight, the notification unit 24 controls the seat actuator 53 to vibrate the seat so as to notify the passenger of the notification information. May be.

  As a result, the notification unit 24 provides notification information to the passenger (for example, the driver P1) in the same manner as the speaker 51 and the handle actuator 52 even when the display device 40 is not present in the direction of movement of the passenger's line of sight. ).

  In the above description, the speaker 51, the handle actuator 52, and the seat actuator 53 are given as examples of the output device 50 different from the display device 40. However, the output device 50 is not limited thereto. The output device 50 different from the display device 40 may be one of the speaker 51, the handle actuator 52, and the seat actuator 53, or may be a combination of two or more.

<3. Notification Control Processing of Notification Device According to Embodiment>
Next, a specific processing procedure for controlling the output device 50 of the notification device 10 will be described with reference to FIGS. 6A and 6B. 6A and 6B are flowcharts showing a processing procedure executed by the notification device 10.

  As shown in FIG. 6A, the control unit 20 of the notification device 10 determines whether or not notification information has occurred (step S10). When it is determined that the notification information has not occurred (No at Step S10), the control unit 20 skips the subsequent processing. On the other hand, when it determines with notification information having generate | occur | produced (step S10, Yes), the control part 20 determines whether the generated notification information is emergency information (step S11).

  When it determines with notification information being emergency information (step S11, Yes), the control part 20 displays notification information on the display apparatus 40 (step S12), and outputs the sound which shows notification information from the speaker 51 (step S12). Step S13).

  In the processing of step S12 and step S13, the sound output is performed after the notification information is displayed. However, the present invention is not limited to this, and the display of the notification information and the sound output may be performed simultaneously. Notification information may be displayed after this. The same applies to the case where sound is output subsequent to the display of the notification information in the subsequent processing.

  When it is determined that the notification information is not emergency information (No at Step S11), the control unit 20 determines whether the notification information is road traffic information (Step S14). When it determines with notification information being road traffic information (step S14, Yes), the control part 20 determines whether there exists a movement of the eyes | visual_axis of driver P1 who is a passenger (step S15).

  When it is determined that there is movement of the line of sight of the driver P1 (step S15, Yes), the control unit 20 predicts the movement direction of the line of sight of the driver P1, and the display device 40 is in the predicted movement direction of the line of sight. It is determined whether or not it exists (step S16).

  When it is determined that the display device 40 exists in the line-of-sight movement direction (step S16, Yes), the control unit 20 displays the notification information on the line-of-sight movement destination display device 40 (step S17). Moreover, the control part 20 outputs the sound which shows notification information from the speaker 51 near driver's seat S1 (step S18).

  On the other hand, when it is determined that the display device 40 is not present in the line-of-sight movement direction (No at Step S16), the control unit 20 outputs a sound indicating notification information from the speaker 51 near the driver seat S1 (Step S19). .

  When it is determined that there is no movement of the line of sight of the driver P1 (step S15, No), the control unit 20 determines whether or not the line of sight of the driver P1 is looking toward the display device 40 (step S20). ). When it is determined that the display device 40 is visually recognized (step S20, Yes), the control unit 20 displays the notification information on the display device 40 that is visually recognized (step S21), and drives the sound indicating the notification information. Output from the speaker 51 in the vicinity of the seat S1 (step S22). In addition, when it determines with the control part 20 not visually recognizing the display apparatus 40 (step S20, No), it performs the process of above-described step S19.

  When it is determined that the notification information is not road traffic information (step S14, No), in other words, the control unit 20 is operated when the notification information is information other than emergency information and road traffic information (for example, peripheral information). It is determined whether there is a passenger other than the person P1 (step S23).

  When it is determined that there is no passenger other than the driver P1 (No at Step S23), the control unit 20 performs the processes after Step S15 described above to notify the driver P1 of the peripheral information.

  On the other hand, when it is determined that there is a passenger other than the driver P1 (step S23, Yes), the control unit 20 calculates the priority of the passenger as shown in FIG. 6B (step S24). Next, the control unit 20 selects a passenger based on the calculated priority (step S25). Specifically, the control unit 20 compares the priority and selects a passenger with a higher priority.

  Next, the control unit 20 determines whether or not the selected passenger is awake (step S26). When it is determined that the occupant is awake (step S26, Yes), the control unit 20 determines whether or not the sight line of the occupant is looking toward the display device 40 (step S27).

  When it is determined that the display device 40 is visually recognized (step S27, Yes), the control unit 20 displays the notification information on the visually recognized display device 40 (step S28). When it is determined that the selected passenger is not viewing the display device 40 (No at Step S27), the control unit 20 outputs a sound indicating the notification information from the speaker 51 near the seat of the selected passenger. (Step S29).

  On the other hand, if it is determined that the selected passenger is not awake, that is, if the control unit 20 is sleeping (No in step S26), is there a passenger other than the selected passenger and driver P1? It is determined whether or not (step S30).

  When it is determined that there is another passenger (step S30, Yes), the control unit 20 returns to the process of step S25, and selects a passenger with the next highest priority after the previously selected passenger. And the control part 20 performs the process after above-mentioned step S26 in order to notify notification information to the newly selected passenger.

  When it is determined that there is no other passenger (step S30, No), that is, when a passenger other than the driver P1 is sleeping, for example, the control unit 20 notifies the driver P1 of the surrounding information. Therefore, the process after step S15 mentioned above is performed.

  As described above, the notification device 10 according to the embodiment includes the notification unit 24, the line-of-sight detection unit 21, and the line-of-sight movement prediction unit 22. The notification unit 24 controls a plurality of types of output devices 50 including the display device 40 that can output the notification information, and causes the notification information to be output via the output device 50. The line-of-sight detection unit 21 detects the line of sight of the passenger. The line-of-sight movement prediction unit 22 predicts the movement direction of the line of sight based on the line of sight detected by the line-of-sight detection unit 21. In addition, when the display device 40 does not exist in the line-of-sight movement direction predicted by the line-of-sight movement prediction unit 22, the notification unit 24 outputs notification information via an output device 50 different from the display device 40. Thereby, notification information can be reliably notified with respect to a passenger.

(Modification)
<4. Notification method according to modification>
Next, a modification of the above embodiment will be described. In the following description, the same reference numerals are assigned to configurations common to the embodiment, and descriptions thereof are omitted.

  In the above embodiment, for example, when the notification information is emergency information, the notification information is output via the display device 40 and the speaker 51 regardless of the detected line of sight of the passenger.

  When the notification information is emergency information and satisfies the condition A, the notification unit 24 according to the modification performs guidance control for guiding the passenger's line of sight to the display device 40 to the output device 50, thereby The display device 40 is urged to be visually recognized. The condition A is an example of a predetermined condition.

  FIG. 7 is a diagram illustrating an example of guidance control. As shown in the left diagram of FIG. 7, the notification unit 24 according to the modified example includes, for example, a peripheral portion of the meter display 41 in the output device 50 when the driver P1 who is a passenger does not visually recognize the display device 40. 41a is blinked. As a result, the notification unit 24 can guide the driver P1 to look at the meter display 41 by guiding the line of sight.

  In addition, as guidance control, it is not limited to control which blinks the above-mentioned display apparatus 40. FIG. That is, the guidance control outputs, for example, a voice indicating that the line of sight is guided to the display device 40 such as “please see the display” from the speaker 51 which is the output device 50, or the handle actuator 52 or the seat which is the output device 50. The line of sight may be guided by controlling the actuator 53 to vibrate the handle 4 or the seat.

  And when the driver P1 visually recognizes the meter display 41, for example, as shown in the right diagram of FIG. 7, the notification unit 24 may display the notification information on the currently visible meter display 41 by such guidance control. Good.

  As described above, in the modified example, when the notification information is emergency information and satisfies the condition A, the guidance control is performed on the output device 50, so that the passenger's line of sight is displayed on the display device 40. It is possible to guide the display device 40 to be visually recognized. Thereby, in a modification, emergency information which is notification information can be notified to a passenger (for example, driver P1) more certainly.

<5. Notification control processing of notification device according to modification>
FIG. 8 is a flowchart illustrating a processing procedure executed by the notification device 10 according to the modification. As illustrated in FIG. 8, the control unit 20 of the notification device 10 performs the processes of Step S10 and Step S11, and when it is determined that the notification information is emergency information (Step S11, Yes), the driver who is a passenger It is determined whether or not the line of sight of P1 is viewed toward the display device 40 (step S11a).

  When it is determined that the driver P1 is viewing the display device 40 (Step S11a, Yes), the control unit 20 displays the notification information on the viewing device 40 (Step S12), and the notification information is displayed. The sound shown is output from the speaker 51 (step S13).

  When it determines with the driver | operator P1 not visually recognizing the display apparatus 40 (step S11a, No), the control part 20 performs above-described guidance control with respect to the output device 50 (step S11b).

  Next, the control unit 20 determines whether or not the driver P1 has visually recognized the display device 40 (step S11c). When it determines with the driver | operator P1 having visually recognized the display apparatus 40 (step S11c, Yes), the control part 20 progresses to the process after step S12, and displays the notification information on the display apparatus 40 which is visually recognizing. The sound is output from the speaker 51.

  On the other hand, when it determines with the driver | operator P1 not visually recognizing the display apparatus 40 (step S11c, No), the control part 20 repeats the process of step S11c. In addition, the control part 20 is the process after step S12 in order to notify notification information to the driver | operator P1, when the state determined that the driver | operator P1 is not visually recognizing the display apparatus 40 continues over predetermined time. You may proceed to.

  On the other hand, when it determines with notification information not being emergency information (step S11, No), the control part 20 progresses to the process after step S14 (refer FIG. 6A). In FIG. 8, the notification control processes in steps S <b> 14 to S <b> 30 are collectively shown as step S <b> 40 for simplification of illustration.

  Further effects and modifications can be easily derived by those skilled in the art. Thus, the broader aspects of the present invention are not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications can be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

DESCRIPTION OF SYMBOLS 10 Notification apparatus 20 Control part 21 Eye-gaze detection part 22 Eye-gaze movement prediction part 23 Priority calculation part 24 Notification part 30 Memory | storage part 40 Display apparatus 41 Meter display 42 Center display 43 Rear display 50 Output apparatus 51 Speaker 52 Handle actuator 53 For seats Actuator 61 Imaging unit 100 Notification system

Claims (8)

A notification unit for controlling each of a plurality of types of output devices including a display device capable of outputting notification information, and for outputting the notification information via the output device;
A line-of-sight detection unit for detecting the line of sight of the passenger;
A line-of-sight movement prediction unit that predicts a movement direction of the line of sight based on the line of sight detected by the line-of-sight detection unit, and
The notification unit
When the display device does not exist in the line-of-sight movement direction predicted by the line-of-sight movement prediction unit, the notification information is output via the output device different from the display device. The plurality of types of output devices are:
Including a sound output device capable of outputting sound,
The notification unit
The sound indicating the notification information is output via the sound output device when the display device does not exist in the movement direction of the visual line predicted by the visual line movement prediction unit. Notification device. The plurality of types of output devices are:
Including a handle vibration output device capable of vibrating a vehicle handle,
The notification unit
3. The handle according to claim 1, wherein when the display device does not exist in the movement direction of the line of sight predicted by the line-of-sight movement prediction unit, the handle vibration output device is controlled to vibrate the handle. Notification device. The plurality of types of output devices are:
Including a seat vibration output device capable of vibrating a vehicle seat;
The notification unit
The seat is vibrated by controlling the seat vibration output device when the display device does not exist in the movement direction of the line of sight predicted by the line-of-sight movement prediction unit. The notification device according to one. A plurality of the passengers;
A priority calculating unit that calculates a priority indicating a degree of preferential notification of the notification information to each of the plurality of passengers;
The notification unit
The notification device according to claim 1, wherein the notification information is output via the output device based on the priority calculated by the priority calculation unit. The notification unit
The guidance control for guiding the passenger's line of sight to the display device is performed on the output device when the notification information satisfies a predetermined condition. The notification device described. A plurality of types of output devices including a display device capable of outputting notification information;
Based on the line of sight detected by the line of sight detected by the notifying unit for controlling the plurality of types of output devices and outputting the notification information via the output device, the line of sight detecting unit for detecting the line of sight of the passenger A line-of-sight movement prediction unit that predicts the direction of movement of the line of sight, and a notification device,
The notification unit
When the display device does not exist in the line-of-sight movement direction predicted by the line-of-sight movement prediction unit, the notification information is output via the output device different from the display device. A notification step of controlling each of a plurality of types of output devices including a display device capable of outputting notification information, and outputting the notification information via the output device;
A line-of-sight detection process for detecting the line of sight of the passenger;
A line-of-sight movement prediction step of predicting a movement direction of the line of sight based on the line of sight detected by the line-of-sight detection step,
The notification step includes
When the display device does not exist in the line-of-sight movement direction predicted by the line-of-sight movement prediction step, the notification information is output via the output device different from the display device.

Images