JP7352989B2 - Driving information processing device and driving information processing program - Google Patents

Description

The present invention relates to a driving information processing device and a driving information processing program that execute information processing for driving a vehicle.

Conventionally, a head-mounted display system with a line of sight detection function that performs a simulation of vehicle driving is known as a driving information processing device that performs information processing for driving a vehicle (see, for example, Patent Document 1). . This head-mounted display system with gaze detection function detects the vehicle driver's gaze while running a driving simulation, and stores the detected gaze position in a storage device in synchronization with the driving simulation image. After the driving simulation is completed, the line of sight position read from the storage device and the driving simulation image can be superimposed and displayed on the monitor.

Japanese Patent Application Publication No. 2001-117046

However, in conventional driving information processing devices, the vehicle driver's line of sight is only confirmed after the driving simulation is finished, so the vehicle driver's line of sight is not sufficient for information processing for vehicle driving. The problem is that it is not being utilized.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a driving information processing device and a driving information processing program that can utilize the line of sight of a vehicle driver more effectively than before in information processing for driving a vehicle. .

The driving information processing device of the present invention is a driving information processing device that executes information processing for driving a vehicle, and includes a line of sight detection unit that detects the line of sight of a driver of the vehicle, and a gaze point in a specific range. is executed by the driver, and when specific conditions are met, including the condition that the driver did not gaze at the specific range, specific processing execution means for executing a process, wherein the specific processing execution means continues for a period of time in which the line of sight detected by the line of sight detection unit exceeds a specific time exceeding 0 seconds with respect to the specific range. The presence of the line of sight is treated as a gaze toward the specific range, and the line of sight detected by the line of sight detection unit continues to exist in the specific range for a time equal to or less than the specific time, and the line of sight detection is performed. The fact that the line of sight detected by the unit does not exist in the specific range is not treated as a gaze towards the specific range.

With this configuration, the driving information processing device of the present invention satisfies either one of the conditions that the driver gazed at a specific range and the condition that the driver did not gaze at the specific range. When specific conditions are met, including: be able to.

In the driving information processing device of the present invention, the specific range includes a mirror surface range of a rearview mirror of the vehicle, a mirror surface range of a door mirror of the vehicle, and a range outside the left and right outside of the vehicle other than the door mirror. The specific condition may include a condition that the driver did not gaze at the specific range during a specific period immediately before the vehicle changed its traveling direction.

With this configuration, the driving information processing device of the present invention can improve the driver's awareness of safety confirmation when changing the traveling direction of the vehicle.

In the driving information processing device of the present invention, the specific range is a range of the notification section of the information regarding driving, the notification section is installed in a part other than the vehicle, and the specific condition is: The condition may include a condition that the driver gazed at the specific range.

With this configuration, the driving information processing device of the present invention can improve the driver's awareness of checking the notification section of driving-related information.

In the driving information processing device of the present invention, the specific range is a range that causes distracted driving when the driver looks at it, and the specific condition is the maximum range when the vehicle is being moved by the driving. The condition may include a condition that the driver gazed at the specific range.

With this configuration, the driving information processing device of the present invention can improve the driver's awareness to avoid distracted driving.

In the driving information processing device of the present invention, the specific range is a range of a specific object in a mirror image of the vehicle, and the specific condition is the specific range where the driver gazed. The number of conditions may be included.

With this configuration, the driving information processing device of the present invention can improve the driver's awareness of objects in the mirror image of the vehicle.

In the driving information processing device of the present invention, the specific process may be a process of executing a notification regarding the driving safety.

With this configuration, the driving information processing device of the present invention can improve the driver's awareness of safe driving.

In the driving information processing device of the present invention, the information processing is a simulation of the driving, the driving information processing device includes a display device that displays an image of the simulation, and the specific range is the display device. The specific processing may be a range in the image displayed on a device, and the specific processing may be processing for causing a specific situation to occur in the vehicle in the simulation.

With this configuration, the driving information processing device of the present invention can perform effective simulation.

In the driving information processing device of the present invention, the specific range is a range indicating a location, the specific condition includes a condition that the driver gazed at the specific range, and the specific condition includes a condition that the driver gazed at the specific range. The process may be a process of determining the location indicated by the specific range as the destination of the driving.

With this configuration, the driving information processing device of the present invention determines the driving destination by utilizing the line of sight of the vehicle driver, so that convenience can be improved.

In the driving information processing device of the present invention, the specific range is a range of a notification section of information regarding traffic regulations, the notification section is installed in a part other than the vehicle, and the specific condition is: The specific process may include a condition that the driver has performed gazing at the specific range, and the specific process may be a process of determining the driving route based on the information included in the specific range. .

With this configuration, the driving information processing device of the present invention determines the driving route by utilizing the line of sight of the vehicle driver, so that convenience can be improved.

In the driving information processing device of the present invention, the information processing is processing related to automatic driving of the vehicle, and the specific condition includes a condition that the driver gazed at the specific range, The specific process may be a process of determining the specific range as an avoidance target in the automatic driving.

With this configuration, the driving information processing device of the present invention utilizes the line of sight of the vehicle driver to determine an object to be avoided during automatic driving, so that convenience can be improved.

The driving information processing program of the present invention is a driving information processing program for executing information processing for driving a vehicle, and the condition is that the driver of the vehicle has gazed at a specific range; and a computer as specific processing execution means that executes a specific process in the information processing when any one of the conditions that the driver does not gaze at the specific range is satisfied. The specific processing execution means detects that the line of sight detected by the line of sight detection unit continues to exist in the specific range for a time exceeding a specific time exceeding 0 seconds. The line of sight detected by the line of sight detection unit is treated as a gaze toward the specific range, and the line of sight detected by the line of sight detection unit is continuously present in the specific range for a time equal to or less than the specific time, and the line of sight detected by the line of sight detection unit is The feature is that the absence of a specific range is not treated as a gaze on the specific range.

With this configuration, the computer that executes the driving information processing program of the present invention can meet the conditions that the driver has gazed at a specific range, and the conditions that the driver has not gazed at the specific range. When a specific condition including any one of It can be used effectively.

The driving information processing device and the driving information processing program of the present invention can utilize the line of sight of a vehicle driver more effectively than before in information processing for driving a vehicle.

FIG. 1 is a block diagram of a driving simulator according to a first embodiment of the present invention. 2 is a block diagram of the computer shown in FIG. 1. FIG. 2 is a flowchart showing an example of the operation of the driving simulator shown in FIG. 1 when a simulation target vehicle changes its traveling direction. 2 is a flowchart showing an example of the operation of the driving simulator shown in FIG. 1 when it is determined that the driver has gazed at a specific range. 4 is a flowchart showing an example of the operation of the driving simulator shown in FIG. 1 when the simulation target vehicle changes the direction of travel, which is different from the example shown in FIG. 3; 2 is a flowchart showing an example of the operation of the driving simulator shown in FIG. 1 in response to a road sign that limits maximum speed. 2 is a flowchart showing an example of the operation of the driving simulator shown in FIG. 1 with respect to distracted driving. 2 is a flowchart showing an example of the operation of the driving simulator shown in FIG. 1 with respect to checking a mirror image using a mirror. 2 is a flowchart showing an example of the operation of the driving simulator shown in FIG. 1 when executing a process of determining a driving destination. 2 is a flowchart showing an example of the operation of the driving simulator shown in FIG. 1 when executing processing for determining a driving route. FIG. 2 is a block diagram of a manual driving support device according to a second embodiment of the present invention. FIG. 3 is a block diagram of an automatic driving support device according to a third embodiment of the present invention. 13 is a flowchart illustrating an example of the operation of the automatic driving support device shown in FIG. 12 when executing a process of determining an avoidance target in automatic driving.

Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
First, the configuration of a driving simulator as a driving information processing device according to a first embodiment of the present invention will be described.

FIG. 1 is a block diagram of a driving simulator according to this embodiment.

A driving simulator 10 shown in FIG. 1 is a device that executes a simulation of driving a car as information processing for driving a car as a vehicle. The driving simulator 10 includes a vehicle operating device 20 for operating a simulation target vehicle (hereinafter referred to as "simulation target vehicle"), a head-mounted display 30 mounted on the head of the driver of the simulation target vehicle, It includes a computer 40 that can communicate with a vehicle operating device 20 and a head-mounted display 30.

The vehicle operating device 20 includes, for example, a steering handle, a clutch pedal, an accelerator pedal, a brake pedal, a parking brake, a transmission operating lever, and a direction indicator operating device.

The head-mounted display 30 includes a display unit 31 as a display device such as an LCD (Liquid Crystal Display) that displays driving simulation images (hereinafter referred to as “simulation images”), and a The vehicle includes a line-of-sight detection unit 32 that detects the driver's line of sight. As the head mounted display 30, for example, "FOVE 0" manufactured by FOVE may be adopted. The simulation image may be expressed in any of VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality). Note that when the simulation image is expressed in either AR or MR, the head-mounted display 30 needs to include a camera for photographing the front of the driver's face.

As a method for detecting the line of sight by the line of sight detection unit 32, any known method for detecting the line of sight may be employed.

FIG. 2 is a block diagram of computer 40.

As shown in FIG. 2, the computer 40 has an operation unit 41, which is an input device such as a keyboard and a mouse, through which various operations are input, a microphone 42, and a network such as a LAN (Local Area Network) and the Internet. Alternatively, a communication unit 43, which is a communication device that directly communicates with an external device by wire or wirelessly without going through a network, and a non-volatile device such as a semiconductor memory or HDD (Hard Disk Drive) that stores various information. It includes a storage section 44 that is a storage device, and a control section 45 that controls the entire computer 40. The computer 40 may be configured by, for example, a PC (Personal Computer).

The storage unit 44 stores a driving simulation program 44a as a driving information processing program for executing a simulation of driving a car. The driving simulation program 44a may be installed in the computer 40 at the manufacturing stage of the computer 40, for example, or may be stored in an external storage such as a USB (Universal Serial Bus) memory, a CD (Compact Disk), or a DVD (Digital Versatile Disk). It may be additionally installed on the computer 40 from a medium or may be additionally installed on the computer 40 from a network.

The control unit 45 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory) that stores programs and various data, and a RAM (Random Access Memory) that is used as a work area for the CPU. There is. The CPU executes a program stored in the ROM or the storage unit 44.

The control unit 45 realizes the driving simulation means 45a by executing the driving simulation program 44a. The driving simulation means 45a satisfies a specific condition, including one of the following: the driver has gazed at a specific range; and the driver has not gazed at a specific range. In this case, a specific process is executed in the simulation, so it constitutes a specific process execution means of the present invention.

Note that the range of gaze target candidates in the simulation image is set in advance in the driving simulation program 44a for each gaze target candidate.

Next, the operation of the driving simulator 10 will be explained.

The driving simulation means 45a displays a simulation image on the display section 31, and also reflects the operation on the vehicle operating device 20 and the line of sight detected by the line of sight detection section 32 in the simulation. Further, the driving simulation means 45a displays information on the display unit 31 according to the direction of the driver's face, similar to the way that the driver's field of view changes depending on the direction of the driver's face when driving a real car. Change the simulation image.

An example in which the line of sight detected by the line of sight detection unit 32 is reflected in the simulation will be described below.

First, the operation of the driving simulator 10 when the simulation target vehicle changes the direction of travel due to a lane change, left turn, right turn, etc. will be described.

FIG. 3 is a flowchart showing an example of the operation of the driving simulator 10 when the simulation target vehicle changes its traveling direction.

The driving simulation means 45a executes the operation shown in FIG. 3 when the simulation target vehicle changes the traveling direction in response to an operation on the vehicle operating device 20. Note that the driving simulation means 45a executes the operation shown in FIG. 3, for example, when an operation is performed on the vehicle operating device 20 to cause the direction indicator to indicate the direction in which the simulation target vehicle should change its traveling direction. Alternatively, the simulation target vehicle may actually start changing its traveling direction after an operation is performed on the vehicle operating device 20 to instruct the turn signal in which direction the simulation target vehicle changes its traveling direction. Alternatively, the operation shown in FIG. 3 may be performed.

As shown in FIG. 3, the driving simulation means 45a determines whether or not the driver gazed at the mirror surface area of the rearview mirror of the simulation target vehicle during a specific period immediately before the simulation target vehicle changed its traveling direction. (S101).

FIG. 4 is a flowchart showing an example of the operation of the driving simulator 10 when it is determined that the driver has gazed at a specific range.

The driving simulation means 45a executes the operation shown in FIG. 4 every time among the gaze target candidates in the simulation image, the gaze detected by the gaze detection unit 32 is within the range of the candidate itself. .

As shown in FIG. 4, the driving simulation means 45a identifies, as the current candidate, a gaze target candidate in the simulation image whose gaze detected by the gaze detection unit 32 is within the range of the candidate itself. (S111).

Next, the driving simulation means 45a determines whether the range in which the line of sight detected by the line of sight detection unit 32 exists has been changed from the current candidate range (S112).

If the driving simulation means 45a determines in S112 that the range in which the line of sight detected by the line of sight detection unit 32 exists has not been changed from the range of the current candidate, the driving simulation means 45a determines that the line of sight detected by the line of sight detection unit 32 exists in the range of the current candidate. The time continuously existing in the range is acquired (S113).

Next, the driving simulation means 45a determines whether the time acquired in S113 exceeds a specific time (hereinafter referred to as "threshold time"), such as 1 second, which exceeds 0 seconds (S114).

If the driving simulation means 45a determines in S114 that the time acquired in S113 does not exceed the threshold time, it executes the process in S112.

If the driving simulation means 45a determines in S114 that the time acquired in S113 exceeds the threshold time, it determines that the driver has gazed at the current candidate range (S115).

When the driving simulation means 45a determines in S112 that the range in which the line of sight detected by the line of sight detection unit 32 exists has been changed from the current candidate range, or executes the process in S115, it ends the operation shown in FIG. 4. .

Since the driving simulation means 45a executes the operation shown in FIG. 4, the fact that the line of sight detected by the line of sight detection unit 32 continues to exist in a specific range for a time exceeding the threshold time is determined as a gaze on this range. The line of sight detected by the line of sight detection unit 32 continues to exist in this range for a time equal to or less than the threshold time, and the line of sight detected by the line of sight detection unit 32 does not exist in this range. , it is not treated as a gaze on this range.

As shown in FIG. 3, when the driving simulation means 45a determines in S101 that the driver gazed at the area of the mirror surface of the rearview mirror during a specific period immediately before the simulation target vehicle changed the direction of travel, The operation shown in 3 ends.

If the driving simulation means 45a determines in S101 that the driver has not gazed at the mirror surface area of the rearview mirror during a specific period immediately before the simulation target vehicle changes the direction of travel, it performs a safety check using the rearview mirror. The display unit 31 issues a notification to warn that the user has not done so (S102), and the operation shown in FIG. 3 ends.

The driving simulation means 45a may perform the operation shown in FIG. 5 instead of the operation shown in FIG.

FIG. 5 is a flowchart showing an example of the operation of the driving simulator 10 when the simulation target vehicle changes the direction of travel, which is different from the example shown in FIG. 3.

As shown in FIG. 5, the driving simulation means 45a executes a process similar to the process in S101 (S121).

If the driving simulation means 45a determines in S121 that the driver did not gaze at the area of the mirror surface of the rearview mirror during a specific period immediately before the simulation target vehicle changes the traveling direction, the driving simulation means 45a determines that the driver does not gaze at the mirror surface area of the rearview mirror during a specific period immediately before the simulation target vehicle changes the traveling direction. In the simulation, a situation in which the simulation target vehicle collides with another vehicle is caused to occur in the simulation target vehicle (S122), and the operation shown in FIG. 5 ends. Here, examples of other vehicles include bicycles and automobiles such as motorcycles.

In the operation shown in FIG. 5, the driving simulation means 45a causes a situation in which the simulation target vehicle collides with another vehicle. However, the driving simulation means 45a may generate a situation in which the simulation target vehicle is about to collide with another vehicle, instead of a situation in which the simulation target vehicle collides with another vehicle.

In the operation shown in FIG. 5, the driving simulation means 45a may further issue a notification warning that the safety check using the rearview mirror is not performed, similarly to the operation shown in FIG.

In the operations shown in FIGS. 3 and 5, the driving simulation means 45a executes the operations shown in FIG. 3 or FIG. 5 based on whether or not the range of the mirror surface of the rearview mirror is gazed at. Similarly, the driving simulation means 45a determines whether or not the range of the mirror surface of the left and right side mirrors of the simulation target vehicle, which is the side mirror in which the simulation target vehicle changes its traveling direction, has been gazed at. An action similar to the action shown in 5 may be executed, or whether the gaze is directed at an area other than the door mirrors on the outside of the left and right sides of the simulation target vehicle where the simulation target vehicle changes its direction of travel. Based on whether or not the information is displayed, an operation similar to the operation shown in FIG. 3 or FIG. 5 may be performed. Here, the driving simulation means 45a sets different threshold times for gazing at the range of the mirror surface of the rearview mirror, gazing at the range of the mirror surface of the door mirror, and gazing at the range outside the left and right outside of the simulation target vehicle other than the door mirror. You can let it go.

Next, an explanation will be given of the operation of the driving simulator 10 with respect to road signs as a notification unit of driving information.

FIG. 6 is a flowchart showing an example of the operation of the driving simulator 10 in response to a road sign that limits the maximum speed.

The driving simulation means 45a executes the operation shown in FIG. 6 when the simulation target vehicle moves.

As shown in FIG. 6, the driving simulation means 45a determines whether the simulation target vehicle has stopped (S131).

If the driving simulation means 45a determines in S131 that the simulation target vehicle has not stopped, it determines whether the simulation target vehicle has passed near a road sign that limits the maximum speed (S132). Here, the road sign is installed in a portion other than the simulation target vehicle, for example, near a road on the driving route of the simulation target vehicle.

If the driving simulation means 45a determines in S132 that the simulation target vehicle has passed near a road sign that limits the maximum speed, it specifies the maximum speed indicated on this road sign as the current speed limit (S133).

If the driving simulation means 45a determines in S132 that the simulation target vehicle has not passed near a road sign that limits the maximum speed, or if the process in S133 is executed, the simulation target vehicle runs at a speed exceeding the current speed limit. It is determined whether or not it has been executed (S134).

When the driving simulation means 45a determines in S134 that the simulation target vehicle is traveling at a speed exceeding the current speed limit, the driving simulation means 45a determines whether the driver was gazing at the range of the road sign indicating the current speed limit. It is determined whether or not (S135).

If the driving simulation means 45a determines in S135 that the driver was gazing at the range of the road sign indicating the current speed limit, it issues a notification warning of speeding via the display unit 31 (S136).

If the driving simulation means 45a determines in S135 that the driver did not pay attention to the range of the road sign that indicated the current speed limit, the driving simulation unit 45a warns the driver of the speeding violation and not checking the road sign. The display unit 31 issues a notification to the user (S137).

When the driving simulation means 45a determines in S134 that the simulation target vehicle is not traveling at a speed exceeding the current speed limit, or executes the process of S136 or S137, it executes the process of S131.

When the driving simulation means 45a determines in S131 that the simulation target vehicle has stopped, it ends the operation shown in FIG. 6.

In the operation shown in FIG. 6, the driving simulation means 45a issues a notification regarding driving safety when the driver does not gaze at the range of the road sign (NO in S135) (S134). However, the driving simulation means 45a may cause a specific situation to occur to the simulation target vehicle in the simulation, as in the operation shown in FIG. 5, when the driver does not pay attention to the range of the road sign. good.

Note that the operation shown in FIG. 6 is an operation when the notification unit of driving information is a road sign that limits the maximum speed. However, the notification unit for driving information may be a road sign other than a road sign that limits the maximum speed, as long as it is installed in a part other than the simulation target vehicle, or, for example, if it is installed in a part other than the simulation target vehicle. It may also be something other than a road sign, such as a signboard informing traffic regulations.

The driving simulation means 45a may set the threshold time of gazing on the range of the notification unit of driving-related information to be different from any of the threshold time of gazing mentioned above, such as gazing on the mirror surface of a rearview mirror, for example.

Next, the operation of the driving simulator 10 regarding distracted driving will be explained.

FIG. 7 is a flowchart showing an example of the operation of the driving simulator 10 regarding distracted driving.

The driving simulation means 45a executes the operation shown in FIG. 7 when the simulation target vehicle moves.

As shown in FIG. 7, the driving simulation means 45a determines whether the simulation target vehicle has stopped (S141).

If the driving simulation means 45a determines in S141 that the simulation target vehicle is not stopped, it determines whether the driver has gazed at a range that would constitute distracted driving if gazed by the driver (hereinafter referred to as the "distracted range"). It is determined whether or not (S142). Here, the range of distraction may include, for example, the range of a car navigation system inside the simulation target car, the range of a mobile terminal inside the simulation target car, and the like.

If the driving simulation means 45a determines in S142 that the driver has gazed at the inattentive range, it causes a specific situation to occur in the simulation target vehicle in the simulation (S143). For example, the driving simulation means 45a may simulate a situation in which the simulation target vehicle collides with an object other than the simulation target vehicle, such as another vehicle, when the driver looks at the inattentive area. It is also possible to cause the simulated vehicle to encounter a situation in which the simulated vehicle is likely to collide with something other than the simulated vehicle, such as a pedestrian jumping out in front of the simulated vehicle. Also good.

When the driving simulation means 45a determines in S142 that the driver did not gaze at the inattentive range, or when the process in S143 ends, it executes the process in S141.

When the driving simulation means 45a determines in S141 that the simulation target vehicle has stopped, it ends the operation shown in FIG. 7.

In the operation shown in FIG. 7, the driving simulation means 45a causes a specific situation to occur in the simulation target vehicle in the simulation when the driver gazes at the inattentive range (S142: YES) (S143). However, the driving simulation means 45a may issue a notification regarding driving safety, such as a notification warning against distracted driving, as shown in the operation shown in FIG. good.

The driving simulation means 45a may set the threshold time for gazing on the area of inattentiveness to be different from any of the threshold times for gazing described above, such as gazing on the mirror surface area of a rearview mirror, for example.

Next, a description will be given of the operation of the driving simulator 10 for checking mirror images of mirrors of the simulation target vehicle, such as rearview mirrors and door mirrors.

FIG. 8 is a flowchart showing an example of the operation of the driving simulator 10 for checking a mirror image using a mirror.

The driving simulation means 45a executes the operation shown in FIG. 8 when the driver starts driving the simulation target vehicle.

As shown in FIG. 8, the driving simulation means 45a determines whether or not the driving of the simulation target vehicle has ended (S151).

If the driving simulation means 45a determines in S151 that the driving of the simulation target vehicle has not finished, it determines whether the driver has gazed at the range of the target to be gazed at in the mirror image (S152). Here, the object to be watched in the mirror image may be, for example, a vehicle other than the simulation target vehicle.

If the driving simulation means 45a determines in S152 that the driver has gazed at the range of the target to be gazed at in the mirror image, it sets a flag to be managed in association with this target (S153). Here, the driving simulation means 45a manages a flag in association with each target to be watched in the mirror image. A flag that is managed in association with an object to be watched in a mirror image is in a fallen state by default.

When the driving simulation means 45a determines in S152 that the driver has not gazed at the range of the target to be gazed at in the mirror image, or when the process in S153 ends, it executes the process in S151.

When the driving simulation means 45a determines in S151 that the driving of the simulation target vehicle has ended, it counts the number of objects to be watched in the mirror image in the current simulation that are associated with flags in a raised state ( S154).

Next, the driving simulation means 45a causes the display unit 31 to issue a notification regarding driving safety according to the number counted in S154 (S155). For example, the driving simulation means 45a may notify how many objects the driver has gazed at among the objects to be gazed at in the mirror image in the current simulation.

After the process of S155, the driving simulation means 45a ends the operation shown in FIG.

In the operation shown in FIG. 8, the driving simulation means 45a executes a notification regarding driving safety according to the number of objects that the driver has gazed at among the range of objects to be gazed at in the mirror image (S155). ). However, the driving simulation means 45a performs a specific situation in the simulation, such as the operation shown in FIG. It may also be generated for the simulation target vehicle.

The driving simulation means 45a may set the threshold time for gazing at the range of the object to be gazed at in the mirror image to be different from any of the threshold time for gazing mentioned above, such as for the range of the mirror surface of the rearview mirror, for example. .

Next, the operation of the driving simulator 10 when executing the process of determining the driving destination will be described.

FIG. 9 is a flowchart showing an example of the operation of the driving simulator 10 when executing the process of determining the driving destination.

The driving simulation means 45a executes the operation shown in FIG. 9 when the driver starts driving the simulation target vehicle.

As shown in FIG. 9, the driving simulation means 45a determines whether or not the driving of the simulation target vehicle has ended (S161).

If the driving simulation means 45a determines in S161 that the driving of the simulation target vehicle has not finished, it determines whether the driver has gazed at the range indicating the location (S162). Here, the range indicating the location may be, for example, a landmark drawn as a landscape in the simulation image, such as a TV tower drawn in a place far away from the simulation target vehicle. In addition, the range indicating the location may be, for example, a place name written on a notification part, such as a road sign, installed in a part other than the simulation target vehicle.

When the driving simulation means 45a determines in S162 that the driver has gazed at a range indicating a location, the driving simulation means 45a sets conditions for determining the location indicated by this range as the driving destination (hereinafter referred to as "destination determination conditions"). ) is satisfied (S163). Here, the destination determination conditions include, for example, that the driver gazes at the range indicating the location for more than a specific time, such as 5 seconds, and that the driver performs a specific operation to determine the destination. It is conceivable that specific words such as "determine a destination" be input into the microphone 42.

If the driving simulation means 45a determines in S163 that the destination determination condition is satisfied, it determines the location indicated by the range determined in S162 as being gazed by the driver as the driving destination (S164).

When the driving simulation means 45a determines in S162 that the driver did not gaze at the range indicating the location, or determines in S163 that the destination determination condition is not satisfied, or executes the process in S164, the driving simulation unit 45a performs the process in S161. Execute processing.

When the driving simulation means 45a determines in S161 that the driving of the simulation target vehicle has ended, it ends the operation shown in FIG.

The driving simulation means 45a may set the threshold time of gazing on the range indicating the location different from any of the threshold time of gazing mentioned above, such as the gazing on the mirror surface of the rearview mirror, for example.

Next, the operation of the driving simulator 10 when executing the process of determining a driving route will be described.

FIG. 10 is a flowchart showing an example of the operation of the driving simulator 10 when executing the process of determining a driving route.

The driving simulation means 45a executes the operation shown in FIG. 10 when the driver starts driving the simulation target vehicle.

As shown in FIG. 10, the driving simulation means 45a determines whether or not the driving of the simulation target vehicle has ended (S171).

If the driving simulation means 45a determines in S171 that the driving of the simulation target vehicle has not finished, it determines whether the driver has gazed at the range of the notification section of information regarding traffic regulations (S172). Here, the notification unit for information regarding traffic regulations is installed in a portion other than the simulation target vehicle, for example, near a road on the driving route of the simulation target vehicle. For example, a signboard that informs of traffic regulations associated with a marathon event, etc. may be used as a notification section for information regarding traffic regulations.

When the driving simulation means 45a determines in S172 that the driver has gazed at the range of the notification section for information regarding traffic regulations, it proposes a driving route on the display section 31 based on the information included in this range (S173). ). For example, the driving simulation means 45a proposes a detour route that takes into account congestion predicted due to traffic regulations, instead of a standard route.

Next, the driving simulation means 45a determines whether conditions for determining the route proposed in S173 as the driving route (hereinafter referred to as "route determining conditions") are satisfied (S174). Here, the route determination conditions include, for example, that a specific operation for determining the route is input to the operation unit 41, that specific words such as "determine route" are input to the microphone 42, etc. is possible.

If the driving simulation means 45a determines in S174 that the route determination conditions are satisfied, it determines the route proposed in S173 as the driving route (S175).

The driving simulation means 45a determines in S172 that the driver did not gaze at the range of the notification section of information regarding traffic regulations, determines in S174 that the route determination condition is not satisfied, or executes the process of S175. Then, the process of S171 is executed.

When the driving simulation means 45a determines in S171 that the driving of the simulation target vehicle has ended, it ends the operation shown in FIG. 10.

In addition, in the operation shown in FIG. 10, the driving simulation means 45a determines the driving route according to the driver's instructions (S173 to S175). However, the driving simulation means 45a may automatically determine the driving route based on information included in the range determined in S172 to have been gazed by the driver.

The driving simulation means 45a may set the threshold time for gazing at the range of the notification section of information regarding traffic regulations to be different from any of the threshold time for gazing described above, such as gazing at the mirror surface of a rearview mirror, for example.

As explained above, when the driving simulator 10 satisfies specific conditions including the condition that the driver did not gaze at a specific range (NO in S101, NO in S121, or NO in S135) Next, specific processing is executed in the simulation of driving a car (S102, S122, or S137). Further, when a specific condition including the condition that the driver gazed at a specific range is satisfied (YES in S142, YES in S152, YES in S162, or YES in S172), the driving simulator 10 A specific process is executed in the simulation of driving a car (S143, S155, S164, or S175). Therefore, the driving simulator 10 can utilize the driver's line of sight more effectively than before in the simulation of driving a car.

When the driving simulator 10 executes the operation shown in FIG. 3, the driver does not gaze at a specific range during a specific period immediately before the simulation target vehicle changes its direction of travel (NO in S101). Since a notification regarding driving safety is executed (S102), it is possible to improve the driver's awareness of safety confirmation when changing the traveling direction of the simulation target vehicle, that is, the driver's awareness of safe driving. can. For example, a driver attempts to turn the simulation vehicle to the left, checks the safety in the rearview mirror, decelerates the simulation vehicle, and checks the outside left side of the simulation vehicle other than the outside mirror, but Because of neglecting to check the safety using the side mirrors of the simulated vehicle, the simulated vehicle collides with the motorcycle when the simulated vehicle uses its steering wheel to make a left turn without noticing the motorcycle approaching the simulated vehicle from the left rear of the simulated vehicle. When this situation occurs, the driving simulator 10 issues a notification on the display unit 31 warning that the safety check using the left door mirror has not been performed, thereby improving the safety of the driver when the simulation target vehicle turns left. You can improve your awareness of confirmation.

When the driving simulator 10 executes the operation shown in FIG. 5, the driver does not gaze at a specific range during a specific period immediately before the simulation target vehicle changes its direction of travel (NO in S121). In addition, since a specific situation is generated for the simulation target vehicle in the simulation (S122), it is possible to execute an effective simulation, and it is also possible to confirm the safety of the driver when changing the traveling direction of the simulation target vehicle. Awareness can be improved.

When executing the operation shown in FIG. 6, the driving simulator 10 executes a notification regarding driving safety when the driver does not gaze at the range of the notification unit for driving information (NO in S135). (S137) Therefore, it is possible to improve the driver's awareness of checking the notification section of driving information, that is, the driver's awareness of safe driving. For example, if the maximum speed is 60 km/h and the simulated car passes near a road sign that limits the maximum speed to 50 km/h, the driver continues to drive the simulated car at a speed exceeding 50 km/h. If the driver is not looking at the range of the road sign indicating the current speed limit, the driving simulator 10 issues a notification to the driver that the driver has not checked the road sign.

The driving simulator 10 executes an effective simulation when causing a specific situation to occur in the simulation target vehicle in the simulation when the driver does not pay attention to the range of the notification unit of driving information. At the same time, it is possible to improve the driver's awareness of checking the notification section for information related to driving.

When executing the operation shown in FIG. 7, the driving simulator 10 causes a specific situation to occur in the simulation target vehicle in the simulation (S143) when the driver gazes at the inattentive range (YES in S142). ), it is possible to perform an effective simulation and improve the driver's awareness to avoid distracted driving. For example, the driving simulator 10 causes a mobile terminal placed in a mobile holder installed near the instruments of the simulation target vehicle to receive a message, and when the driver views this message, the driver receives the message. By causing a pedestrian to jump out in front of the simulated vehicle the moment they see it, it is possible to give the driver a realistic experience that makes them feel cold, and improves the driver's awareness to avoid distracted driving. .

When the driving simulator 10 issues a notification regarding driving safety when the driver looks at the area of inattentiveness, the driving simulator 10 evaluates the driver's awareness of not inattentive driving, that is, the driver's awareness of safe driving. Awareness can be improved.

When the driving simulator 10 executes the operation shown in FIG. 8, the driving simulator 10 issues a notification regarding driving safety according to the number of objects that the driver has gazed at out of the range of objects to be gazed at in the mirror image. (S155) Therefore, the driver's awareness of the object in the mirror image, that is, the safe The driver's awareness of driving can be improved.

The driving simulator 10 is effective when causing a specific situation to occur in the simulation target vehicle in the simulation according to the number of objects that the driver gazes at out of the range of objects to be gazed at in the mirror image. In addition to being able to perform simulations, it is also possible to improve the driver's awareness of objects that are outside the normal field of vision.

The driving simulator 10 utilizes the driver's line of sight to determine the driving destination (S164) and the driving route (S175), thereby improving convenience.

Note that although the display device that displays the simulation image is realized by a head-mounted display in this embodiment, it may be realized by a device other than the head-mounted display.

Although the line-of-sight detection unit that detects the driver's line-of-sight in the simulation image is realized by a head-mounted display in this embodiment, it may be realized by a device other than the head-mounted display.

(Second embodiment)
First, the configuration of a manual driving support device as a driving information processing device according to a second embodiment of the present invention will be described.

FIG. 11 is a block diagram of manual driving support device 210 according to this embodiment.

The manual driving support device 210 shown in FIG. 11 is a device that is installed in a car (not shown) driven by a driver and supports the driver in driving the car. The manual driving support device 210 includes an operation unit 211 that is an input device such as a keyboard and a mouse for inputting various operations, a display unit 212 that is a display device such as an LCD that displays various information, and a driver's face. A camera 213 for photographing the front of the driver, a line-of-sight detection unit 214 for detecting the driver's line of sight, and a microphone 215 are connected directly via a network such as a LAN or the Internet, or directly by wire or wirelessly without going through a network. , a communication unit 216 that is a communication device that communicates with external devices, a storage unit 217 that is a non-volatile storage device such as a semiconductor memory or HDD that stores various information, and the entire manual driving support device 210. The control unit 218 is also provided with a control unit 218. The manual driving support device 210 is capable of communicating with an external device such as a car equipped with the manual driving support device 210 via the communication unit 216.

The display unit 212, camera 213, and line of sight detection unit 214 may be realized by a head-mounted display as in the first embodiment. However, if the display unit 212, camera 213, and line of sight detection unit 214 are implemented by a head-mounted display, the image displayed on the display unit 212 is expressed in either AR or MR.

The storage unit 217 stores a manual driving support program 217a as a driving information processing program for executing information processing to support driving of a car by a driver. The manual driving support program 217a may be installed in the manual driving support device 210 at the manufacturing stage of the manual driving support device 210, for example, or may be downloaded from an external storage medium such as a USB memory, CD, or DVD to the manual driving support device 210. It may be additionally installed on the manual driving support device 210 from the network, or may be additionally installed on the manual driving support device 210 from the network.

The control unit 218 includes, for example, a CPU, a ROM that stores programs and various data, and a RAM that is used as a work area for the CPU. The CPU executes a program stored in the ROM or storage unit 217. That is, manual driving support device 210 is a computer.

The control unit 218 implements manual driving support means 218a by executing the manual driving support program 217a. The manual driving support means 218a determines a specific condition including either one of a condition that the driver gazed at a specific range and a condition that the driver did not gaze at a specific range. If the condition is satisfied, a specific process is executed in information processing for assisting the driver in driving a car, and thus constitutes a specific process execution means of the present invention.

Note that the manual driving support means 218a specifies the range of the target candidate in the image captured by the camera 213 by performing image processing on the image captured by the camera 213 based on a specific standard.

Next, the operation of the manual driving support device 210 will be explained.

The manual driving support means 218a, like the driving simulation means 45a (see FIG. 2), calculates the range of the mirror surface of the rearview mirror of the vehicle and the mirror surface of the door mirror of the vehicle during a specific period immediately before the vehicle changes its direction of travel. If the driver does not gaze at a specific range, such as a range outside the left and right sides of the vehicle other than the door mirrors, the display unit 212 issues a notification regarding driving safety, thereby reducing the progress of the vehicle. It is possible to improve the driver's awareness of safety confirmation when changing direction, that is, the driver's awareness of safe driving.

The manual driving support means 218a, like the driving simulation means 45a, allows the display unit 212 to issue a notification regarding driving safety when the driver does not gaze at the range of the notification unit for driving information. Accordingly, it is possible to improve the driver's awareness of checking the driving-related information notification section installed in a part other than the automobile, that is, the driver's awareness of safe driving.

Similar to the driving simulation means 45a, the manual driving support means 218a prevents distracted driving by displaying a notification regarding driving safety on the display unit 212 when the driver gazes at the distracted range. In other words, the driver's awareness of safe driving can be improved.

The manual driving support means 218a, like the driving simulation means 45a, can improve convenience by determining the driving destination and driving route by utilizing the driver's line of sight. can. The manual driving support means 218a inputs the determined destination and route into the car navigation system inside the automobile.

When the driver gazes at a range indicating a location, the manual driving support means 218a transmits traffic congestion information on the route to the location indicated by the range at which the driver gazes, via the communication unit 216, to the Internet. A search result is displayed on the display unit 212 by searching from above, or a location indicated by the range of gaze performed by the driver, such as information on a restaurant existing in the location indicated by the range of gaze performed by the driver. It is also possible to search for information on the Internet via the communication unit 216 and display the search results on the display unit 212.

Of the images taken by the camera 213, the manual driving support means 218a may save an image of the range where the driver gazed. For example, the manual driving support means 218a may save a range surrounded by four points where the driver gazed.

(Third embodiment)
First, the configuration of an automatic driving support device as a driving information processing device according to a third embodiment of the present invention will be described.

FIG. 12 is a block diagram of automatic driving support device 310 according to this embodiment.

The automatic driving support device 310 shown in FIG. 12 is a device that is installed in a car (not shown) that performs automatic driving and supports automatic driving by the car. The automatic driving support device 310 includes an operation unit 311 that is an input device such as a keyboard and a mouse for inputting various operations, a display unit 312 that is a display device such as an LCD that displays various information, and a driver's face. A camera 313 for photographing the front of the driver, a line-of-sight detection unit 314 for detecting the driver's line of sight, and a microphone 315 are connected directly via a network such as a LAN or the Internet, or directly by wire or wirelessly without going through a network. , a communication unit 316 that is a communication device that communicates with external devices, a storage unit 317 that is a non-volatile storage device such as a semiconductor memory or HDD that stores various information, and the entire automatic driving support device 310. The control unit 318 is also provided with a control unit 318. The automatic driving support device 310 can communicate with an external device such as a car equipped with the automatic driving support device 310 via the communication unit 316.

The display unit 312, camera 313, and line of sight detection unit 314 may be realized by a head-mounted display as in the first embodiment. However, when the display unit 312, camera 313, and line of sight detection unit 314 are implemented by a head-mounted display, the image displayed on the display unit 312 is expressed in either AR or MR.

The storage unit 317 stores an automatic driving support program 317a as a driving information processing program for executing information processing for supporting automatic driving by a car. The automatic driving support program 317a may be installed in the automatic driving support device 310 at the manufacturing stage of the automatic driving support device 310, for example, or may be downloaded from an external storage medium such as a USB memory, CD, or DVD to the automatic driving support device 310. The automatic driving support device 310 may be additionally installed on the automatic driving support device 310 from the network.

The control unit 318 includes, for example, a CPU, a ROM that stores programs and various data, and a RAM that is used as a work area for the CPU. The CPU executes a program stored in the ROM or storage unit 317. That is, automatic driving support device 310 is a computer.

The control unit 318 implements automatic driving support means 318a by executing the automatic driving support program 317a. The automatic driving support means 318a determines a specific condition including either one of a condition that the driver gazed at a specific range and a condition that the driver did not gaze at a specific range. If the condition is satisfied, a specific process is executed in information processing for supporting automatic driving by a car, and thus constitutes a specific process execution means of the present invention.

Note that the automatic driving support means 318a specifies the range of candidates for the gaze target in the image captured by the camera 313 by performing image processing on the image captured by the camera 313 based on a specific standard.

Next, the operation of the automatic driving support device 310 will be explained.

The automatic driving support device 310 can determine an object to be avoided during automatic driving. The operation of the automatic driving support device 310 when executing the process of determining an avoidance target in automatic driving will be described below.

FIG. 13 is a flowchart illustrating an example of the operation of the automatic driving support device 310 when executing the process of determining an avoidance target in automatic driving.

The automatic driving support means 318a executes the operation shown in FIG. 13 when automatic driving by a car is started.

As shown in FIG. 13, the automatic driving support means 318a determines whether automatic driving has ended (S321).

When the automatic driving support means 318a determines in S321 that automatic driving has not ended, it determines whether the driver has gazed at a specific range (S322).

When the automatic driving support means 318a determines in S322 that the driver has gazed at a specific range, it determines this range as an avoidance target in automatic driving (S323), and uses the target determined in S323 as an avoidance target in automatic driving. It is input into the automatic driving system of the automobile as an object to be avoided (S324).

When the automatic driving support means 318a determines in S322 that the driver has not gazed at a specific range, or when the process in S324 ends, it executes the process in S321.

When the automatic driving support means 318a determines in S321 that automatic driving has ended, it ends the operation shown in FIG. 13.

As described above, the automatic driving support device 310 determines the target to be avoided during automatic driving by utilizing the driver's line of sight (S323), so that convenience can be improved.

The automatic driving support means 318a, like the manual driving support means 218a (see FIG. 11), utilizes the driver's line of sight to determine the driving destination and the driving route. Convenience can be improved. The automatic driving support means 318a inputs the determined destination and route into the automatic driving system of the automobile.

Similarly to the manual driving support means 218a, when the driver gazes at a range indicating a location, the automatic driving support means 318a detects traffic jams on the route to the location indicated by the range where the driver gazes. Information can be searched for on the Internet via the communication unit 316 and the search results displayed on the display unit 312, or information on restaurants existing in the area indicated by the range of gaze performed by the driver can be displayed. It is also possible to search the Internet for information on the location indicated by the execution range via the communication unit 316 and display the search results on the display unit 312.

Similarly to the manual driving support means 218a, the automatic driving support means 318a may save the image of the range where the driver gazed among the images taken by the camera 313.

In each of the embodiments described above, driving of a vehicle is explained. However, the present invention is also applicable to the operation of vehicles other than vehicles, such as the operation of ships and aircraft.

The present invention is configured to be applicable to xR technology, which is a general term for any reality technology such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality). It's okay to be.

10 Driving simulator (driving information processing device)
31 Display section (display device)
32 Line of sight detection unit 40 Computer 44a Driving simulation program (driving information processing program)
45a Driving simulation means (specific processing execution means)
210 Manual driving support device (driving information processing device, computer)
214 Line of sight detection unit 217a Manual driving support program (driving information processing program)
218a Manual driving support means (specific processing execution means)
310 Automatic driving support device (driving information processing device, computer)
314 Line of sight detection unit 317a Automatic driving support program (driving information processing program)
318a Automatic driving support means (specific processing execution means)

Claims (4)

A driving information processing device that executes information processing for driving a vehicle,
a line-of-sight detection unit that detects the line-of-sight of the driver of the vehicle;
When a specific condition is satisfied, including one of the following: a condition that the driver performed a gaze on a specific range; and a condition that a gaze on the specific range was not performed by the driver. and specific processing execution means for executing specific processing in information processing,
The specific processing execution means determines that the line of sight detected by the line of sight detection unit continues to exist in the specific range for a time exceeding a specific time exceeding 0 seconds, as a gaze on the specific range. and that the line of sight detected by the line of sight detection unit continues to exist in the specific range for a time equal to or less than the specific time, and that the line of sight detected by the line of sight detection unit is in the specific range. does not treat the fact that it does not exist as a gaze on the specific range,
At least one of the specific ranges is a range of road signs indicating the current speed limit,
The specific condition includes a condition that the driver did not gaze at the range of the road sign when the vehicle was traveling at a speed exceeding the current speed limit,
The specific processing execution means checks the road sign when the vehicle is traveling at a speed exceeding the current speed limit when the driver does not pay attention to the range of the road sign. Notify that the specified process has not been carried out,
The specific processing execution means checks the road sign when the vehicle is not traveling at a speed exceeding the current speed limit when the driver does not pay attention to the range of the road sign. A driving information processing device characterized in that a driving information processing device does not execute a notification to warn that the driver is not driving . The driving information processing device according to claim 1, wherein the specific processing execution means executes a notification to warn against speeding when the vehicle is traveling at a speed exceeding a current speed limit. . A driving information processing program for executing information processing for driving a vehicle, the program comprising:
When certain conditions are met, including either one of the following: the driver of the vehicle has gazed at a particular range; and the driver has not gazed at the particular range. causing the computer to function as a specific processing execution means for executing specific processing in the information processing,
The specific processing execution means determines that the line of sight detected by the line of sight detection unit that detects the line of sight of the driver continues to exist in the specific range for a time exceeding a specific time exceeding 0 seconds. Treated as a gaze on the specific range, the line of sight detected by the line of sight detection unit continues to exist in the specific range for a time equal to or less than the specific time, and the line of sight detected by the line of sight detection unit The fact that the line of sight does not exist for the specific range is not treated as a gaze on the specific range,
At least one of the specific ranges is a range of road signs indicating the current speed limit,
The specific condition includes a condition that the driver did not gaze at the range of the road sign when the vehicle was traveling at a speed exceeding the current speed limit,
The specific processing execution means checks the road sign when the vehicle is traveling at a speed exceeding the current speed limit when the driver does not pay attention to the range of the road sign. Notify that the specified process has not been carried out,
The specific processing execution means checks the road sign when the vehicle is not traveling at a speed exceeding the current speed limit when the driver does not pay attention to the range of the road sign. A driving information processing program characterized in that it does not execute a notification to warn that it is not running. 4. The driving information processing program according to claim 3, wherein the specific processing execution means executes a notice to warn against speeding when the vehicle is traveling at a speed exceeding a current speed limit. .

Images