JP2022174052A - Operating information processing device and operating information processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operating information processing device and an operating information processing program capable of utilizing a line of sight of a driver of a vehicle in information processing for vehicle driving more effectively than conventional ones.

SOLUTION: A driving simulator executes a notification that safety confirmation by a back mirror is not performed (S102) in the case that a driver does not execute a gaze into the range of the mirror surface of the back mirror in a specific period just before a simulation object vehicle changes a direction of travel (NO in S101). The driving simulator regards the continuation of the existence of the line of sight of the driver detected by a line of sight detection part in a specific range for a threshold time as gaze at the specific range, and does not regard the continuation of the existence of the line of sight of the driver detected by the line of sight detection part in the specific range for a time of the threshold time or below and the nonexistence of the line of sight detected by the line of sight detection part in the specific range as a gaze at the specific range.

SELECTED DRAWING: Figure 3

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to a driving information processing device and a driving information processing program for executing information processing for driving a vehicle.

2. Description of the Related Art Conventionally, as a driving information processing device that executes information processing for driving a vehicle, a head-mounted display system with a line-of-sight detection function that executes simulation of driving a vehicle is known (see, for example, Patent Document 1). . This head-mounted display system with line-of-sight detection function detects the line-of-sight of the vehicle driver during execution of the driving simulation, and stores the position of the detected line-of-sight in a storage device in synchronism with the image of the driving simulation. After the driving simulation is completed, the line-of-sight position read from the storage device and the driving simulation image can be superimposed and displayed on the monitor.

Japanese Patent Application Laid-Open No. 2001-117046

However, in the conventional driving information processing apparatus, since the line of sight of the vehicle driver is only confirmed after the driving simulation is completed, the line of sight of the vehicle driver is sufficient for information processing for driving the vehicle. There is a problem that it is not utilized in

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a driving information processing apparatus and a driving information processing program that can more effectively utilize the line of sight of a vehicle driver than in the past in information processing for driving a vehicle. .

A driving information processing device according to the present invention is a driving information processing device that executes information processing for driving a vehicle, and includes a line-of-sight detection unit that detects a line-of-sight of the driver of the vehicle, is executed by the driver, and the condition that the driver did not gaze at the specific range, the specific condition is satisfied in the information processing. a specific process execution means for executing a process, wherein the specific process execution means is configured such that the line of sight detected by the line of sight detection unit continues for a time exceeding a specific time exceeding 0 seconds with respect to the specific range. presence is treated as gazing at the specific range, and the line of sight detected by the line-of-sight detection unit continues to exist in the specific range for a time equal to or less than the specific time; and the line-of-sight detection. The fact that the line of sight detected by the unit does not exist in the specific range is not treated as a gaze in the specific range.

With this configuration, the driving information processing apparatus of the present invention can set either the condition that the driver has gazed at the specific range or the condition that the driver has not gazed at the specific range. When specific conditions including one are satisfied, specific processing is executed in the information processing for driving the vehicle, so the line of sight of the vehicle driver can be used more effectively than before in the information processing for driving the vehicle. be able to.

In the information processing apparatus for driving of the present invention, the specific range includes a range of the mirror surfaces of the rearview mirrors of the vehicle, a range of the mirror surfaces of the door mirrors of the vehicle, and areas on the left and right outside of the vehicle other than the door mirrors. and the specific condition may include a condition that the driver did not gaze at the specific range in a specific period immediately before the vehicle changes direction.

With this configuration, the driving information processing apparatus of the present invention can improve the driver's awareness of safety confirmation when changing the traveling direction of the vehicle.

In the information processing device for driving of the present invention, the specific range is the range of the notification section of information related to driving, the notification section is installed in a portion other than the vehicle, and the specific condition is A condition may be included that the driver gazes at the specific range.

With this configuration, the driving information processing device of the present invention can improve the driver's awareness of checking the information notification unit for driving information.

In the information processing device for driving of the present invention, the specific range is a range in which distracted driving occurs when the driver gazes at it, and the specific condition is the maximum amount of time that the vehicle is being moved by the driving. It may include a condition that the driver gazes at the specific range.

With this configuration, the information processing device for driving of the present invention can improve the driver's awareness of avoiding inattentive driving.

In the driving information processing apparatus of the present invention, the specific range is a range of a specific object in a mirror image of the vehicle, and the specific condition is the specific range gazed at by the driver. number of conditions may be included.

With this configuration, the driving information processing apparatus of the present invention can improve the driver's awareness of the object in the mirror image of the vehicle.

In the information processing device for driving of the present invention, the specific process may be a process of executing the notification regarding the driving safety.

With this configuration, the driving information processing apparatus of the present invention can improve the driver's awareness of safe driving.

In the information processing device for driving of the present invention, the information processing is a simulation of the driving, the information processing device for driving includes a display device for displaying an image of the simulation, and the specific range is the display. It may be a range in the image displayed on the device, and the specific process may be a process of causing a specific situation to occur in the vehicle in the simulation.

With this configuration, the information processing device for driving according to the present invention can perform an effective simulation.

In the driving information processing apparatus of the present invention, the specific range is a range indicating a location, the specific condition includes a condition that the driver gazes at the specific range, and may be a process of determining the location indicated by the specific range as the destination of the driving.

With this configuration, the information processing device for driving of the present invention determines the destination of driving by utilizing the line of sight of the driver of the vehicle, so convenience can be improved.

In the driving information processing device of the present invention, the specific range is a range of a notification unit for information on traffic regulation, the notification unit is installed in a portion other than the vehicle, and the specific condition is A condition may be included in which the driver gazes at the specific range, and the specific process may be a process of determining the driving route based on the information included in the specific range. .

With this configuration, the information processing device for driving of the present invention determines the driving route by utilizing the line of sight of the driver of the vehicle, so that convenience can be improved.

In the driving information processing device of the present invention, the information processing is processing related to automatic driving of the vehicle, and the specific condition includes a condition that the driver gazes at the specific range, The specific process may be a process of determining the specific range as an avoidance target in the automatic driving.

With this configuration, the information processing device for driving of the present invention uses the line of sight of the driver of the vehicle to determine the avoidance target in automatic driving, so that convenience can be improved.

A driving information processing program of the present invention is a driving information processing program for executing information processing for driving a vehicle, the condition that the driver of the vehicle gazes at a specific range, and a computer as specific processing executing means for executing specific processing in the information processing when specific conditions including any one of the conditions that the driver did not gaze at the specific range are satisfied. The specific processing executing means detects that the line of sight detected by the line of sight detection unit continues to exist in the specific range for a time exceeding a specific time exceeding 0 seconds. and that the line of sight detected by the line of sight detection unit continues to exist in the specific range for a time equal to or less than the specific time, and that the line of sight detected by the line of sight detection unit is the It is characterized in that non-existence for a specific range is not treated as fixation for the specific range.

With this configuration, the computer that executes the information processing program for driving of the present invention can set the condition that the driver has gazed at the specific range, and the condition that the driver has not gazed at the specific range. When a specific condition including any one of is satisfied, a specific process is executed in the information processing for driving the vehicle. It can be used effectively.

The information processing device for driving and the information processing program for driving of the present invention can effectively utilize the line of sight of the driver of the vehicle in the information processing for driving the vehicle.

1 is a block diagram of a driving simulator according to a first embodiment of the invention; FIG. 2 is a block diagram of the computer shown in FIG. 1; FIG. FIG. 3 is a flow chart showing an example of the operation of the driving simulator shown in FIG. 1 when the vehicle to be simulated changes its traveling direction; FIG. FIG. 3 is a flow chart showing an example of the operation of the driving simulator shown in FIG. 1 when it is determined that the driver has gazed at a specific range; FIG. 4 is a flowchart showing an example of the operation of the driving simulator shown in FIG. 1 when the simulation target vehicle changes its traveling direction, different from the example shown in FIG. 3; FIG. 2 is a flow chart showing an example of the operation of the driving simulator shown in FIG. 1 with respect to a road sign restricting maximum speed; FIG. FIG. 2 is a flow chart showing an example of the operation of the driving simulator shown in FIG. 1 for inattentive driving; FIG. 2 is a flow chart showing an example of the operation of the driving simulator shown in FIG. 1 for confirmation of a mirror image by a mirror; FIG. 4 is a flow chart showing an example of the operation of the driving simulator shown in FIG. 1 when executing processing for determining a driving destination; FIG. FIG. 3 is a flow chart showing an example of the operation of the driving simulator shown in FIG. 1 when executing processing for determining a driving route; FIG. FIG. 2 is a block diagram of a manual driving support device according to a second embodiment of the invention; FIG. It is a block diagram of an automatic driving support device according to a third embodiment of the present invention. FIG. 13 is a flowchart showing an example of the operation of the automatic driving support device shown in FIG. 12 when executing processing for determining an avoidance target in automatic driving; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
First, the configuration of a driving simulator as a driving information processing device according to the first embodiment of the present invention will be described.

FIG. 1 is a block diagram of a driving simulator according to this embodiment.

A driving simulator 10 shown in FIG. 1 is a device that executes a simulation of driving an automobile as information processing for driving an automobile as a vehicle. The driving simulator 10 includes a vehicle operation device 20 for operating a vehicle to be simulated (hereinafter referred to as "simulated vehicle"), a head mounted display 30 mounted on the head of the driver of the simulated vehicle, A computer 40 that can communicate with the vehicle operation device 20 and the head mounted display 30 is provided.

The vehicle operating device 20 includes, for example, a steering wheel, a clutch pedal, an accelerator pedal, a brake pedal, a parking brake, a transmission operating lever, and a direction indicator operating device.

The head-mounted display 30 includes a display unit 31 as a display device such as an LCD (Liquid Crystal Display) that displays an image of a driving simulation (hereinafter referred to as a “simulation image”); and a line-of-sight detection unit 32 for detecting the line of sight of the driver. As the head mounted display 30, for example, "FOVE 0" manufactured by FOVE may be employed. The simulation image may be represented by any of VR (Virtual Reality), AR (Augmented Reality) and MR (Mixed Reality). When the simulation image is expressed by either AR or MR, the head mounted display 30 needs to be equipped with a camera for capturing the front of the driver's face.

As a method for detecting a line of sight by the line of sight detection unit 32, any known method for detecting a line of sight may be employed.

FIG. 2 is a block diagram of computer 40. As shown in FIG.

As shown in FIG. 2, a computer 40 includes an operation unit 41, which is an input device such as a keyboard and a mouse for inputting various operations, a microphone 42, and via a network such as a LAN (Local Area Network) or the Internet. Alternatively, the communication unit 43, which is a communication device that communicates directly with an external device by wire or wirelessly without going through a network, a semiconductor memory that stores various information, a non-volatile such as a HDD (Hard Disk Drive) A storage unit 44 that is a storage device and a control unit 45 that controls the entire computer 40 are provided. The computer 40 may be configured by, for example, a PC (Personal Computer).

The storage unit 44 stores a driving simulation program 44a as a driving information processing program for executing a driving simulation of an automobile. The driving simulation program 44a may be installed in the computer 40 at the manufacturing stage of the computer 40, or may be stored in an external storage such as a USB (Universal Serial Bus) memory, a CD (Compact Disk), or a DVD (Digital Versatile Disk). It may be additionally installed in the computer 40 from a medium or additionally installed in the computer 40 from the network.

The control unit 45 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory) storing programs and various data, and a RAM (Random Access Memory) used as a work area for the CPU. there is The CPU executes programs stored in the ROM or storage unit 44 .

The control unit 45 implements driving simulation means 45a by executing the driving simulation program 44a. The driving simulation means 45a satisfies specific conditions including any one of the condition that the driver gazed at the specific range and the condition that the driver did not gaze at the specific range. In this case, a specific process is executed in the simulation, so it constitutes the specific process execution means of the present invention.

Note that the range of gaze target candidates in the simulation image is set in advance in the driving simulation program 44a for each gaze target candidate.

Next, operation of the driving simulator 10 will be described.

The driving simulation means 45a displays a simulation image on the display unit 31, and reflects the operation on the vehicle operation device 20 and the line of sight detected by the line of sight detection unit 32 in the simulation. Further, the driving simulation means 45a displays on the display unit 31 according to the orientation of the driver's face in the same way that the driver's field of view changes in accordance with the orientation of the driver's face when driving an actual automobile. Change the simulation image.

An example in which the line of sight detected by the line of sight detection unit 32 is reflected in the simulation will be described below.

First, the operation of the driving simulator 10 when the vehicle to be simulated changes lanes, turns left, turns right, etc., will be described.

FIG. 3 is a flow chart showing an example of the operation of the driving simulator 10 when the simulation target vehicle changes its traveling direction.

The driving simulation means 45a performs the operations shown in FIG. Note that the driving simulation means 45a performs the operation shown in FIG. 3, for example, when an operation is performed on the vehicle operation device 20 to instruct the direction indicator to change the traveling direction of the simulation target vehicle. Alternatively, the simulation target vehicle actually starts changing the direction of travel after the vehicle operating device 20 is operated to instruct the direction indicator to change the direction of travel of the simulation target vehicle. Alternatively, the operation shown in FIG. 3 may be executed.

As shown in FIG. 3, the driving simulation means 45a determines whether or not the driver gazed at the range of the mirror surface of the rearview mirror of the simulation target vehicle during a specific period immediately before the simulation target vehicle changed its traveling direction. (S101).

FIG. 4 is a flow chart showing an example of the operation of the driving simulator 10 when it is determined that the driver has gazed at a specific range.

The driving simulation means 45a performs the operation shown in FIG. 4 each time the candidate whose line of sight detected by the line-of-sight detection unit 32 exists within the range of the target candidate in the simulation image is changed. .

As shown in FIG. 4, the driving simulation unit 45a identifies, as the current candidate, among the candidates for the target of gaze in the simulation image, those for which the line of sight detected by the line-of-sight detection unit 32 exists within the candidate's own range. (S111).

Next, the driving simulation means 45a determines whether or not the range in which the line of sight detected by the line of sight detector 32 exists has been changed from the current candidate range (S112).

When the driving simulation unit 45a determines in S112 that the range in which the line of sight detected by the line of sight detection unit 32 exists has not been changed from the range of the current candidate, the line of sight detected by the line of sight detection unit 32 is the current candidate. A time that continues to exist for the range is acquired (S113).

Next, the driving simulation means 45a determines whether or not the time obtained in S113 exceeds a specific time (hereinafter referred to as "threshold time") such as 0 second, for example, 1 second (S114).

When the driving simulation means 45a determines in S114 that the time acquired in S113 does not exceed the threshold time, it executes the process of S112.

When the driving simulation unit 45a determines in S114 that the time acquired in S113 exceeds the threshold time, it determines that the driver has gazed at the current candidate range (S115).

When the driving simulation unit 45a determines in S112 that the range in which the line of sight detected by the line of sight detection unit 32 exists has been changed from the current candidate range, or executes the processing of S115, the operation shown in FIG. 4 ends. .

Since the driving simulation means 45a executes the operation shown in FIG. 4, the fact that the line of sight detected by the line of sight detection unit 32 continues to exist in a specific range for a period of time exceeding the threshold time is regarded as a gaze for this range. It is determined that the line of sight detected by the line of sight detection unit 32 continues to exist in this range for a time equal to or less than the threshold time, and that the line of sight detected by the line of sight detection unit 32 does not exist in this range. , do not treat it as a fixation for this range.

As shown in FIG. 3, when the driving simulation means 45a determines in S101 that the driver has gazed at the range of the mirror surface of the rearview mirror during a specific period immediately before the simulation target vehicle changes its traveling direction, 3 ends.

When the driving simulation means 45a determines in S101 that the driver did not gaze at the range of the mirror surface of the rearview mirror during a specific period immediately before the simulation target vehicle changes its traveling direction, the driving simulation means 45a performs safety confirmation using the rearview mirror. The display unit 31 issues a notification that the user is not connected (S102), and the operation shown in FIG. 3 ends.

The driving simulation means 45a may perform the operation shown in FIG. 5 instead of the operation shown in FIG.

FIG. 5 is a flow chart showing an example different from the example shown in FIG. 3 of the operation of the driving simulator 10 when the simulation target vehicle changes its traveling direction.

As shown in FIG. 5, the driving simulation means 45a performs the same process as the process of S101 (S121).

When the driving simulation means 45a determines in S121 that the driver did not gaze at the range of the mirror surface of the rearview mirror during the specific period immediately before the simulation target vehicle changes its traveling direction, the driving simulation means 45a moves the rear or side of the simulation target vehicle. In the simulation, the simulated vehicle collides with another vehicle (S122), and the operation shown in FIG. 5 ends. Here, other vehicles include, for example, bicycles and automobiles such as motorcycles.

In the operation shown in FIG. 5, the driving simulation means 45a causes the simulation target vehicle to collide with another vehicle. However, the driving simulation means 45a may generate a situation in which the simulation target vehicle is about to collide with another vehicle instead of the situation in which the simulation target vehicle collides with another vehicle.

In the operation shown in FIG. 5, the driving simulation means 45a may further perform a notification to warn that safety confirmation is not being performed using the rearview mirror, as in the operation shown in FIG.

In the operations shown in FIGS. 3 and 5, the driving simulation means 45a executes the operations shown in FIGS. 3 and 5 based on whether or not the range of the mirror surface of the rearview mirror is gazed. Similarly, the driving simulation means 45a determines whether or not the range of the mirror surface of the left and right door mirrors of the simulation target vehicle, in which the simulation target vehicle changes its traveling direction, is gazed. 5 may be performed, or whether the outside of the right and left sides of the simulation target vehicle, in which the simulation target vehicle changes its traveling direction, is gazed at, other than the door mirrors. Based on the result, operations similar to those shown in FIG. 3 or FIG. 5 may be performed. Here, the driving simulation means 45a sets the threshold time to be different between gazing at the range of the mirror surface of the rearview mirror, gazing at the range of the mirror surface of the door mirror, and gazing at the range other than the door mirror on the left and right outside of the simulation target vehicle. You can let

Next, the operation of the driving simulator 10 with respect to road signs as a notification unit for information on driving will be described.

FIG. 6 is a flow chart showing an example of the operation of the driving simulator 10 for a road sign restricting maximum speed.

The driving simulation means 45a performs the operations shown in FIG. 6 when the simulation target vehicle moves.

As shown in FIG. 6, the driving simulation means 45a determines whether or not the simulation target vehicle has stopped (S131).

When the driving simulation means 45a determines in S131 that the simulation target vehicle is not stopped, it determines whether or not the simulation target vehicle has passed the vicinity of the road sign restricting the maximum speed (S132). Here, the road sign is installed in a portion other than the simulation target vehicle, for example, near the road along the driving route of the simulation target vehicle.

When the driving simulation means 45a determines in S132 that the simulation target vehicle has passed the vicinity of the road sign restricting the maximum speed, it identifies the maximum speed indicated by this road sign as the current speed limit (S133).

If the driving simulation means 45a determines in S132 that the simulation target vehicle did not pass near the road sign that limits the maximum speed, or if the process of S133 is executed, the simulation target vehicle travels at a speed exceeding the current speed limit. It is determined whether or not (S134).

When the driving simulation means 45a determines in S134 that the simulation target vehicle is traveling at a speed exceeding the current speed limit, it is determined whether the driver has gazed at the range of the road sign indicating the current speed limit. It is determined whether or not (S135).

When the driving simulation means 45a determines in S135 that the driver has been gazing at the range of the road sign indicating the current speed limit, the driving simulation means 45a issues a notice warning of speeding by the display unit 31 (S136).

If the driving simulation means 45a determines in S135 that the driver did not pay attention to the range of the road sign that indicated the current speed limit, it warns that the speed violation and the road sign have not been checked. The display unit 31 executes a notification to do so (S137).

If the driving simulation means 45a determines in S134 that the simulation target vehicle is not traveling at a speed exceeding the current speed limit, or if the processing of S136 or S137 is executed, then the processing of S131 is executed.

When the driving simulation means 45a determines in S131 that the vehicle to be simulated has stopped, the operation shown in FIG. 6 ends.

In the operation shown in FIG. 6, the driving simulation means 45a executes notification regarding driving safety (S134) when the driver does not gaze at the range of the road sign (NO in S135). However, even if the driving simulation means 45a causes a specific situation to occur in the simulation target vehicle in the simulation, like the operation shown in FIG. 5, when the driver does not gaze at the road sign range. good.

It should be noted that the operation shown in FIG. 6 is an operation in the case where the notifying portion of the information regarding driving is a road sign restricting the maximum speed. However, as long as the notification unit of the information related to driving is installed in a part other than the vehicle to be simulated, it may be a road sign other than the road sign restricting the maximum speed. A sign other than a road sign, such as a sign announcing traffic regulations, may be used.

The driving simulation means 45a may make the threshold time of gaze for the range of the notification part of the information about driving different from any of the threshold times of gaze described above, such as gaze at the range of the mirror surface of the rearview mirror.

Next, the operation of the driving simulator 10 for inattentive driving will be described.

FIG. 7 is a flow chart showing an example of the operation of the driving simulator 10 for inattentive driving.

The driving simulation means 45a performs the operation shown in FIG. 7 when the simulation target vehicle moves.

As shown in FIG. 7, the driving simulation means 45a determines whether or not the simulation target vehicle has stopped (S141).

If the driving simulation means 45a determines in S141 that the vehicle to be simulated is not stopped, has the driver gazed at a range in which inattentive driving occurs when gazed at by the driver (hereinafter referred to as "inattentive driving range")? It is determined whether or not (S142). Here, the inattentive range includes, for example, the range of the car navigation system inside the simulation target vehicle, the range of the mobile terminal inside the simulation target vehicle, and the like.

When the driving simulation means 45a determines in S142 that the driver has gazed at the inattentive range, the driving simulation means 45a causes a specific situation to occur in the simulation target vehicle (S143). For example, the driving simulation means 45a simulates a situation in which the simulation target vehicle collides with an object other than the simulation target vehicle, such as another vehicle, when the driver gazes at the inattentive range. Alternatively, a situation in which the simulation target vehicle is about to collide with an object other than the simulation target vehicle, such as a pedestrian running out in front of the simulation target vehicle, is generated in the simulation target vehicle. Also good.

The driving simulation means 45a executes the process of S141 when it determines in S142 that the driver did not gaze at the inattentive range or when the process of S143 ends.

When the driving simulation means 45a determines in S141 that the simulation target vehicle has stopped, the operation shown in FIG. 7 ends.

In the operation shown in FIG. 7, the driving simulation means 45a generates a specific situation in the simulation target vehicle in the simulation when the driver gazes at the inattentive range (YES in S142) (S143). However, when the driver gazes at the inattentive driving range, the driving simulation means 45a may execute a notification regarding driving safety, such as a notification warning against inattentive driving, as shown in FIG. good.

The driving simulation means 45a may make the gaze threshold time for the range of inattentiveness different from any of the threshold times for gaze described above, such as gaze at the range of the mirror surface of the rearview mirror, for example.

Next, the operation of the driving simulator 10 for confirming a mirror image through the mirrors of the vehicle to be simulated, such as rearview mirrors and door mirrors, will be described.

FIG. 8 is a flow chart showing an example of the operation of the driving simulator 10 for checking the mirror image by the mirror.

The driving simulation means 45a performs the operation shown in FIG. 8 when the driver starts driving the simulation target vehicle.

As shown in FIG. 8, the driving simulation means 45a determines whether or not driving of the simulation target vehicle has ended (S151).

When the driving simulation means 45a determines in S151 that the driving of the simulation target vehicle has not ended, it determines whether or not the driver has gazed at the target range to be gazed at in the mirror image (S152). Here, for example, a vehicle other than the vehicle to be simulated can be considered as an object to be gazed at in the mirror image.

When the driving simulation means 45a determines in S152 that the driver has gazed at the range of the object to be gazed at in the mirror image, the driving simulation means 45a sets a flag managed in association with this object (S153). Here, the driving simulation means 45a associates and manages a flag for each object to be gazed at in the mirror image. A flag managed in association with an object to be gazed at in a mirror image is in a collapsed state by default.

The driving simulation means 45a executes the process of S151 when it determines in S152 that the driver has not gazed at the range of the object to be gazed at in the mirror image, or when the process of S153 ends.

When the driving simulation means 45a determines in S151 that the simulation target vehicle has finished driving, it counts the number of objects that are associated with standing flags among the objects to be watched in the mirror image in this simulation ( S154).

Next, the driving simulation means 45a uses the display unit 31 to issue a notification regarding driving safety according to the number counted in S154 (S155). For example, the driving simulation means 45a may notify how many objects among the objects to be watched in the mirror image by the mirror in the current simulation.

After the process of S155, the driving simulation means 45a ends the operation shown in FIG.

In the operation shown in FIG. 8, the driving simulation means 45a executes notification regarding driving safety in accordance with the number of objects gazed at by the driver in the range of objects to be gazed at in the mirror image (S155). ). However, the driving simulation means 45a performs a specific situation in the simulation, as shown in FIG. You may make it generate|occur|produce with respect to a simulation object vehicle.

The driving simulation means 45a may change the gaze threshold time for the target range to be gazed at in the mirror image from any of the gaze threshold times described above, such as gaze at the mirror surface range of the rearview mirror, for example. .

Next, the operation of the driving simulator 10 when executing the process of determining the driving destination will be described.

FIG. 9 is a flow chart showing an example of the operation of the driving simulator 10 when executing the process of determining the driving destination.

The driving simulation means 45a performs the operation shown in FIG. 9 when the driver starts driving the simulation target vehicle.

As shown in FIG. 9, the driving simulation means 45a determines whether or not driving of the simulation target vehicle has ended (S161).

When the driving simulation means 45a determines in S161 that the driving of the simulation target vehicle has not ended, it determines whether or not the driver has gazed at the range indicating the location (S162). Here, as the range indicating the place, for example, a landmark drawn as a landscape in the simulation image, such as a television tower drawn at a distance from the vehicle to be simulated, can be considered. As the range indicating the place, for example, a place name written on a notification part such as a road sign installed in a part other than the simulation target vehicle can be considered.

When the driving simulation means 45a determines in S162 that the driver has gazed at the range indicating the location, the driving simulation means 45a sets the conditions for determining the location indicated by this range as the driving destination (hereinafter referred to as the "destination determination condition"). ) is satisfied (S163). Here, the destination determination conditions include, for example, that the driver gazes at the range indicating the location for more than a specific period of time, such as five seconds, and that a specific operation for determining the destination is performed. It is conceivable that a specific word such as "determine destination" is input into the microphone 42, or the like.

When the driving simulation means 45a determines in S163 that the destination determination condition is satisfied, the driving simulation means 45a determines, as the driving destination, the location indicated by the range determined in S162 that the driver has gazed (S164).

The driving simulation means 45a determines in S162 that the driver did not gaze at the range indicating the location, determines in S163 that the destination determination condition is not satisfied, or executes the process of S164. Execute the process.

When the driving simulation means 45a determines in S161 that the simulation target vehicle has finished driving, it ends the operation shown in FIG.

The driving simulation means 45a may change the gaze threshold time for the range indicating the place from any of the gaze threshold times described above, for example, the gaze for the mirror surface range of the rearview mirror.

Next, the operation of the driving simulator 10 when executing the process of determining the driving route will be described.

FIG. 10 is a flow chart showing an example of the operation of the driving simulator 10 when executing the process of determining the driving route.

The driving simulation means 45a performs the operation shown in FIG. 10 when the driver starts driving the simulation target vehicle.

As shown in FIG. 10, the driving simulation means 45a determines whether or not the simulation target vehicle has finished driving (S171).

When the driving simulation means 45a determines in S171 that the driving of the simulation target vehicle has not ended, it determines whether or not the driver has gazed at the range of the notification portion of the information on traffic regulation (S172). Here, the notification section for information on traffic regulation is installed in a portion other than the simulation target vehicle, for example, in the vicinity of the road along the driving route of the simulation target vehicle. For example, a signboard for notifying traffic regulations associated with the holding of a marathon event or the like can be considered as the notification unit for information on traffic regulations.

When the driving simulation means 45a determines in S172 that the driver has gazed at the range of the notification section of the information on traffic regulation, the display section 31 suggests a driving route based on the information included in this range (S173). ). For example, the driving simulation means 45a proposes a detour route in consideration of traffic congestion expected by traffic regulation, instead of a standard route.

Next, the driving simulation means 45a determines whether or not the conditions for determining the route proposed in S173 as the driving route (hereinafter referred to as "route determination conditions") are satisfied (S174). Here, as the route determination condition, for example, a specific operation for determining the route is input to the operation unit 41, a specific word such as "determine the route" is input to the microphone 42, and the like. can be considered.

When the driving simulation means 45a determines in S174 that the route determination condition is satisfied, it determines the route proposed in S173 as the driving route (S175).

The driving simulation means 45a determines in S172 that the driver did not pay attention to the range of the notification portion of the information on traffic regulation, or determines in S174 that the route determination condition is not satisfied, or executes the processing of S175. Then, the process of S171 is executed.

When the driving simulation means 45a determines in S171 that the driving of the simulation target vehicle has ended, the operation shown in FIG. 10 ends.

In the operation shown in FIG. 10, the driving simulation means 45a determines the driving route according to the driver's instructions (S173-S175). However, the driving simulation means 45a may automatically determine the driving route based on the information included in the range determined in S172 that the driver's gaze was performed.

The driving simulation means 45a may change the threshold time for gaze to the range of the notification part of the information on traffic regulation from any of the threshold times for gaze described above, such as gaze to the range of the mirror surface of the rearview mirror, for example.

As explained above, the driving simulator 10 satisfies a specific condition including the condition that the driver did not gaze at a specific range (NO in S101, NO in S121, or NO in S135). Then, a specific process is executed in the automobile driving simulation (S102, S122 or S137). Further, when the driving simulator 10 satisfies a specific condition including the condition that the driver has gazed at a specific range (YES in S142, YES in S152, YES in S162, or YES in S172), A specific process is executed in the automobile driving simulation (S143, S155, S164 or S175). Therefore, the driving simulator 10 can more effectively utilize the line of sight of the driver than in the past in the driving simulation of the automobile.

When the driving simulator 10 executes the operation shown in FIG. 3, the driver does not gaze at a specific range during a specific period immediately before the simulation target vehicle changes direction (NO in S101). In addition, since a notification regarding driving safety is executed (S102), the driver's awareness of safety confirmation when changing the traveling direction of the simulation target vehicle, that is, the driver's awareness of safe driving can be improved. can. For example, when the driver tries to turn the simulation target vehicle left, he checks the safety using the rearview mirror, decelerates the simulation target vehicle, and checks the safety of the area other than the door mirror on the left outside of the simulation target vehicle. The simulation target vehicle collides with the motorcycle when the simulation target vehicle is operated to turn left by the steering wheel without noticing the motorcycle approaching the simulation target vehicle from the left rear side due to neglecting the safety confirmation with the door mirror of the simulation target vehicle. When such a situation occurs, the driving simulator 10 uses the display unit 31 to notify the driver that the left door mirror is not used for safety confirmation, thereby improving the safety of the driver when the simulation target vehicle turns left. Awareness of confirmation can be improved.

When the driving simulator 10 executes the operation shown in FIG. 5, the driver does not gaze at a specific range during a specific period immediately before the simulation target vehicle changes direction (NO in S121). Furthermore, in the simulation, a specific situation is caused to the simulation target vehicle (S122), so that an effective simulation can be performed and the driver can confirm safety when changing the traveling direction of the simulation target vehicle. Awareness can be improved.

When executing the operation shown in FIG. 6, the driving simulator 10 executes notification regarding driving safety when the driver does not pay attention to the range of the notification section of the information regarding driving (NO in S135). (S137) Therefore, it is possible to improve the driver's awareness of checking the notification part of the information regarding driving, that is, the driver's awareness of safe driving. For example, in a state where the maximum speed was 60 km/h, when the simulation target vehicle passes near a road sign that limits the maximum speed to 50 km/h, the driver continues to drive the simulation target vehicle at a speed exceeding 50 km/h. When the driver does not pay attention to the range of the road sign indicating the current speed limit, the driving simulator 10 issues a notice that the road sign is not checked.

The driving simulator 10 performs an effective simulation when a specific situation occurs to the simulation target vehicle in the simulation when the driver does not pay attention to the range of the notification part of the information on driving. In addition, it is possible to improve the driver's awareness of checking the notification unit for information regarding driving.

When the driving simulator 10 executes the operation shown in FIG. 7, when the driver gazes at the inattentive range (YES in S142), the driving simulator 10 causes a specific situation to occur in the simulation target vehicle in the simulation (S143). ), an effective simulation can be performed, and the driver's awareness of avoiding inattentive driving can be improved. For example, the driving simulator 10 causes a mobile terminal placed in a mobile phone holder installed near the gauges of the vehicle to be simulated to receive a message, and when the driver sees this message, the driver receives the message. By causing a pedestrian to jump out in front of the simulation target vehicle at the moment of seeing it, it is possible to make the driver cool as a realistic experience, and improve the driver's awareness of avoiding distracted driving. .

When the driving simulator 10 executes a notification regarding driving safety when the driver gazes at the inattentive range, the driving simulator 10 is configured to increase the driver's awareness of not inattentive driving, that is, the driver's awareness of safe driving. Awareness can be improved.

When executing the operation shown in FIG. 8, the driving simulator 10 executes a notification regarding driving safety in accordance with the number of objects gazed at by the driver, out of the range of targets to be gazed at in the mirror image. (S155) Therefore, the driver's awareness of the object in the mirror image by the mirror, not the object that the driver can see directly without going through the mirror of the car in the normal field of view of the driver while driving, that is, the safety The driver's awareness of driving can be improved.

The driving simulator 10 is effective when causing a specific situation in the simulation target vehicle in the simulation according to the number of objects gazed at by the driver in the range of objects to be gazed at in the mirror image. In addition, the driver's awareness of objects outside the normal field of view can be improved.

The driving simulator 10 utilizes the line of sight of the driver to determine the driving destination (S164) and the driving route (S175), thereby improving convenience.

Although the display device for displaying the simulation image is realized by the head mounted display in this embodiment, it may be realized by a device other than the head mounted display.

Although the line-of-sight detection unit that detects the line-of-sight of the driver in the simulation image is implemented by the head-mounted display in the present embodiment, it may be implemented by a device other than the head-mounted display.

(Second embodiment)
First, the configuration of a manual driving support device as a driving information processing device according to the second embodiment of the present invention will be described.

FIG. 11 is a block diagram of manual driving support device 210 according to the present embodiment.

A manual driving assistance device 210 shown in FIG. 11 is a device that is mounted on a vehicle (not shown) driven by a driver to assist the driver in driving the vehicle. The manual driving support device 210 includes an operation unit 211 that is an input device such as a keyboard and a mouse for inputting various operations, a display unit 212 that is a display device such as an LCD that displays various information, and a driver's face. A camera 213 for photographing the front of the vehicle, a line-of-sight detection unit 214 for detecting the driver's line of sight, a microphone 215, and via a network such as a LAN or the Internet, or directly by wire or wirelessly without a network. , a communication unit 216 that is a communication device that communicates with an external device, a storage unit 217 that is a non-volatile storage device such as a semiconductor memory or an HDD that stores various information, and the entire manual driving support device 210. and a control unit 218 . The manual driving assistance device 210 can communicate with an external device such as a vehicle in which the manual driving assistance device 210 is mounted via a communication unit 216 .

The display unit 212, the camera 213, and the line-of-sight detection unit 214 may be implemented by a head-mounted display as in the first embodiment. However, when the display unit 212, the camera 213, and the line-of-sight detection unit 214 are realized by a head-mounted display, the image displayed on the display unit 212 is expressed by either AR or MR.

The storage unit 217 stores a manual driving assistance program 217a as a driving information processing program for executing information processing for assisting the driver in driving the automobile. For example, the manual driving assistance program 217a may be installed in the manual driving assistance device 210 at the manufacturing stage of the manual driving assistance device 210, or may be installed in the manual driving assistance device 210 from an external storage medium such as a USB memory, CD, or DVD. , or may be additionally installed in the manual driving support device 210 from the network.

The control unit 218 includes, for example, a CPU, a ROM that stores programs and various data, and a RAM that is used as a work area for the CPU. The CPU executes programs stored in the ROM or storage unit 217 . That is, the manual driving assistance device 210 is a computer.

The control unit 218 implements a manual driving assistance means 218a by executing a manual driving assistance program 217a. The manual driving assistance means 218a sets specific conditions including any one of the condition that the driver gazed at the specific range and the condition that the driver did not gaze at the specific range. When the condition is satisfied, a specific process is executed in the information processing for assisting the driver in driving the vehicle, and therefore constitutes the specific process executing means of the present invention.

Note that the manual driving support means 218a specifies the candidate range of gaze targets in the image captured by the camera 213 by performing image processing on the image captured by the camera 213 based on specific criteria.

Next, the operation of the manual driving assistance device 210 will be described.

Similar to the driving simulation means 45a (see FIG. 2), the manual driving support means 218a is configured to change the range of the mirror surface of the rearview mirror and the mirror surface of the door mirror of the vehicle during a specific period immediately before the vehicle changes its traveling direction. By having the display 212 perform a notification regarding driving safety when the driver fails to gaze at a particular area, such as an area outside the left and right sides of the vehicle, other than the door mirrors, the vehicle progresses. It is possible to improve the driver's awareness of safety confirmation when changing directions, that is, the driver's awareness of safe driving.

As with the driving simulation means 45a, the manual driving support means 218a executes notification regarding driving safety using the display section 212 when the driver does not pay attention to the range of the notification section of the information regarding driving. Thus, it is possible to improve the driver's awareness of checking the notification section for driving-related information installed in a portion other than the automobile, that is, the driver's awareness of safe driving.

Similar to the driving simulation means 45a, the manual driving support means 218a, when the driver gazes at the inattentive driving range, causes the display unit 212 to issue a notification regarding driving safety, thereby preventing inattentive driving. It is possible to improve the driver's awareness of safety, that is, the driver's awareness of safe driving.

As with the driving simulation means 45a, the manual driving support means 218a can improve convenience by determining the driving destination or the driving route by utilizing the driver's line of sight. can. The manual driving assistance means 218a inputs the determined destination and route to the car navigation system inside the automobile.

The manual driving support means 218a transmits, via the communication unit 216, the congestion information on the route to the location indicated by the driver's gaze to the Internet via the communication unit 216 when the driver gazes at the range indicating the location. The location indicated by the range gazed at by the driver, such as information on a restaurant located in the location indicated by the range gazed at by the driver, is displayed on the display unit 212 by searching from above. It is also possible to search for information related to information on the Internet via the communication unit 216 and display the search results on the display unit 212 .

The manual driving support means 218a may store the image of the range where the driver gazes, among the images captured by the camera 213 . For example, the manual driving assistance means 218a may store a range surrounded by four points where the driver gazed.

(Third Embodiment)
First, the configuration of an automatic driving support device as a driving information processing device according to the third embodiment of the present invention will be described.

FIG. 12 is a block diagram of automatic driving support device 310 according to the present embodiment.

An automatic driving support device 310 shown in FIG. 12 is a device that is mounted on a vehicle (not shown) that executes automatic driving to support automatic driving by the vehicle. The automatic driving support device 310 includes an operation unit 311 that is an input device such as a keyboard and a mouse for inputting various operations, a display unit 312 that is a display device such as an LCD that displays various information, and a driver's face. A camera 313 for photographing the front of the vehicle, a line-of-sight detection unit 314 for detecting the line of sight of the driver, a microphone 315, and via a network such as a LAN or the Internet, or directly by wire or wirelessly without a network. , a communication unit 316 that is a communication device that communicates with an external device, a semiconductor memory that stores various information, a storage unit 317 that is a non-volatile storage device such as an HDD, and the entire automatic driving support device 310. and a control unit 318 . The automatic driving assistance device 310 can communicate with an external device such as a vehicle in which the automatic driving assistance device 310 is installed via a communication unit 316 .

The display unit 312, the camera 313, and the line-of-sight detection unit 314 may be realized by a head-mounted display as in the first embodiment. However, when the display unit 312, the camera 313, and the line-of-sight detection unit 314 are realized by a head-mounted display, the image displayed on the display unit 312 is expressed by either AR or MR.

The storage unit 317 stores an automatic driving support program 317a as a driving information processing program for executing information processing for supporting automatic driving by a vehicle. The automatic driving assistance program 317a may be installed in the automatic driving assistance device 310 at the manufacturing stage of the automatic driving assistance device 310, for example, or may be downloaded from an external storage medium such as a USB memory, CD, DVD, etc. to the automatic driving assistance device 310. , or may be additionally installed in the automatic driving support device 310 from the network.

The control unit 318 includes, for example, a CPU, a ROM storing programs and various data, and a RAM used as a work area for the CPU. The CPU executes programs stored in the ROM or storage unit 317 . That is, the automatic driving support device 310 is a computer.

The control unit 318 implements an automatic driving assistance means 318a by executing an automatic driving assistance program 317a. The automatic driving support means 318a sets a specific condition including any one of a condition that the driver has gazed at the specific range and a condition that the driver has not gazed at the specific range. When the condition is satisfied, a specific process is executed in the information processing for assisting automatic driving by the automobile, and therefore constitutes the specific process executing means of the present invention.

Note that the automatic driving support means 318a specifies the candidate range of gaze targets in the image captured by the camera 313 by performing image processing on the image captured by the camera 313 based on specific criteria.

Next, the operation of the automatic driving support device 310 will be described.

The automatic driving support device 310 can determine an avoidance target in automatic driving. The operation of the automatic driving support device 310 when executing processing for determining an avoidance target in automatic driving will be described below.

FIG. 13 is a flowchart showing an example of the operation of the automatic driving support device 310 when executing processing for determining an avoidance target in automatic driving.

The automatic driving support means 318a performs the operation shown in FIG. 13 when the automatic driving of the automobile is started.

As shown in FIG. 13, the automatic driving support means 318a determines whether or not automatic driving has ended (S321).

When the automatic driving support means 318a determines in S321 that the automatic driving has not ended, it determines whether or not the driver has gazed at a specific range (S322).

When the automatic driving support means 318a determines in S322 that the driver has gazed at a specific range, it determines this range as an avoidance target in automatic driving (S323). It is input to the automatic driving system of the car as an avoidance target (S324).

The automatic driving support means 318a executes the process of S321 when it determines in S322 that the driver did not gaze at the specific range or when the process of S324 ends.

When the automatic driving support means 318a determines in S321 that the automatic driving has ended, the operation shown in FIG. 13 ends.

As described above, the automatic driving support device 310 utilizes the line of sight of the driver of the vehicle to determine the avoidance target in automatic driving (S323), so convenience can be improved.

Similar to the manual driving assistance means 218a (see FIG. 11), the automatic driving assistance means 318a utilizes the line of sight of the driver to determine the driving destination or the driving route. Convenience can be improved. The automatic driving support means 318a inputs the determined destination and route to the automatic driving system of the automobile.

Similar to the manual driving assistance means 218a, the automatic driving assistance means 318a, when the driver gazes at the range indicating the location, detects congestion on the route to the location indicated by the gazed range by the driver. Information is searched from the Internet via the communication unit 316 and the search result is displayed on the display unit 312, or the driver gazes at information such as restaurant information existing in the location indicated by the range where the driver gazes. Information about the location indicated by the range where is executed may be searched from the Internet via the communication unit 316 and the search result may be displayed on the display unit 312 .

Similarly to the manual driving assistance means 218a, the automatic driving assistance means 318a may store an image of the range where the driver gazes, out of the images captured by the camera 313. FIG.

In each embodiment described above, the driving of the vehicle is described. However, the present invention is also applicable to the operation of vehicles other than vehicles, such as the operation of ships and the operation of aircraft.

The present invention is configured so that it can be applied to xR technology, which is a general term for arbitrary reality technologies such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality). May be.

10 Driving simulator (driving information processing device)
31 display unit (display device)
32 Line-of-sight detection unit 40 Computer 44a Driving simulation program (information processing program for driving)
45a driving simulation means (specific processing execution means)
210 Manual driving support device (driving information processing device, computer)
214 Line-of-sight detection unit 217a Manual driving support program (information processing program for driving)
218a manual driving support means (specific processing execution means)
310 Automatic driving support device (driving information processing device, computer)
314 line-of-sight detection unit 317a automatic driving support program (driving information processing program)
318a Automatic driving support means (specific processing execution means)

Claims (11)

A driving information processing device that executes information processing for driving a vehicle,
a line-of-sight detection unit that detects a line of sight of a driver of the vehicle;
If a specific condition including any one of the condition that the driver gazed at the specific range and the condition that the driver did not gaze at the specific range is satisfied. a specific process executing means for executing a specific process in information processing,
The specific processing execution means detects that the line of sight detected by the line-of-sight detection unit continues to exist in the specific range for a time exceeding a specific time exceeding 0 seconds as the gaze to the specific range. the line of sight detected by the line of sight detection unit continues to exist in the specific range for a time equal to or less than the specific time; The information processing device for driving, characterized in that the non-existence with respect to the specific range is not treated as a fixation with respect to the specific range. The specific range is any one of the range of the mirror surface of the rearview mirror of the vehicle, the range of the mirror surface of the door mirror of the vehicle, and the range of the left and right outside of the vehicle other than the door mirror,
2. A vehicle according to claim 1, wherein said specific condition includes a condition that said driver did not gaze at said specific range during a specific period immediately before said vehicle changes its direction of travel. Driving information processing device. The specific range is the range of the notification unit of information related to driving,
The notification unit is installed in a portion other than the vehicle,
2. The driving information processing apparatus according to claim 1, wherein said specific condition includes a condition that said driver gazes at said specific range. The specific range is a range of distracted driving when gazed at by the driver,
2. The driving according to claim 1, wherein the specific condition includes a condition that the driver gazes at the specific range while the vehicle is being moved by the driving. Information processing equipment for The specific range is a range of a specific object in a mirror image of the vehicle,
2. The information processing device for driving according to claim 1, wherein the specific condition includes a condition of the number of numbers in the specific range in which the driver gazes. 6. The driving information processing apparatus according to claim 1, wherein the specific process is a process of executing the notification regarding the safety of driving. The information processing is a simulation of the driving,
The driving information processing device includes a display device for displaying the simulation image,
The specific range is a range in the image displayed on the display device,
6. The driving information processing apparatus according to claim 1, wherein the specific process is a process for causing a specific situation to occur in the vehicle in the simulation. The specific range is a range indicating a location,
The specific condition includes a condition that the driver gazes at the specific range,
2. The driving information processing apparatus according to claim 1, wherein the specific process is a process of determining the location indicated by the specific range as the destination of the driving. The specific range is the range of the notification part of information on traffic regulation,
The notification unit is installed in a portion other than the vehicle,
The specific condition includes a condition that the driver gazes at the specific range,
2. The driving information processing apparatus according to claim 1, wherein the specific process is a process of determining the driving route based on the information included in the specific range. The information processing is processing related to automatic driving of the vehicle,
The specific condition includes a condition that the driver gazes at the specific range,
2. The driving information processing apparatus according to claim 1, wherein the specific process is a process of determining the specific range as an avoidance target in the automatic driving. A driving information processing program for executing information processing for driving a vehicle,
When certain conditions are met, including any one of the condition that gaze to a particular area was performed by the driver of the vehicle, and the condition that gaze to the particular area was not performed by the driver. and causing the computer to function as specific processing execution means for executing specific processing in the information processing,
The specific processing execution means detects that the line of sight detected by the line-of-sight detection unit continues to exist in the specific range for a time exceeding a specific time exceeding 0 seconds as the gaze to the specific range. the line of sight detected by the line of sight detection unit continues to exist in the specific range for a time equal to or less than the specific time; The information processing program for driving, characterized in that the non-existence with respect to the specific range is not treated as a fixation with respect to the specific range.

Images