JP2020071773A - Line-of-sight detection apparatus - Google Patents

Abstract

To allow a line-of-sight detection unit to highly accurately detect a line-of-sight from when a vehicle starts moving.SOLUTION: A line-of-sight detection apparatus in an embodiment includes: a line-of-sight detection unit which detects a line-of-sight of a driver of a vehicle; a pattern image display control unit which displays a pattern image for personal authentication on a field of view of the driver; a personal authentication unit which compares moving pattern of the line-of-sight detected by the line-of-sight detection unit with a predetermined moving pattern before the driver starts driving the vehicle, and executes personal authentication processing for the driver; and a calibration processing unit which executes calibration on the line-of-sight detected by the line-of-sight detection unit on the basis of a result of comparing the moving pattern of the line-of-sight detected by the line-of-sight detection unit with the predetermined moving pattern.SELECTED DRAWING: Figure 4

Description

  Embodiments of the present invention relate to a visual line detection device.

  Technology for calibrating the line-of-sight detection process of the driver based on the image displayed on the HUD (Head Up Display), the driver when the driver is looking at the vehicle equipment such as the rearview mirror and the side mirror The technology to perform the calibration of the line-of-sight detection process of the vehicle, the technology to perform the calibration of the line-of-sight detection process of the driver based on whether or not the driver is gazing at the objects around the vehicle have been developed. There is.

JP, 2009-183473, A JP, 2010-030361, A JP, 2017-004117, A

  However, in the calibration of the driver's line-of-sight detection processing based on the image displayed on the HUD, the display range of the image with respect to the HUD is narrower than the range in which the driver has to look at the line of sight while driving the vehicle. Therefore, the accuracy of the calibration of the driver's line-of-sight detection processing becomes low.

  In addition, the technology to calibrate the line-of-sight detection process of the driver when the driver is looking at the equipment of the vehicle and the objects around the vehicle can be performed only while the vehicle is driving, so sufficient calibration is required. Until this is done, the vehicle is driven with low line-of-sight detection accuracy.

  Therefore, one of the problems of the embodiment is to provide a visual line detection device capable of detecting the visual line with high accuracy from the start of driving.

  The line-of-sight detection device of the embodiment, as an example, a line-of-sight detection unit that detects the line-of-sight of the driver of the vehicle, a pattern image display control unit that displays a pattern image for personal authentication with respect to the driver's field of view, driving Before the driver starts driving the vehicle, the movement pattern of the line of sight detected by the line-of-sight detection unit is compared with a predetermined movement pattern, and a personal authentication unit that executes personal authentication processing of the driver and a line-of-sight detection unit are performed. A calibration processing unit that performs calibration of the sight line detected by the sight line detection unit based on a comparison result between the detected movement pattern of the sight line and a predetermined movement pattern. Therefore, as an example, the line-of-sight can be detected with high accuracy by the line-of-sight detection unit from the start of driving the vehicle.

  In the line-of-sight detection device of the embodiment, as an example, the personal authentication unit starts the drive unit of the vehicle when the personal authentication process of the driver is successful. Therefore, as an example, it is possible to prevent the vehicle from being stolen.

  Further, the line-of-sight detection device of the embodiment, as an example, the personal authentication unit, further, when the line-of-sight detection unit detects the line of sight of a movement pattern different from the predetermined movement pattern continuously a predetermined number of times, the drive unit of the vehicle. Is prohibited from starting. Therefore, as an example, the vehicle theft can be more effectively prevented.

  Further, the line-of-sight detection device of the embodiment, as an example, the personal authentication unit, when the line-of-sight detection unit detects a line of sight of a movement pattern different from the predetermined movement pattern continuously a predetermined number of times, the imaging unit of the driver A captured image obtained by capturing a face is stored in the storage unit. Therefore, as an example, it is possible to identify the driver who is going to commit an illegal act such as the theft of the vehicle based on the face image included in the captured image stored in the storage unit.

  Further, the line-of-sight detection device of the embodiment further includes, as an example, an obstacle detection unit that detects an obstacle existing around the vehicle, the pattern image display control unit further, after the driver starts driving the vehicle, Obstacle information indicating the position of the detected obstacle is displayed in the driver's field of view, and the calibration processing unit further detects the position of the obstacle information in the driver's field of view and the line-of-sight detection unit. The visual line detected by the visual line detection unit is calibrated. Therefore, as an example, the line-of-sight detection unit can detect the line-of-sight with higher accuracy.

FIG. 1 is a perspective view showing an example of a state in which a part of a vehicle compartment of a vehicle equipped with the visual line detection device according to the first embodiment is seen through. FIG. 2 is a diagram illustrating an example of the arrangement of the image pickup devices included in the vehicle according to the first embodiment. FIG. 3 is a block diagram showing an example of the functional configuration of the vehicle according to the first embodiment. FIG. 4 is a block diagram showing an example of a functional configuration of an ECU included in the vehicle according to the first embodiment. FIG. 5 is a flowchart showing an example of the flow of line-of-sight calibration detected by the ECU of the vehicle according to the first embodiment. FIG. 6 is a diagram for explaining an example of a personal authentication process of a driver by the vehicle according to the first embodiment. FIG. 7 is a block diagram showing an example of a functional configuration of an ECU included in the vehicle according to the second embodiment. FIG. 8 is a flowchart showing an example of the flow of calibration detected by the ECU of the vehicle according to the second embodiment. FIG. 9 is a diagram showing an example of a process of displaying obstacle information by the vehicle according to the second embodiment.

  Hereinafter, exemplary embodiments of the present invention will be disclosed. The configurations of the embodiments shown below and the actions, results, and effects provided by the configurations are examples. The present invention can be realized by a configuration other than those disclosed in the following embodiments, and at least one of various effects based on the basic configuration and derivative effects can be obtained.

  The vehicle equipped with the line-of-sight detection device according to the present embodiment may be a vehicle that uses an internal combustion engine (engine) as a drive source (internal combustion engine vehicle), or a vehicle that uses an electric motor (motor) as a drive source (electric vehicle). , A fuel cell vehicle, etc.) or a vehicle (hybrid vehicle) using both of them as drive sources. Further, the vehicle can be equipped with various transmissions and various devices (systems, parts, etc.) necessary for driving the internal combustion engine and the electric motor. Further, the method, the number, the layout, etc. of the devices relating to the driving of the wheels in the vehicle can be set variously.

(First embodiment)
FIG. 1 is a perspective view showing an example of a state in which a part of a vehicle compartment of a vehicle equipped with the visual line detection device according to the first embodiment is seen through. FIG. 2 is a diagram showing an example of the arrangement of the image pickup devices included in the vehicle according to the first embodiment. As shown in FIG. 1, the vehicle 1 includes a vehicle body 2, a steering unit 4, and a monitor device 11.

  The vehicle body 2 has a vehicle interior 2a in which a driver (not shown) gets in. A steering unit 4 and the like are provided in the vehicle interior 2a so as to face the seat 2b of the driver as an occupant. The steering unit 4 is, for example, a steering wheel protruding from the dashboard (instrument panel) 12.

  Further, a monitor device 11 is provided at the center of the dashboard 12 in the vehicle interior 2a in the vehicle width direction, that is, in the left-right direction. The monitor device 11 is provided with a display device 8 (see FIG. 3) and an audio output device 9 (see FIG. 3). The display device 8 is, for example, an LCD (Liquid Crystal Display) or an OELD (Organic Electroluminescent Display). The audio output device 9 is, for example, a speaker. The display device 8 is covered with a transparent operation input unit 10 (see FIG. 3) such as a touch panel.

  The occupant can visually recognize the image displayed on the display screen of the display device 8 through the operation input unit 10. In addition, the occupant can perform the operation input by operating the operation input unit 10 by touching, pushing, or moving the operation input unit 10 with a finger or the like at a position corresponding to the image displayed on the display screen of the display device 8. ..

  Further, as shown in FIG. 2, a driver monitor camera 201 is installed on the handle column 202. The driver monitor camera 201 is, for example, a CCD (Charge Coupled Device) camera or the like. The driver monitor camera 201 has its viewing angle and posture adjusted such that the face of the driver 302 seated on the seat 2b is located at the center of the field of view. The driver monitor camera 201 sequentially captures the face of the driver 302 and sequentially outputs the image data of the images obtained by the capture.

  Further, as shown in FIG. 1, the vehicle 1 is a four-wheeled vehicle (four-wheeled vehicle) and has two left and right front wheels 3F and two left and right rear wheels 3R. Further, in the present embodiment, these four wheels 3 are provided so that they can be steered (steerable).

  Further, as shown in FIG. 1, the vehicle 1 is equipped with a plurality of image capturing units 15 (vehicle-mounted cameras). In this embodiment, the vehicle 1 is equipped with, for example, four image capturing units 15a to 15d. The image pickup unit 15 is a digital camera having an image pickup device such as a CCD or a CIS (CMOS Image Sensor). The image capturing unit 15 can capture an image of the surroundings of the vehicle 1 at a predetermined frame rate. Then, the image capturing unit 15 outputs a captured image obtained by capturing the surroundings of the vehicle 1. The imaging unit 15 has a wide-angle lens or a fish-eye lens, respectively, and can image, for example, a range of 140 ° to 220 ° in the horizontal direction. In addition, the optical axis of the imaging unit 15 may be set obliquely downward.

  Specifically, the imaging unit 15a is located, for example, on the rear end of the vehicle body 2 and is provided on the wall below the trunk lid. Then, the image capturing unit 15a can capture an image of a region behind the vehicle 1 in the periphery of the vehicle 1. The imaging unit 15b is located, for example, on the right end of the vehicle body 2 and is provided on the right door mirror. Then, the image capturing unit 15b can capture an image of a region on the side of the vehicle 1 in the periphery of the vehicle 1. The imaging unit 15c is located, for example, on the front side of the vehicle body 2, that is, on the front end of the vehicle 1 in the front-rear direction, and is provided on the front bumper, the front grill, or the like. Then, the image capturing unit 15c can capture an image of a region in front of the vehicle 1 in the periphery of the vehicle 1. The imaging unit 15d is located, for example, on the left side of the vehicle body 2, that is, on the left side end in the vehicle width direction, and is provided on the left side door mirror. Then, the imaging unit 15d can capture an image of a region on the side of the vehicle 1 in the periphery of the vehicle 1.

  Next, the functional configuration of the vehicle according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram showing an example of the functional configuration of the vehicle according to the first embodiment. As illustrated in FIG. 3, the vehicle 1 includes an ECU (Electronic Control Unit) 14, a monitor device 11, a steering system 13, a brake system 18, a steering angle sensor 19, an accelerator sensor 20, a shift sensor 21, a wheel speed sensor 22. Etc. are electrically connected via an in-vehicle network 23 as a telecommunication line. The in-vehicle network 23 is configured as a CAN (Controller Area Network), for example.

  The steering system 13 is an electric power steering system, an SBW (Steer By Wire) system, or the like. The steering system 13 has an actuator 13a and a torque sensor 13b. The steering system 13 is electrically controlled by the ECU 14 and the like, operates the actuator 13a, applies torque to the steering unit 4 to supplement the steering force, and steers the wheels 3. The torque sensor 13b detects the torque applied to the steering section 4 by the driver and transmits the detection result to the ECU 14.

  The brake system 18 is an ABS (Anti-lock Brake System) that controls the lock of the brake of the vehicle 1, a skid prevention device (ESC: Electronic Stability Control) that suppresses the skid of the vehicle 1 during cornering, and enhances the braking force. Includes electric braking system that assists braking and BBW (Brake By Wire).

  The brake system 18 has an actuator 18a and a brake sensor 18b. The brake system 18 is electrically controlled by the ECU 14 and the like, and applies a braking force to the wheels 3 via the actuator 18a. The brake system 18 detects a lock of the brake, an idling of the wheel 3, a sign of skidding, and the like from the rotational difference between the left and right wheels 3, and controls to lock the brake, prevent the wheel 3 from idling, and skid. Run. The brake sensor 18b is a displacement sensor that detects the position of a brake pedal that is a movable part of the braking operation unit, and transmits the detection result of the position of the brake pedal to the ECU 14.

  The steering angle sensor 19 is a sensor that detects a steering amount of the steering unit 4 such as a steering wheel. In the present embodiment, the steering angle sensor 19 is configured by a Hall element or the like, detects the rotation angle of the rotating portion of the steering unit 4 as the steering amount, and transmits the detection result to the ECU 14. The accelerator sensor 20 is a displacement sensor that detects the position of an accelerator pedal that is a movable portion of the acceleration operation unit, and sends the detection result to the ECU 14.

  The shift sensor 21 is a sensor that detects the position of the movable portion (bar, arm, button, etc.) of the gear shift operation unit, and sends the detection result to the ECU 14. The wheel speed sensor 22 is a sensor that has a hall element or the like and detects the rotation amount of the wheel 3 and the rotation number of the wheel 3 per unit time, and transmits the detection result to the ECU 14.

  The ECU 14 is composed of a computer or the like, and controls the entire vehicle 1 by cooperation of hardware and software. Specifically, the ECU 14 includes, for example, a CPU (Central Processing Unit) 14a, a ROM (Read Only Memory) 14b, a RAM (Random Access Memory) 14c, a display control unit 14d, a voice control unit 14e, an SSD (Solid State Drive, Flash memory) 14f and the like. The CPU 14a controls the entire vehicle 1. The CPU 14a can read a program installed and stored in a non-volatile storage device such as the ROM 14b, and execute arithmetic processing according to the program.

  The ROM 14b stores various programs and parameters necessary for executing the programs. The RAM 14c temporarily stores various data used in the calculation by the CPU 14a. The display control unit 14d mainly performs image processing using image data obtained by image capturing of the image capturing unit 15 provided so as to capture an image of the outside of the vehicle 1 in the arithmetic processing performed by the ECU 14, and is displayed by a display device. The image data composition is executed. The voice control unit 14e mainly executes the process of the voice data output from the voice output device in the calculation process of the ECU 14. The SSD 14f is a rewritable non-volatile storage unit and can store data even when the power of the ECU 14 is turned off.

  The CPU 14a, the ROM 14b, the RAM 14c, etc. can be integrated in the same package. Further, the ECU 14 may be configured to use another logic operation processor such as a DSP (Digital Signal Processor) or a logic circuit instead of the CPU 14a. An HDD (Hard Disk Drive) may be provided instead of the SSD 14f, or the SSD 14f and the HDD may be provided separately from the ECU 14. The configurations, arrangements, electrical connection forms, and the like of the various sensors and actuators described above are merely examples, and can be set (changed) in various ways.

  FIG. 4 is a block diagram showing an example of a functional configuration of an ECU included in the vehicle according to the first embodiment. As shown in FIG. 4, the ECU 14 mainly includes a pattern image storage unit 400, an image acquisition unit 401, a line-of-sight detection unit 402, a pattern image display control unit 403, an individual authentication unit 404, and a calibration processing unit 405. There is. In the image acquisition unit 401, the line-of-sight detection unit 402, the pattern image display control unit 403, the personal authentication unit 404, and the calibration processing unit 405 illustrated in FIG. 4, the CPU 14a included in the ECU 14 executes the program stored in the ROM 14b. It will be realized. These configurations may be implemented by hardware.

  The pattern image storage unit 400 is realized by a non-volatile storage medium such as the SSD 14f, and displays a pattern image for personal authentication (hereinafter referred to as a pattern image for personal authentication) displayed in the field of view of the user such as the front window of the vehicle 1. Remember. Here, the personal authentication pattern image is a pattern image used for authenticating the driver 302 of the vehicle 1, and is, for example, an image in which a plurality of symbols (for example, dots) are arranged in a preset pattern.

  The image acquisition unit 401 acquires a captured image obtained by capturing an image of the driver 302 with the driver monitor camera 201.

  The line-of-sight detection unit 402 detects the line of sight of the driver 302 of the vehicle 1. In the present embodiment, the line-of-sight detection unit 402 detects the image of the eyes of the driver 302 from the captured image acquired by the image acquisition unit 401, and detects the line of sight of the driver 302 based on the detected image of the eyes. To do.

  The pattern image display control unit 403 displays a personal authentication pattern image in the field of view of the driver 302. In the present embodiment, the pattern image display control unit 403 reads the personal authentication pattern image from the pattern image storage unit 400, and displays the read personal authentication pattern image on the entire front window of the vehicle 1.

  For example, when the pattern image display control unit 403 displays an image in which a plurality of points are arranged in a preset pattern as a personal authentication pattern image, each point included in the personal authentication pattern image is the front of the vehicle 1. The pattern image for personal authentication is displayed on the front window so that it is distributed over the entire window.

  In the present embodiment, the pattern image display control unit 403 displays the personal authentication pattern image in the visual field of the driver 302 before the driver 302 starts driving the vehicle 1 (before starting the engine of the vehicle 1). I shall.

  In the present embodiment, the pattern image display control unit 403 has described the example in which the personal authentication pattern image is displayed on the front window of the vehicle 1. However, the pattern image display control unit 403 displays the personal authentication pattern image in the field of view of the driver 302. If so, it is not limited to this. For example, when the vehicle 1 has a HUD (Head Up Display), the pattern image display control unit 403 may display the personal authentication pattern image on the HUD. Further, when the vehicle 1 has a projector, the pattern image display control unit 403 may project the personal authentication pattern image on the screen provided in the front window or the like of the vehicle 1.

  The personal authentication unit 404 compares the line-of-sight movement pattern detected by the line-of-sight detection unit 402 with a predetermined movement pattern before the driver 302 starts driving the vehicle 1, and performs personal identification processing of the driver 302. Run. Here, the predetermined movement pattern is a preset pattern in which the driver 302 moves the line of sight.

  In the present embodiment, it is assumed that the predetermined movement pattern for each driver 302 of the vehicle 1 is stored in a nonvolatile storage medium such as the SSD 14f. The personal authentication unit 404 executes the personal authentication processing of the driver 302 depending on whether or not the movement pattern of the line of sight detected on the personal authentication pattern image matches any of the predetermined movement patterns stored in the SSD 14f. To do.

  Then, when the movement pattern of the line of sight detected by the line-of-sight detection unit 402 and the predetermined movement pattern match, the personal authentication unit 404 determines that the personal authentication of the driver 302 has succeeded. In this case, the personal authentication unit 404 starts the drive unit (for example, engine, motor) of the vehicle 1.

  On the other hand, if the movement pattern of the line of sight detected by the line-of-sight detection unit 402 does not match the predetermined movement pattern, the personal authentication unit 404 determines that the personal authentication of the driver 302 has failed. In this case, the personal authentication unit 404 does not start the drive unit of the vehicle 1. As a result, the vehicle 1 can be prevented from being stolen.

  The personal authentication of the driver 302 can be performed based on the face image included in the captured image acquired by the image acquisition unit 401. However, the accuracy of the personal authentication of the driver 302 based on the face image is high. Sometimes insufficient. Therefore, by performing personal authentication using the movement pattern of the line of sight detected by the line-of-sight detection unit 402, it is possible to improve the accuracy of personal authentication of the driver 302 and more effectively prevent the vehicle 1 from being stolen. ..

  Further, when the line-of-sight detection unit 402 detects a line of sight of a movement pattern different from the predetermined movement pattern continuously for a predetermined number of times (in other words, the personal authentication processing of the driver 302 is preset. If the number of consecutive failures has failed), the start of driving the vehicle 1 is locked (prohibited). Accordingly, it is possible to prevent the driver 302's personal authentication process from being repeated until the line-of-sight of the movement pattern that matches the predetermined movement pattern is detected by the line-of-sight detection unit 402, so that the vehicle 1 can be stolen more effectively. Can be prevented. Here, the predetermined number of times is a preset number of times, and can be arbitrarily set by the driver 302.

  Here, locking the start of the drive unit of the vehicle 1 means that if the personal authentication process of the driver 302 has failed a predetermined number of times in succession, then the line-of-sight movement pattern detected by the line-of-sight detection unit 402 is , Even if it matches the predetermined movement pattern, the drive unit of the vehicle 1 is not started.

  Furthermore, when the line-of-sight detection unit 402 detects a line-of-sight of a movement pattern different from the predetermined movement pattern continuously for a predetermined number of times, the personal authentication unit 404 acquires a captured image (in other words, the line-of-sight). The captured image used for the detection unit 402 to detect the line of sight of the driver 302 is stored in a nonvolatile storage medium such as the SSD 14f. As a result, it becomes possible to identify the driver 302 who is going to commit an illegal act such as theft of the vehicle 1 based on the face image included in the captured image stored in the SSD 14f.

  The calibration processing unit 405 uses the line-of-sight detection unit 402 based on the result of comparison between the line-of-sight movement pattern detected by the line-of-sight detection unit 402 and a predetermined movement pattern before the driver 302 starts driving the vehicle 1. Calibrate the detected line of sight. As a result, before the driving of the vehicle 1 is started, calibration of the line-of-sight detected by the line-of-sight detection unit 402 is performed using the movement pattern of the line of sight of the driver 302 with respect to the personal authentication pattern displayed in the field of view of the driver 302. Becomes feasible. As a result, the line-of-sight by the line-of-sight detection unit 402 can be detected with high accuracy from the start of driving the vehicle 1.

  Here, the calibration of the line of sight detected by the line-of-sight detection unit 402 is performed by comparing a movement pattern of the line-of-sight detected by the line-of-sight detection unit 402 with a predetermined movement pattern, The movement pattern is compared with a predetermined movement pattern to detect the difference, or the movement pattern of the line of sight detected by the line-of-sight detection unit 402 is compared with the predetermined movement pattern to detect the difference and the detection. This is to adjust the detection result of the visual line by the visual line detection unit 402 based on the difference.

  Next, an example of the flow of calibration of the line of sight detected by the ECU 14 of the vehicle 1 according to the present embodiment will be described with reference to FIGS. 5 and 6. FIG. 5 is a flowchart showing an example of the flow of line-of-sight calibration detected by the ECU of the vehicle according to the first embodiment. FIG. 6 is a diagram for explaining an example of a personal authentication process of a driver by the vehicle according to the first embodiment.

  When the start key of the vehicle 1 is instructed by the ignition key or the start button of the vehicle 1, the pattern image display control unit 403, prior to the start of the drive unit of the vehicle 1, causes the pattern image storage unit 400 to perform personal authentication. The read pattern image is read, and the read personal authentication pattern image is displayed on the entire front window of the vehicle 1 (step S501).

  For example, as shown in FIG. 6, the pattern image display control unit 403 displays the personal authentication pattern image G including the nine points P1 to P9 in the front window W of the vehicle 1. At that time, as shown in FIG. 6, the pattern image display control unit 403 sets the personal authentication pattern image G so that the points P1 to P9 included in the personal authentication pattern image G are dispersed over the entire front window W. It is preferable to display it on the front window W.

  Returning to FIG. 5, when the personal authentication pattern image is displayed on the front window of the vehicle 1, the image acquisition unit 401 acquires a captured image obtained by capturing the image of the driver 302 by the driver monitor camera 201 (step S502). ). The sight line detection unit 402 detects the sight line of the driver 302 based on the captured image acquired by the image acquisition unit 401 (step S503).

  The personal authentication unit 404 compares the line-of-sight movement pattern detected by the line-of-sight detection unit 402 with a predetermined movement pattern, and executes personal authentication processing of the driver 302 (step S504). Specifically, the personal authentication unit 404 reads out a predetermined movement pattern from the SSD 14f. Here, the movement pattern is, for example, a movement pattern of a line of sight that connects at least two points among a plurality of points included in the personal authentication pattern image with a single stroke.

  Next, the personal authentication unit 404 compares the line-of-sight movement pattern detected by the line-of-sight detection unit 402 with a predetermined movement pattern. For example, as shown in FIG. 6, the personal authentication unit 404 detects a plurality of one of the nine points P1 to P9 included in the personal-authentication pattern image and the movement pattern 601 of the single-line gaze detected by the line-of-sight detection unit 402. A point (for example, points P1, P2, P3, P5, P7, P8, P9) is compared with a predetermined movement pattern 602 that connects the points with one stroke.

  Then, when the movement pattern 601 of the line of sight detected by the line-of-sight detection unit 402 and the predetermined movement pattern 602 match, the personal authentication unit 404 determines that the driver 302 has been successfully authenticated, and the vehicle 1 Start the drive of. In the present embodiment, the personal authentication unit 404 determines that the positional shift of the line-of-sight movement pattern 601 detected by the line-of-sight detection unit 402 based on the predetermined movement pattern 602 is within a preset allowable range. It is assumed that the movement pattern 601 of the sight line detected by the sight line detection unit 402 and the predetermined movement pattern 602 are determined to match.

  On the other hand, if the movement pattern 601 of the line of sight detected by the line-of-sight detection unit 402 and the predetermined movement pattern 602 do not match, the personal authentication unit 404 determines that the authentication of the driver 302 has failed, Do not start the drive unit of the vehicle 1. In the present embodiment, the personal authentication unit 404 determines that the displacement of the line-of-sight movement pattern 601 detected by the line-of-sight detection unit 402 based on the predetermined movement pattern 602 is outside a preset allowable range. It is determined that the line-of-sight movement pattern 601 detected by the line-of-sight detection unit 402 and the predetermined movement pattern 602 do not match.

  Returning to FIG. 5, the personal authentication unit 404 determines whether the personal authentication processing of the driver 302 has succeeded (step S505). When the personal authentication process of the driver 302 has failed (step S505: No), the personal authentication unit 404 determines whether the personal authentication process of the driver 302 has failed a predetermined number of times consecutively (step S506). .. In other words, the personal authentication unit 404 determines whether or not the line-of-sight detection unit 402 has detected a line-of-sight of a movement pattern different from the predetermined movement pattern continuously for a predetermined number of times.

  When the personal authentication processing of the driver 302 has not failed a predetermined number of times consecutively (step S506: No), the personal authentication unit 404 returns to step S504 and executes the personal authentication processing of the driver 302 again. On the other hand, when the personal authentication process of the driver 302 has failed a predetermined number of times consecutively (step S506: Yes), the personal authentication unit 404 locks the start of the drive unit of the vehicle 1 (step S507). Further, the personal authentication unit 404 stores the captured image used by the visual line detection unit 402 to detect the visual line of the driver 302 in the SSD 14f (step S508).

  If it is determined in step S505 that the personal authentication process of the driver 302 has succeeded (step S505: Yes), the personal authentication unit 404 starts the drive unit of the vehicle 1 (step S509). Further, the calibration processing unit 405 executes calibration of the line of sight detected by the line-of-sight detection unit 402 based on the result of comparison between the movement pattern of the line-of-sight detected by the line-of-sight detection unit 402 and a predetermined movement pattern. (Step S510).

  In the present embodiment, when it is determined that the personal identification process of the driver 302 has succeeded, the sight line detected by the sight line detection unit 402 is calibrated after the drive unit of the vehicle 1 is started. The present invention is not limited to this as long as the line-of-sight calibration detected by the line-of-sight detection unit 402 is executed before the driver 302 starts driving the vehicle. For example, when the driver 032's personal authentication processing is successful, the calibration processing unit 405 may execute the calibration of the visual line detected by the visual line detection unit 402 before starting the drive unit of the vehicle 1. ..

  As described above, according to the vehicle 1 according to the first embodiment, the movement pattern of the line of sight of the driver 302 with respect to the personal authentication pattern displayed in the field of view of the driver 302 is used before the driving of the vehicle 1 is started. Thus, the sight line detected by the sight line detection unit 402 can be calibrated, so that the sight line detection unit 402 can detect the sight line with high accuracy from the start of driving of the vehicle 1.

(Second embodiment)
The present embodiment detects an obstacle existing around the vehicle, and after the driver starts driving the vehicle, causes the driver's field of view to display obstacle information indicating the position of the detected obstacle, In this example, the position of obstacle information in the driver's field of view is compared with the detected driver's line of sight, and the detected line of sight is calibrated. In the following description, a configuration different from that of the first embodiment will be described.

  FIG. 7 is a block diagram showing an example of a functional configuration of an ECU included in the vehicle according to the second embodiment. As shown in FIG. 7, in the present embodiment, the ECU 700 includes a pattern image storage unit 400, an image acquisition unit 701, a line-of-sight detection unit 402, a pattern image display control unit 702, an individual authentication unit 404, a calibration processing unit 703, and Also, an obstacle detection unit 704 is mainly provided. The image acquisition unit 701, the line-of-sight detection unit 402, the pattern image display control unit 702, the personal authentication unit 404, the calibration processing unit 703, and the obstacle detection unit 704 illustrated in FIG. 7 are stored in the ROM 14b by the CPU 14a included in the ECU 700. It is realized by executing the specified program. These configurations may be implemented by hardware.

  The image acquisition unit 701 acquires a captured image obtained by capturing an image of the surroundings of the vehicle 1 by the imaging unit 15 after the driving of the vehicle 1 is started (hereinafter, referred to as a surrounding captured image). In the present embodiment, the image acquisition unit 701 acquires the surrounding captured image obtained by the image capturing of the image capturing unit 15. However, the vehicle 1 has a LIDAR (Light Detection and Ranging) camera, a TOF (Time Of Flight) camera, and a millimeter. An image representing the measurement result of the distance between the vehicle 1 and the surrounding objects measured by the wave radar, the ultrasonic sensor, or the like may be acquired as the ambient image.

  The obstacle detection unit 704 detects an obstacle existing around the vehicle 1. In the present embodiment, the obstacle detection unit 704 detects an obstacle existing around the vehicle 1 based on the surrounding captured image acquired by the image acquisition unit 701.

  The pattern image display control unit 702 displays obstacle information in the field of view of the driver 302 when the obstacle detection unit 704 detects an obstacle after the driver 302 starts driving the vehicle 1. Here, the obstacle information is information indicating the position of the obstacle detected by the obstacle detection unit 704. In the present embodiment, the pattern image display control unit 702 causes the driver 302 to detect the detected obstacle on the front window of the vehicle 1 at a position corresponding to the obstacle detected by the obstacle detection unit 704. An image that prompts the user to direct his or her line of sight to an object is displayed as obstacle information. Here, the corresponding position is a position where the driver 302 can visually recognize the obstacle on the front window.

  The calibration processing unit 703 compares the corresponding position with the line of sight detected by the line-of-sight detection unit 402, and executes calibration of the line-of-sight detected by the line-of-sight detection unit 402. As a result, the line-of-sight detected by the line-of-sight detection unit 402 can be calibrated even after the operation of the vehicle 1 is started, and thus the line-of-sight detection unit 402 can detect the line of sight with higher accuracy.

  Next, an example of the flow of calibration of the line of sight detected by the ECU 700 of the vehicle 1 according to the present embodiment will be described with reference to FIGS. 8 and 9. FIG. 8 is a flowchart showing an example of the flow of calibration detected by the ECU of the vehicle according to the second embodiment. FIG. 9 is a diagram showing an example of a process of displaying obstacle information by the vehicle according to the second embodiment.

  When the drive unit of the vehicle 1 is started by the personal authentication unit 404 and the driving of the vehicle 1 by the driver 302 is started, the image acquisition unit 701 causes the image capturing unit 15 to capture an image of the surroundings of the vehicle 1 and obtain a surrounding image. An image is acquired (step S801). Further, the obstacle detection unit 704 detects an obstacle existing around the vehicle 1 on the basis of the acquired surrounding captured image (step S802). When the obstacle detection unit 704 does not detect an obstacle existing around the vehicle 1 (step S803: No), the process returns to step S801.

  On the other hand, when the obstacle detection unit 704 detects an obstacle existing around the vehicle 1 (step S803: Yes), the pattern image display control unit 702 causes the detected obstacle on the front window of the vehicle 1. The obstacle information is displayed at the corresponding position corresponding to (step S804).

  For example, as shown in FIG. 9, the pattern image display control unit 702 causes the driver 302 to visually recognize an obstacle 901 (for example, another vehicle) on the front window W of the vehicle 1 at a corresponding position 902. A frame 903 surrounding the obstacle 901 is displayed as obstacle information. As a result, the pattern image display control unit 702 prompts the driver 302 to direct his or her line of sight to the position where the obstacle information is displayed on the front window W.

  Returning to FIG. 8, when the obstacle information is displayed on the front window of the vehicle 1, the calibration processing unit 703 detects the corresponding position on the front window at which the obstacle information is displayed and the line-of-sight detection unit 402. The line-of-sight is compared and the line-of-sight is calibrated (step S805).

  For example, as shown in FIG. 9, the calibration processing unit 703 compares the line of sight 904 detected by the line of sight detection unit 402 with the position of the obstacle 901 (or the corresponding position 902) and detects the difference. To do. Then, the calibration processing unit 703 executes calibration for adjusting the detection result of the visual line detected by the visual line detection unit 402 based on the detected difference.

  As described above, according to the vehicle 1 according to the second embodiment, it is possible to execute the calibration of the visual line detected by the visual line detection unit 402 even after the driving of the vehicle 1 is started. The line of sight can be detected with higher accuracy.

1 vehicle 14 ECU
14a CPU
14b ROM
14c RAM
14f SSD
15 image pickup unit 201 driver monitor camera 400 pattern image storage unit 401, 701 image acquisition unit 402 line-of-sight detection unit 403, 702 pattern image display control unit 404 individual authentication unit 405, 703 calibration processing unit 704 obstacle detection unit

Claims (5)

A line-of-sight detection unit that detects the line of sight of the driver of the vehicle,
A pattern image display control unit for displaying a pattern image for personal authentication with respect to the field of view of the driver,
Before the driver starts driving the vehicle, a personal authentication unit that performs a personal authentication process of the driver by comparing a movement pattern of the line of sight detected by the line of sight detection unit with a predetermined movement pattern. ,
Based on the comparison result of the movement pattern of the line of sight detected by the line-of-sight detection unit and the predetermined movement pattern, a calibration processing unit that performs calibration of the line-of-sight detected by the line-of-sight detection unit,
A visual axis detection device comprising.   The line-of-sight detection device according to claim 1, wherein the personal authentication unit starts the drive unit of the vehicle when the personal authentication process of the driver is successful.   The personal authentication unit further prohibits starting of the drive unit of the vehicle when the line-of-sight detection unit detects a line-of-sight of a movement pattern different from the predetermined movement pattern continuously a predetermined number of times. The line-of-sight detection device described.   The personal authentication unit, when the line of sight of a movement pattern different from the predetermined movement pattern is detected by the line-of-sight detection unit continuously for the predetermined number of times, an image obtained by imaging the driver's face by the imaging unit. The visual line detection device according to claim 3, wherein the image is stored in a storage unit. Further comprising an obstacle detection unit for detecting an obstacle existing around the vehicle,
The pattern image display control unit further displays obstacle information indicating the detected position of the obstacle with respect to the field of view of the driver after the driver starts driving the vehicle,
The calibration processing unit further compares the position of the obstacle information in the visual field of the driver with the line of sight detected by the line-of-sight detection unit, and calibrates the line-of-sight detected by the line-of-sight detection unit. The line-of-sight detection device according to any one of claims 1 to 4, which executes.

Images