JP6950597B2 - On-board unit, driving evaluation system, information processing method, and program - Google Patents

Description

The present invention relates to an on-board unit, a driving evaluation system, an information processing method, and a program.

In Patent Document 1, a driver's face image is imaged by a camera arranged on the steering column, the upper and lower contour positions and the center line of the face are detected from the edge information of the captured face image, and the detected center line is detected. Disclosed is a technique for detecting the face orientation of a driver based on the above, that is, detecting the face orientation in the left-right direction when the front direction is zero degree based on the distance from the center line to the contour position of the face. There is.

[Problems to be solved by the invention]
In the driver monitor sensor described in Patent Document 1, the driver's face is photographed from the front by a camera, and the degree of face orientation is detected with reference to a preset front direction.

By the way, there are individual differences in the seating position and driving posture of the driver. Even when the driver is looking in the front direction of the vehicle, if the seating position and driving posture of the driver are different, the orientation of the driver's face in the image captured by the camera is different for each driver. Is detected by.

Further, when the driver monitor sensor is retrofitted to the vehicle, the camera of the driver monitor sensor may not be installed at a position where the driver's face can be photographed from the front. When the camera is installed at a position offset from the front of the driver, if the driver's sitting position and driving posture are different, the orientation of the driver's face in the image is likely to be detected in a different orientation for each driver. ..

In the driver monitor sensor described in Patent Document 1, even if there is a deviation between the reference front direction, which is preset on the device side, and the face orientation when the driver is looking at the front direction. The degree of face orientation is detected without considering this deviation. Therefore, in the driver monitor sensor described in Patent Document 1, the detected face orientation does not reflect the individual difference in the face orientation of each driver when the driver is driving, and the face orientation is not reflected. There is a problem that the detection accuracy becomes low.

Japanese Unexamined Patent Publication No. 2009-232945

Means for solving problems and their effects

The present invention has been made in view of the above problems, and provides an in-vehicle device, a driving evaluation system, an information processing method, and a program capable of improving the accuracy of processing using the direction of the driver's face or the direction of the line of sight. The purpose is.

In order to achieve the above object, the in-vehicle device (1) according to the present disclosure is
The acquisition department that acquires the authentication information of the driver who drives the vehicle,
A calculation unit that calculates reference information for the driver's face direction or line-of-sight direction,
The authentication information of each of the drivers acquired by the acquisition unit and the reference information of the direction of each face or the direction of the line of sight of each of the drivers calculated by the calculation unit are associated with each driver. Registration department to register with
An authentication unit that authenticates the driver based on the information registered by the registration unit and the authentication information of the driver acquired by the acquisition unit.
When the driver is authenticated by the authentication unit, the certified reference information of the driver's face direction or the line-of-sight direction is acquired from the information registered by the registration unit, and the acquired face of the driver is acquired. It is characterized in that it includes a first processing unit that performs a predetermined process using reference information of the direction of the head or the direction of the line of sight.

According to the in-vehicle device (1), each authentication information of the driver and reference information of each face direction or line-of-sight direction of the driver are registered in association with each other for each driver. A predetermined process is performed using the reference information of the face direction or the line-of-sight direction of each of the registered drivers. Therefore, the predetermined process can be executed so that the influence of individual differences in the direction of each face or the direction of the line of sight of the driver is reduced, and the predetermined process is accurately executed for each driver. be able to.

Further, the in-vehicle device (2) according to the present disclosure uses preset reference information of the face direction or the line-of-sight direction when the driver is not authenticated by the authentication unit in the in-vehicle device (1). It is characterized in that it is provided with a second processing unit that performs a predetermined processing.

According to the in-vehicle device (2), when the driver is not authenticated by the authentication unit, a predetermined process is performed using the preset reference information of the face direction or the line-of-sight direction. Even if the driver is not authenticated by the authentication unit, the predetermined process can be appropriately executed.

Further, the vehicle-mounted device (3) according to the present disclosure includes an imaging unit for imaging the driver in the vehicle-mounted device (1) or (2), and the calculation unit is the driver imaged by the imaging unit. It is characterized in that the reference information of the direction of the driver's face or the direction of the line of sight is calculated from the image of.

According to the in-vehicle device (3), the reference information of the driver's face direction or the line-of-sight direction can be efficiently calculated.

Further, in the in-vehicle device (4) according to the present disclosure, in the in-vehicle device (3), the reference information of the direction of the driver's face or the direction of the line of sight is a plurality of images of the driver captured by the imaging unit. It is characterized in that it includes statistical values of the driver's face direction or line-of-sight direction statistically processed using the above.

According to the in-vehicle device (4), the reference information of the driver's face direction or the line-of-sight direction is a statistical value obtained by statistically processing using a plurality of images of the driver captured by the imaging unit. Since it is included, the accuracy of the reference information of the direction of each face or the direction of the line of sight of the driver can be improved. As the statistical value of the driver's face direction or line-of-sight direction, for example, any of the mode value, the median value, or the average value of the face direction or the line-of-sight direction can be used.

Further, in the vehicle-mounted device (5) according to the present disclosure, in the vehicle-mounted device (3) or (4), the acquisition unit acquires the authentication information from the driver's image captured by the imaging unit. It is characterized by being.

According to the in-vehicle device (5), since the imaging unit can also be used for acquiring the authentication information, it is not necessary to add equipment for acquiring the authentication information, and the device cost can be suppressed. can.

Further, in the in-vehicle device (6) according to the present disclosure, in any of the in-vehicle devices (1) to (5), the reference information of the driver's face direction or the line-of-sight direction is set in advance. Alternatively, it is characterized by containing information indicating a deviation from the reference in the direction of the line of sight.

According to the in-vehicle device (6), the predetermined process can be executed by correcting the deviation from the preset reference of the face direction or the line-of-sight direction, and a predetermined process is performed for each driver. The process can be executed accurately.

Further, the vehicle-mounted device (7) according to the present disclosure is characterized in that, in any of the vehicle-mounted devices (1) to (6), the predetermined process is the driver's inattentive determination process.

According to the in-vehicle device (7), the driver's inattentiveness determination process is performed using the reference information of the driver's face orientation or the line-of-sight direction, so that the driver's face orientation is performed. Alternatively, the inattentive determination process in consideration of the direction of the line of sight, in other words, the individual difference in the direction of the face or the direction of the line of sight can be executed, and the inattentive determination process can be executed accurately for each driver.

Further, the vehicle-mounted device (8) according to the present disclosure is characterized in that, in any of the vehicle-mounted devices (1) to (6), the predetermined process is a determination process of the driver's safety confirmation operation at an intersection. It is supposed to be.

According to the in-vehicle device (8), the driver's safety confirmation operation at the intersection is determined by using the reference information of each driver's face direction or line-of-sight direction. It is possible to execute the safety confirmation operation determination process in consideration of individual differences in the direction of the face or the direction of the line of sight, in other words, the direction of the face or the direction of the line of sight, and the safety confirmation is performed for each driver. The operation determination process can be executed with high accuracy.

Further, the driving evaluation system according to the present disclosure is a driving evaluation system that evaluates the driving of the driver of the vehicle based on the information acquired from the in-vehicle device mounted on the vehicle.
An acquisition unit that acquires the authentication information of the driver who drives the vehicle, and
A calculation unit that calculates reference information for the driver's face direction or line-of-sight direction,
The authentication information of each of the drivers acquired by the acquisition unit and the reference information of the direction of each face or the direction of the line of sight of each of the drivers calculated by the calculation unit are associated with each driver. Registration department to register with
An authentication unit that authenticates the driver based on the information registered by the registration unit and the authentication information of the driver acquired by the acquisition unit.
When the driver is authenticated by the authentication unit, the certified reference information of the driver's face direction or the line-of-sight direction is acquired from the information registered by the registration unit, and the acquired face of the driver is acquired. It is characterized in that it is provided with a processing unit that performs a predetermined process using reference information of the direction of the head or the direction of the line of sight.

According to the driving evaluation system, each authentication information of the driver and reference information of the direction of each face or the direction of the line of sight of the driver are registered in association with each driver, and the registration is performed. Predetermined processing is performed using the reference information of the direction of each face or the direction of the line of sight of the driver. Therefore, the predetermined process can be executed so that the influence of individual differences in the direction of each face or the direction of the line of sight of the driver is reduced, and the predetermined process, for example, driving evaluation, can be performed for each driver. The process can be executed accurately.

The information processing method according to this disclosure is
The acquisition step to acquire the authentication information of the driver who drives the vehicle, and
A calculation step for calculating reference information of the driver's face direction or line-of-sight direction, and
The authentication information of each of the drivers acquired by the acquisition step is associated with the reference information of the face direction or the line-of-sight direction of each of the drivers calculated by the calculation step for each driver. And the registration step to register in the memory
An authentication step for authenticating the driver based on the information registered in the storage unit by the registration step and the authentication information of the driver acquired by the acquisition step.
When the driver is authenticated by the authentication step, the authenticated reference information of the driver's face direction or the line-of-sight direction is acquired from the information registered in the storage unit, and the acquired driver's face is acquired. It is characterized by including a processing step of performing a predetermined processing using reference information of the direction of the line of sight or the direction of the line of sight.

According to the information processing method, each authentication information of the driver and reference information of each face direction or line-of-sight direction of the driver are associated with each driver and registered in the storage unit. Then, a predetermined process is performed using the reference information of the face direction or the line-of-sight direction of each of the drivers registered in the storage unit. Therefore, the predetermined process can be executed so that the influence of individual differences in the direction of each face or the direction of the line of sight of the driver is reduced, and the predetermined process is accurately executed for each driver. be able to.

The information processing program related to this disclosure is
On at least one computer
The acquisition step to acquire the authentication information of the driver who drives the vehicle, and
A calculation step for calculating reference information of the driver's face direction or line-of-sight direction, and
The authentication information of each of the drivers acquired by the acquisition step is associated with the reference information of the face direction or the line-of-sight direction of each of the drivers calculated by the calculation step for each driver. And the registration step to register in the memory
An authentication step for authenticating the driver based on the information registered in the storage unit by the registration step and the authentication information of the driver acquired by the acquisition step.
When the driver is authenticated by the authentication step, the authenticated reference information of the driver's face direction or the line-of-sight direction is acquired from the information registered in the storage unit, and the acquired driver's face is acquired. It is characterized in that a processing step of performing a predetermined processing is executed by using the reference information of the direction of the line of sight or the direction of the line of sight.

According to the above program, the authentication information of each of the drivers and the reference information of the direction of each face or the direction of the line of sight of the drivers are associated with each other by at least one computer. A predetermined process is executed using the reference information registered in the storage unit and the direction of each face or the direction of the line of sight of the driver registered in the storage unit. Therefore, the predetermined process can be executed so as to reduce the influence of individual differences in the direction of each face or the direction of the line of sight of the driver, and the predetermined process is accurately executed for each driver. A device or system capable of being realized can be realized.

It is an example of the functional block diagram which shows the characteristic part of the vehicle-mounted device which concerns on embodiment. It is a block diagram which shows the composition of the main part of the vehicle-mounted device which concerns on embodiment. It is a schematic block diagram which shows an example of the operation evaluation system to which the vehicle-mounted device which concerns on embodiment is applied. It is a flowchart which shows an example of the processing operation performed by the monitoring unit of the vehicle-mounted device which concerns on embodiment. It is a flowchart which shows an example of the processing operation performed by the control unit of the vehicle-mounted device which concerns on embodiment. It is a flowchart which shows an example of the processing operation performed by the control unit of the vehicle-mounted device which concerns on embodiment. It is a figure which shows an example of the structure of the data transmitted from an in-vehicle device to a server device. It is a flowchart which shows an example of the processing operation performed by the central processing unit of the server device which concerns on embodiment. It is an example of the functional block diagram which shows the characteristic part of the operation evaluation system which concerns on another embodiment.

Hereinafter, an in-vehicle device, an operation evaluation system, an information processing method, and an embodiment of a program according to the present invention will be described with reference to the drawings.

[Application example]
FIG. 1 is an example of a functional configuration block diagram showing a feature portion of the vehicle-mounted device according to the embodiment.
The on-board unit 10 is, for example, a device that is retrofitted to the vehicle 2, and is a device that monitors the state of the driver of the vehicle 2 with the driver camera 21, or acquires data used for the driver's driving evaluation and externally. It is applied as a device to transmit to.

The driver's condition includes at least one of the driver's face orientation, gaze direction, eye opening, drowsiness, and inattentiveness.
The data used for the driver's driving evaluation includes at least one of the driver's condition, the position of the vehicle 2, and the behavior of the vehicle 2.
The driver's driving evaluation includes, for example, an evaluation of the driver's driving operation behavior, an evaluation of a safety confirmation operation at an intersection, an evaluation of a concentration state during driving, and the like.

The driver's driving behavior includes improper driving behavior such as sudden steering, sudden acceleration, sudden deceleration, sudden start, overspeed, or impact detection.
The safety confirmation operation at an intersection includes the driver's safety confirmation timing, confirmation angle, confirmation time, or approach speed to the intersection at each intersection that has passed.
The concentrated state during driving includes the driver's drowsiness detection state (time, frequency), inattentive detection state (time, frequency), and the like.

The vehicle 2 on which the vehicle-mounted device 10 is mounted is not particularly limited, but in the present embodiment, a vehicle managed by a business operator engaged in various businesses can be targeted. For example, trucks managed by transportation companies, buses managed by bus companies, taxis managed by taxi companies, car sharing vehicles managed by car sharing companies, car rentals managed by car rental companies, owned by companies. Company-owned vehicles that are owned by the company, company-owned vehicles that are leased from a car leasing company, etc. can be targeted.

In the vehicle 2 used for the above-mentioned purpose, the driver may be replaced. In addition, the seating position (for example, seat position, etc.) and driving posture of the driver vary from driver to driver.
Therefore, when monitoring the driver's condition with the driver camera 21 of the on-board unit 10, for example, even if the driver is looking in the front direction of the vehicle 2, the seating position and the driving posture of the driver are different. The orientation of the driver's face in the image captured by the driver camera 21 is detected in a different orientation for each driver.

Further, when the vehicle-mounted device 10 is retrofitted to the vehicle 2, the driver camera 21 may not be installed at a position where the driver's face can be photographed from the front. When the driver camera 21 is installed at a position deviated from the front of the driver, the orientation of the driver's face in the image of the driver camera 21 differs for each driver if the seating position and the driving posture of the driver are different. The orientation makes it easier to detect.

Therefore, when detecting the direction of the driver's face or the direction of the line of sight, for example, when performing a process of determining the driver's inattentiveness or determining the driver's safety confirmation operation, the device side in advance. There may be a discrepancy between the set reference for the front direction and the direction of the face or the direction of the line of sight when the driver is looking in the front direction. If the direction of the face or the direction of the line of sight is detected without considering this deviation, the detection accuracy of the direction of the face or the direction of the line of sight will be lowered, and the accuracy of determining the driver's inattentiveness and safety confirmation operation will also be lowered. It will be.

The vehicle-mounted device 10 according to the present embodiment associates the authentication information for authenticating each driver with the reference information of the face direction or the line-of-sight direction for each driver for each driver. to register. Further, the on-board unit 10 authenticates the driver who is driving the vehicle 2, and uses the reference information of the face direction or the line-of-sight direction of the authenticated driver to perform inattentive and safety confirmation operation determination processing and the like. Perform a predetermined process. According to such an in-vehicle device 10, it is possible to reduce the influence of individual differences in the direction of each driver's face or the direction of the line of sight, and it is possible to execute highly accurate processing for each driver.

As shown in FIG. 1, the on-board unit 10 includes a driver camera 21, an image acquisition unit 22a, an authentication information acquisition unit 22b, an orientation detection unit 22c, a calculation unit 19aa, a registration unit 19ab, an authentication unit 19ac, and a first processing unit 19ad. , And a second processing unit 19ae. Further, the vehicle-mounted device 10 includes an image information storage unit 17a, a registration information storage unit 17b, and a processing information storage unit 17c.

The image acquisition unit 22a acquires an image captured by the driver camera 21.
The authentication information acquisition unit 22b acquires authentication information for authenticating the driver from the image acquired by the image acquisition unit 22a. For example, information necessary for face recognition such as data indicating feature points of each organ of the driver's face, or information necessary for iris recognition such as an iris pattern is acquired.

In the vehicle-mounted device 10 illustrated in FIG. 1, the authentication information acquisition unit 22b is configured to acquire the driver's authentication information from the image captured by the driver camera 21, but the configuration is not limited to this. As another form of the authentication information acquisition unit 22b, for example, a vein authentication sensor provided on the steering wheel of the vehicle 2, a fingerprint authentication sensor, a heartbeat authentication sensor provided on the driver's seat of the vehicle 2, and a driver's voice are used. A configuration or the like in which authentication information is acquired from at least one of the voiceprint authentication sensors can be adopted.

The orientation detection unit 22c detects data used for calculating a reference for the driver's face orientation or gaze direction, such as the driver's face orientation or gaze direction, from the image acquired by the image acquisition unit 22a. ..

In the image information storage unit 17a, the image data acquired by the image acquisition unit 22a, the authentication information acquired by the authentication information acquisition unit 22b, the direction of the driver's face or the direction of the line of sight detected by the orientation detection unit 22c, etc. Image information is stored in chronological order.

The calculation unit 19aa calculates the reference information of the direction of each driver's face or the direction of the line of sight by using the image information stored in the image information storage unit 17a. For example, the mode, median, or average value calculated by statistically processing the data of the driver's face direction or line-of-sight direction (yaw angle, pitch angle, etc.) detected from each image is used as a reference. It can be information. In addition, information indicating a deviation (deviation angle) from a preset reference of the face orientation or the line-of-sight direction (frontward reference direction) may be calculated.

Further, the calculation unit 19aa may perform a process of calculating the reference information of the driver for which the reference information of the direction of the driver's face or the direction of the line of sight is not registered in the registration information storage unit 17b, or the registration information storage. A process of updating the reference information of the driver in which the reference information of the direction of the driver's face or the direction of the line of sight is registered in the unit 17b may also be performed.

The registration unit 19ab associates the reference information of each driver's face direction or line-of-sight direction calculated by the calculation unit 19aa with each driver's authentication information for each driver, and is a registration information storage unit. Register in 17b. The registration information storage unit 17b stores the driver ID, the driver authentication information, and the reference information of the driver's face direction or line-of-sight direction in association with each other. The driver ID is a driver identification code, and is given to the authentication information when the authentication information not registered in the registration information storage unit 17b is acquired for the first time by the on-board unit 10.

The authentication unit 19ac authenticates the driver who is driving the vehicle 2 based on the information registered in the registration information storage unit 17b and the driver's authentication information acquired by the authentication information acquisition unit 22b.

When the driver is authenticated by the authentication unit 19ac, the first processing unit 19ad acquires the reference information of the face direction or the line-of-sight direction of the authenticated driver from the registration information storage unit 17b, and acquires the driving. A predetermined process is performed using the reference information of the direction of the person's face or the direction of the line of sight.

When the driver is not authenticated by the authentication unit 19ac, the second processing unit performs a predetermined process using the preset reference information of the face direction or the line-of-sight direction.
The predetermined process is, for example, a driver's inattentive determination process, a driver's safety confirmation operation determination process at an intersection, or the like, but is not limited thereto.

According to such an in-vehicle device 10, the registration unit 19ab registers the authentication information of each driver and the reference information of each driver's face direction or line-of-sight direction in association with each driver. It is registered in the information storage unit 17b. Then, when the driver is authenticated by the authentication unit 19ac, a predetermined process is performed using the reference information of the authenticated driver's face direction or line-of-sight direction acquired from the information registered in the registration information storage unit 17b. Is done.
Therefore, since the predetermined processing is performed using the reference information of each driver's face direction or line-of-sight direction, the influence of individual differences in each driver's face direction or line-of-sight direction is reduced. A predetermined process can be executed, and a predetermined process can be accurately executed for each driver.

[Configuration example]
FIG. 2 is a block diagram showing a main configuration of the vehicle-mounted device 10 according to the embodiment.
The on-board unit 10 includes a platform unit 11 and a driver monitoring unit 20. Further, the drive recorder unit 30 is connected to the vehicle-mounted device 10.

The platform unit 11 is equipped with an acceleration sensor 12 that detects the acceleration of the vehicle 2, an angular velocity sensor 13 that detects the rotational angular velocity of the vehicle 2, and a GPS (Global Positioning System) receiving unit 14 that detects the position of the vehicle 2. .. Further, the platform unit 11 includes a communication unit 15 that performs communication processing with an external device via a communication network, a notification unit 16 that outputs a predetermined sound or voice for issuing a warning, a storage unit 17, and an external interface ( External I / F) 18 is equipped. Further, the platform unit 11 is equipped with a control unit 19 that controls the processing operation of each unit.

The acceleration sensor 12 is composed of, for example, a three-axis acceleration sensor that detects acceleration in three directions of the XYZ axes. As the 3-axis accelerometer, a semiconductor type accelerometer such as a piezoresistive type can be used in addition to the capacitance type. As the acceleration sensor 12, a two-axis or one-axis acceleration sensor may be used, but at least one capable of detecting the acceleration in the front-rear direction generated in the vehicle 2 is used. The acceleration data detected by the acceleration sensor 12 is stored in the RAM 19b of the control unit 19 in association with the detection time.

The angular velocity sensor 13 is a sensor capable of detecting at least an angular velocity corresponding to rotation around the vertical axis (yaw direction), that is, angular velocity data corresponding to rotation (turning) of the vehicle 2 in the left-right direction, for example, a gyro sensor (yaw rate). It is also called a sensor).
The angular velocity sensor 13 includes a 1-axis gyro sensor around the vertical axis, a 2-axis gyro sensor that detects an angular velocity around the horizontal axis (pitch direction) in the left-right direction, and a horizontal axis around the front-back direction (roll direction). A 3-axis gyro sensor that also detects the angular velocity of the above may be used. As these gyro sensors, in addition to the vibration type gyro sensor, an optical type or a mechanical type gyro sensor can be used.

Further, regarding the detection direction of the angular velocity around the vertical axis of the angular velocity sensor 13, for example, clockwise may be set in the positive direction and counterclockwise may be set in the negative direction. In this case, if the vehicle 2 turns to the right, positive angular velocity data is detected, and if the vehicle 2 turns to the left, negative angular velocity data is detected. The angular velocity sensor 13 detects the angular velocity in a predetermined cycle (for example, a cycle of several tens of ms), and the detected angular velocity data is stored in the RAM 19b of the control unit 19 in association with the detection time, for example. For the acceleration sensor 12 and the angular velocity sensor 13, an inertial sensor in which these are mounted in one package may be used.

The GPS receiving unit 14 receives GPS signals from artificial satellites via the antenna 14a at a predetermined cycle, and detects the position information (latitude and longitude) of the current location. The position information detected by the GPS receiving unit 14 is stored in the RAM 19b of the control unit 19 in association with the detection time. The device for detecting the position of the vehicle 2 is not limited to the GPS receiving unit 14. For example, positioning devices compatible with other satellite positioning systems such as Japan's Quasi-Zenith Satellite, Russia's GLONASS, Europe's Galileo, and China's Compass may be used.
Further, using the position information detected by the GPS receiving unit 14, for example, it is possible to calculate the change in position per unit time and detect the vehicle speed.

The communication unit 15 includes a communication module that performs data output processing or the like to an external device via a communication network.

The storage unit 17 is composed of, for example, a removable storage device such as a memory card, or one or more storage devices such as a hard disk drive (HDD) and a solid state drive (SSD). The storage unit 17 includes an image information storage unit 17a illustrated in FIG. 1, a registration information storage unit 17b, and a processing information storage unit 17c. Further, the storage unit 17 may store, for example, data acquired from each unit of the vehicle-mounted device 10 or the drive recorder unit 30.

The external I / F 18 includes an interface circuit, a connector, and the like for exchanging data and signals with an in-vehicle device such as a drive recorder unit 30.

The control unit 19 is composed of a microcomputer including a CPU (Central Processing Unit) 19a, a RAM (Random Access Memory) 19b, and a ROM (Read Only Memory) 19c. The control unit 19 performs a process of storing various acquired data in the RAM 19b or the storage unit 17. Further, the control unit 19 reads out the program stored in the ROM 19c, the RAM 19b, or various data stored in the storage unit 17, and executes the program.

The CPU 19a of the control unit 19 executes the processing of the calculation unit 19aa, the registration unit 19ab, the authentication unit 19ac, the first processing unit 19ad, and the second processing unit 19ae illustrated in FIG. Further, an example of the "program" according to the present invention is stored in the ROM 19c.

The driver monitoring unit 20 includes a driver camera 21, an image analysis unit 22, and an interface (I / F) 23.
The driver camera 21 includes, for example, a lens unit (not shown), an image sensor unit, a light irradiation unit, a camera control unit that controls each of these units, and the like.

The image sensor unit includes, for example, an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), a filter, a microlens, and the like. The image sensor unit includes an infrared sensor such as a CCD, CMOS, or a photodiode capable of forming an image by receiving light in the visible region and forming an image by receiving infrared rays such as near infrared rays or ultraviolet rays. But it may be. The light irradiation unit includes a light emitting element such as an LED (Light Emitting Diode), and an infrared LED or the like may be used so that the state of the driver can be imaged day or night. The driver camera 21 may be a monocular camera or a stereo camera.

The camera control unit includes, for example, a processor and the like. The camera control unit controls the image pickup element unit and the light irradiation unit, irradiates light (for example, near infrared rays) from the light irradiation unit, and controls the image pickup element unit to image the reflected light. The driver camera 21 captures an image at a predetermined frame rate (for example, 30 to 60 frames per second) while the vehicle 2 is driving, and the data of the image captured by the driver camera 21 is output to the image analysis unit 22.

The image analysis unit 22 is configured to include, for example, an image processing processor, and information on at least one of the driver's face direction, line-of-sight direction, and eye opening degree from the image captured by the driver camera 21. Performs processing to detect (driver's condition). The image analysis unit 22 executes the processing as the image acquisition unit 22a, the authentication information acquisition unit 22b, and the orientation detection unit 22c illustrated in FIG.

Data indicating the driver's condition, image data, and imaging date / time data detected by the image analysis unit 22 are sent to the platform unit 11 via the interface (I / F) 23, and the storage unit 17 of the platform unit 11 It is stored in the image information storage unit 17a of the above.

The orientation of the driver's face detected by the image analysis unit 22 is, for example, a pitch angle which is an angle (vertical orientation) around the X-axis (left-right axis) of the driver's face, and a Y-axis of the face ( It may be indicated by the Yaw angle, which is the angle around the vertical axis (horizontal axis), and the Roll angle, which is the angle around the Z axis (front-back axis) of the face (left-right tilt), and is at least left and right. Includes yaw angle to indicate orientation. Further, these angles can be indicated by angles with respect to a predetermined reference direction. For example, the reference direction may be set in the front direction of the driver or in the optical axis direction of the driver camera 21. ..

Further, the direction of the driver's line of sight detected by the image analysis unit 22 is, for example, the direction of the driver's face detected from the image and information on the eye region (position of the inner corner of the eye, the outer corner of the eye, or the pupil). It is estimated from the relationship with and can be indicated by a line-of-sight vector V (three-dimensional vector) on three-dimensional coordinates. The line-of-sight vector V is, for example, a pitch angle which is an angle (up and down direction) around the X axis (left and right axis) of the driver's face, and a yaw which is an angle (left and right direction) around the Y axis (up and down axis) of the face. It may be estimated from at least one of the angle and the roll angle (left-right tilt) around the Z-axis (front-back axis) of the face and the information of the eye area. Further, the line-of-sight vector V shows a part of the values of the three-dimensional vector in common with the value of the vector of the face orientation (for example, the origin of the three-dimensional coordinates is common), or uses the vector of the face orientation as a reference. It may be indicated by the relative angle (relative value of the vector of the direction of the face).

The driver's eye opening detected by the image analysis unit 22 is, for example, information on the driver's eye region detected from the image (the inner corner of the eye, the outer corner of the eye, the positions of the upper and lower eyelids, the position of the pupil, and the like. ) Is estimated.

The drive recorder unit 30 is connected to the external I / F 18 of the platform unit 11. The drive recorder unit 30 includes an outside camera 31 and an inside camera 32.

The outside camera 31 is a camera that captures an image of the front of the vehicle 2, and the inside camera 32 is a camera that captures an image of the interior of the vehicle 2. The outside camera 31 and the inside camera 32 may be composed of, for example, a visible light camera, but may be composed of a near infrared camera or the like.

The out-of-vehicle camera 31 and the in-vehicle camera 32 each capture an image at a predetermined frame rate (for example, 30 to 60 frames per second), and the images captured by the out-of-vehicle camera 31 and the in-vehicle camera 32 and data such as the imaging date and time are stored in the platform unit. It is sent to 11 and stored in the RAM 19b or the storage unit 17 of the platform unit 11. The drive recorder unit 30 may be configured to include only the external camera 31.

The in-vehicle device 10 can have a compact configuration in which the platform unit 11 and the driver monitoring unit 20 are housed in one housing. In that case, the location of the vehicle-mounted device 10 installed in the vehicle is not particularly limited as long as the driver camera 21 can capture at least the field of view including the driver's face. For example, it may be installed near the center of the dashboard of the vehicle 2, near the steering wheel column, near the instrument panel, near the rearview mirror, or at the A-pillar portion.

In addition, information including the specifications of the driver camera 21 (for example, the angle of view, the number of pixels (vertical x horizontal), etc.) and the position / orientation (for example, the mounting angle, the distance from a predetermined origin (the center position of the handle, etc.), etc.) is the driver. It may be stored in the monitoring unit 20 or the platform unit 11. Further, the driver monitoring unit 20 may be configured separately from the platform unit 11 in addition to the form of being integrated with the platform unit 11. Further, the power supply for the vehicle-mounted device 10 is supplied from, for example, the accessory power supply for the vehicle 2.

FIG. 3 is a schematic configuration diagram showing an example of an operation evaluation system to which the vehicle-mounted device 10 according to the embodiment is applied.
The driving evaluation system 1 is a system for evaluating the safety confirmation operation of the driver 3 who is in the vehicle 2, and is an in-vehicle device 10 mounted on at least one or more vehicles 2 and each in-vehicle device 10. It is configured to include at least one or more server devices 40 that evaluate the operation of each driver 3 using the data acquired from.

The vehicle-mounted device 10 and the server device 40 are configured to be able to communicate with each other via the communication network 4. The communication network 4 may include a wireless communication network such as a mobile telephone network (3G / 4G) including a base station or a wireless LAN (Local Area Network), a wired communication network such as a public telephone network, the Internet, or a dedicated network. It may include a telecommunications line such as.

The vehicle-mounted device 10 acquires data used for driving evaluation of the driver 3 of the vehicle 2 and transmits it to the server device 40 at a predetermined timing. The data transmitted from the vehicle-mounted device 10 to the server device 40 includes at least one of the state of the driver 3, the position of the vehicle 2, and the behavior of the vehicle 2 detected by the vehicle-mounted device 10. There is. The data showing the behavior of the vehicle 2 includes the data of the acceleration or the angular velocity generated in the vehicle 2 detected by the vehicle-mounted device 10.

Further, the terminal device 80 of the business operator that manages the vehicle 2 (hereinafter, referred to as the business operator terminal) is configured to be able to communicate with the server device 40 via the communication network 4. The business terminal 80 may be a personal computer having a communication function, or a mobile information terminal such as a mobile phone, a smartphone, or a tablet device. Further, the operator terminal 80 may be configured to be able to communicate with the vehicle-mounted device 10 via the communication network 4.

The server device 40 includes a communication unit 41, a server computer 50, and a storage unit 60, and these are connected via a communication bus 42.
The communication unit 41 is composed of a communication device for realizing transmission / reception of various data and signals to and from the vehicle-mounted device 10 and the business terminal 80 via the communication network 4.

The server computer 50 includes a central processing unit 51 including one or more CPUs, and a main memory 52 in which the control program 53 is stored. The central processing unit 51 executes processing such as operation evaluation according to the control program 53 in the main memory 52.

The storage unit 60 is composed of one or more large-capacity storage devices such as a hard disk drive and a solid state drive, and stores information of the data storage unit 61, the evaluation data storage unit 62, and the driver 3. It is configured to include an information storage unit 63.
Further, the information registered in the registration information storage unit 17b of the vehicle-mounted device 10 is transmitted to the server device 40 at a predetermined timing. The driver information storage unit 63 stores the registration information of the registration information storage unit 17b acquired from each on-board unit 10.

In the driving evaluation system 1, at least one of the state of the driver 3, the position of the vehicle 2, and the behavior of the vehicle 2 transmitted by the server device 40 when a predetermined event is detected from each in-vehicle device 10. Data including any of them is stored in the data storage unit 61 of the storage unit 60. When a predetermined event is detected, for example, when the vehicle-mounted device 10 detects the passage of the vehicle 2 at an intersection, or when the vehicle-mounted device 10 detects the inattentiveness of the driver 3.

The server device 40 uses the data of each on-board unit 10 stored in the data storage unit 61 to evaluate the operation of each driver 3. The driving evaluation item includes an evaluation regarding a safety confirmation operation at an intersection, but may further include an evaluation regarding a driving operation action, an evaluation regarding a concentration state during driving, and the like.

The server device 40 may execute the driving evaluation process of each item after the one-day driving of each vehicle 2 is completed by using the data acquired from each vehicle-mounted device 10, or the server device 40 may execute the driving evaluation processing of each item at regular intervals. The evaluation process of each item may be executed within the fixed period, and the execution timing of the operation evaluation process is not particularly limited. Then, the server device 40 performs a process of storing the operation evaluation data of each driver 3 obtained in the operation evaluation process in the evaluation data storage unit 62 of the storage unit 60.

Further, the server device 40 is configured to be able to provide a cloud service. For example, the server device 40 may be composed of a server system including a Web server, an application server, and a database server. The Web server performs processing such as an access request from the browser of the business terminal 80. The application server accesses the database server, for example, in response to a request from the Web server, and performs processing such as searching for data necessary for processing and extracting data. The database server manages various data acquired from the in-vehicle device 10, for example, and performs processing such as searching, extracting, and saving the data in response to a request from the application server.

For example, the server device 40 processes various requests requested from the browser of the business operator terminal 80, for example, a login to a website, a request for sending a driving evaluation report, and the like, and creates a processing result, for example, through the browser. The data of the operation evaluation report is transmitted to the business operator terminal 80, and a process of displaying the operation evaluation report on the display screen of the business operator terminal 80 is executed.

The business operator confirms the driving evaluation report of each driver 3 displayed on the display screen of the business operator terminal 80, and gives guidance to each driver 3 to improve safety awareness. By providing such guidance, it is possible to raise the safety awareness of the driver 3 and reduce the risk of accidents.

[Operation example]
FIG. 4 is a flowchart showing a processing operation performed by the driver monitoring unit 20 in the vehicle-mounted device 10 according to the embodiment. This processing operation is executed, for example, at the timing when an image is captured by the driver camera 21 (for example, every frame or every frame at a predetermined interval).

First, the image analysis unit 22 acquires the image captured by the driver camera 21 (step S1).

Next, the image analysis unit 22 performs a process of detecting the driver's face (for example, a face region) from the acquired image (step S2). The method for detecting a face from an image is not particularly limited, but it is preferable to adopt a method for detecting a face at high speed and with high accuracy. For example, even if the difference in brightness (luminance difference) of a local region of the face, the edge intensity, the relationship between these local regions, etc. is detected as a feature amount, and the process of searching the facial region from the image is executed. good.

The image analysis unit 22 performs a process of detecting the position and shape of facial organs such as eyes, nose, mouth, and eyebrows from the facial region detected in step S2 (step S3). The method for detecting the facial organs from the facial region in the image is not particularly limited, but it is preferable to adopt a method capable of detecting the facial organs at high speed and with high accuracy. For example, a method may be adopted in which the image analysis unit 22 creates a three-dimensional face shape model, fits the three-dimensional face shape model to the face region on the two-dimensional image, and detects the position and shape of each organ of the face. According to this method, it is possible to accurately detect the position and shape of each organ of the face regardless of the installation position of the driver camera 21 and the orientation of the face in the image. As a technique for fitting a three-dimensional face shape model to a human face in an image, for example, the technique described in Japanese Patent Application Laid-Open No. 2007-249280 can be applied, but the technique is not limited thereto.

Next, the image analysis unit 22 detects the orientation of the driver's face based on the position and shape data of each organ of the face obtained in step S3 (step S4). For example, the pitch angle of vertical rotation (X-axis), the yaw angle of horizontal rotation (Y-axis), and the roll angle of total rotation (Z-axis) included in the parameters of the above three-dimensional face shape model. It is preferable to detect at least the yaw angle as information regarding the orientation of the driver's face.

Next, the image analysis unit 22 determines the orientation of the driver's face obtained in step S4 and the position and shape of the driver's facial organs obtained in step S3, particularly the characteristic points of the eyes (outer corners of eyes, inner corners of eyes, and pupils). The direction of the line of sight is detected based on the position and shape (step S5).

The direction of the line of sight is, for example, the features of the image of the eye in various face directions and the direction of the line of sight (for example, the relative positions of the outer corner of the eye, the inner corner of the eye, and the pupil, the sclera (so-called white eye) part and the iris (so-called black eye) part. Relative position, shading, texture, etc.) may be detected by learning in advance using a learning device and evaluating the degree of similarity with the learned feature amount data. Alternatively, using the fitting result of the above-mentioned three-dimensional face shape model, the size and center position of the eyeball are estimated from the size and orientation of the face and the position of the eyes, and the position of the pupil is detected to detect the center of the eyeball. The vector connecting the center of the pupil and the center of the pupil may be detected as the direction of the line of sight.

Next, the image analysis unit 22 detects the eye opening based on the position and shape of the driver's facial organs obtained in step S3, particularly the position and shape of the feature points of the eyes (outer corners of the eyes, inner corners of the eyes, pupils, eyelids). (Step S6).

Next, the image analysis unit 22 acquires face recognition information as driver authentication information (step S7). The face recognition information includes, for example, information regarding the position and shape of facial organs such as eyes, nose, mouth, and eyebrows detected in step S3. In step S7, instead of the face authentication information, information necessary for iris authentication such as an iris pattern may be acquired.

Next, the image analysis unit 22 determines the direction of the driver's face detected in step S4, the direction of the driver's line of sight detected in step S5, the eye opening of the driver detected in step S6, and step S7. The acquired face recognition information, the driver's image, and the imaging time are associated with each other and transmitted as image information to the platform unit 11 (step S7). After that, the image analysis unit 22 returns to step S1 and repeats the process.

FIG. 5 is a flowchart showing a processing operation performed by the control unit 19 in the vehicle-mounted device 10 according to the embodiment. This processing operation is executed, for example, at the timing of acquiring image information from the driver monitoring unit 20.

First, the control unit 19 acquires image information from the driver monitoring unit 20 (step S11). The image information is the information transmitted from the driver monitoring unit 20 in step S7 shown in FIG.

Next, the control unit 19 stores the image information acquired from the driver monitoring unit 20 in the image information storage unit 17a (step S12).

Next, the control unit 19 acquires face recognition information from the image information stored in the image information storage unit 17a (step S13).

Next, the control unit 19 performs a process of collating the face authentication information acquired in step S13 with the face authentication information of each driver registered in the registration information storage unit 17b (step S14).

Next, the control unit 19 determines whether or not the face authentication information acquired in step S13 is registered in the registration information storage unit 17b (step S15).

If the control unit 19 determines in step S15 that the face recognition information acquired in step S13 is registered in the registration information storage unit 17b, the control unit 19 then associates the face recognition information with the face recognition information from the registration information storage unit 17b. The process of reading the reference information of the driver's face direction or the line-of-sight direction stored in the above is performed (step S16).

The control unit 19 sets the reference of the face orientation or the line-of-sight direction used for the inattentive determination to the reference information of the driver's face orientation or the line-of-sight direction read out in step S16, and sets the image in step S12. The inattentive determination process is executed using the image information stored in the information storage unit 17a (step S17).
For example, the driver's face direction or line-of-sight direction (yaw angle or pitch angle) included in the image information and the driver's face direction or line-of-sight direction reference information (yaw angle or pitch angle). The angle difference with and from is calculated, and the presence or absence of inattentiveness is determined based on whether or not the calculated angle difference exceeds the threshold value of a predetermined inattentiveness determination angle (yaw angle or pitch angle).

Next, the control unit 19 determines whether or not inattentiveness is detected (step S18).
If the control unit 19 determines in step S18 that no inattentiveness has been detected, then the process ends, while if it determines in step S18 that inattentiveness has been detected, then the inattentiveness is detected by the notification unit 16. A process for notifying the fact is performed (step S19).

After performing the notification processing in step S19, the control unit 19 stores the inattentive detection data in the processing information storage unit 17c (step S20). The inattentive detection data includes, for example, information such as an image in which the inattentiveness is detected, the date and time of imaging thereof, the position of the vehicle at that time, and the driver ID.

Next, the control unit 19 transmits the inattentive detection data stored in the processing information storage unit 17c to the server device 40 (step S21).

FIG. 6 is a diagram showing an example of the structure of data transmitted from the vehicle-mounted device 10 to the server device 40.
The data transmitted from the in-vehicle device 10 to the server device 40 includes the identification information (serial number, etc.) of the in-vehicle device 10, the identification code of the inattentiveness, the detection date and time of the inattentiveness, and the orientation of the driver's face (pitch, yaw, and roll). ), Line-of-sight direction (pitch and yaw), eye opening (right eye and left eye), vehicle acceleration (at least front and rear), angular velocity (yaw), driver ID, driver image, outside image, and vehicle position Information (latitude and longitude), running speed, etc. are included. The structure of the data transmitted to the server device 40 is not limited to the structure shown in FIG. 6, but at least the identification information (serial number, etc.) of the on-board unit 10, the inattentive identification code, and the inattentive detection are detected. It is preferable that the date and time, the driver ID, and the driver image are included.

Further, when the vehicle 2 passes through the intersection, the same data is transmitted to the server device 40. In that case, the inattentive identification code and the inattentive detection date and time shown in FIG. 6 become the intersection passage identification code and the intersection passage date and time.

On the other hand, in step S15, if the control unit 19 determines that the face recognition information acquired in step S13 is not registered in the registration information storage unit 17b, the control unit 19 then operates the face recognition information acquired in step S13. A process of assigning an ID (identification code) for identifying a person and registering the person in the registration information storage unit 17b is performed (step S22).

Next, the control unit 19 sets the reference of the face orientation or the line-of-sight direction used for the inattentive determination to the initial value of the face orientation or the line-of-sight direction preset at the time of mounting the vehicle-mounted device 10, and steps S12. In the image information storage unit 17a, the inattentive determination process is executed using the image information stored in the image information storage unit 17a (step S23).
The inattentive determination process includes, for example, the driver's face orientation or line-of-sight direction (yaw angle or pitch angle) included in the image information, and the initial value of the face orientation or line-of-sight direction (yaw angle or pitch). The angle difference from the angle) is calculated, and the presence or absence of inattentiveness is determined based on whether or not the calculated angle difference exceeds the threshold value of the predetermined inattentiveness determination angle (yaw angle or pitch angle).

After that, the control unit 19 executes the processes of steps S18 to S21 described above to finish the process.

FIG. 7 is a flowchart showing a processing operation performed by the control unit 19 in the vehicle-mounted device 10 according to the embodiment. This processing operation is executed, for example, at a predetermined timing for calculating the reference of the face direction or the line-of-sight direction for each driver.

First, the control unit 19 uses the image information stored in the image information storage unit 17a to determine whether or not the predetermined calculation timing for calculating the reference of the face direction or the line-of-sight direction for each driver has been reached ( Step S31).

Whether or not the predetermined calculation timing has been reached depends on whether or not a predetermined period has passed since the power of the on-board unit 10 was turned on, whether or not the ignition switch (starting switch) off signal was acquired from the vehicle 2, or whether or not the ignition switch (starting switch) was turned off. It may be determined whether or not an image having a predetermined number of frames or a predetermined capacity is stored in the image information storage unit 17a.

Alternatively, whether or not the predetermined calculation timing has been reached may be determined by whether or not the specific traveling state of the vehicle 2 has continued for a predetermined period. The specific traveling state of the vehicle 2 is, for example, a state in which the vehicle speed is within a certain range (a state of constant speed traveling), or a state in which the angular velocity detected by the vehicle 2 is within a range indicating a straight-ahead state (a state of straight-ahead traveling). It may be.

If the control unit 19 determines that the predetermined calculation timing has been reached in step S31, the control unit 19 then obtains the driver ID in which the reference information of the driver's face direction or the line-of-sight direction is not registered from the registration information storage unit 17b. The reading process is performed (step S32).
The driver ID, the driver's authentication information, and the reference information of the driver's face direction or the line-of-sight direction are registered in the registration information storage unit 17b in association with each other. At the stage where the reference information of the line-of-sight direction is not calculated (unregistered), the driver ID and the driver's authentication information are registered in association with each other. In step S32, a process is performed in which the driver ID in which the reference information of the driver's face direction or the line-of-sight direction is not registered is read out.

Next, the control unit 19 determines whether or not there is a driver ID in which the reference information of the driver's face direction or line-of-sight direction is not registered (step S33), and determines whether or not there is a driver ID in the driver's face direction or line-of-sight direction. If it is determined that there is a driver ID whose reference information is not registered, the process proceeds to step S34.

In step S34, the control unit 19 performs a process of acquiring face recognition information from the image information stored in the image information storage unit 17a.

Next, the control unit 19 performs a process of collating the face recognition information associated with the driver ID read from the registration information storage unit 17b with the face recognition information acquired from the image information storage unit 17a (step S35). ).

Next, the control unit 19 determines whether or not the verification is OK, that is, whether or not the face authentication information matches (step S36), and if it is determined that the verification is OK, the process proceeds to step S37.

In step S37, the control unit 19 reads from the image information storage unit 17a the image information associated with the face recognition information determined to match in step S36, and from these image information, the orientation of the driver's face. Alternatively, a process of calculating reference information in the direction of the line of sight is performed.

The image information includes information (yaw angle, pitch angle, etc.) such as the direction of the driver's face or the direction of the line of sight detected from a plurality of images. The control unit 19 calculates, for example, the mode, average, or median of the driver's face direction or line-of-sight direction (yaw angle, pitch angle, etc.) detected from a plurality of images, and these values. Either of the above may be used as a reference value for the direction of the driver's face or the direction of the line of sight.

The reference information of the driver's face direction or the line-of-sight direction is the angle information indicating the driver's face direction or the line-of-sight direction, for example, the angle information of at least one of the yaw angle, the pitch angle, and the roll angle. It may be the mode, the median, or the average, but the mode is preferred. In addition, the reference information of the direction of the driver's face or the direction of the line of sight may include the mode, median, or average value for the position of each organ of the driver's face.

Next, the control unit 19 performs a process of registering the reference information of the driver's face direction or the line-of-sight direction calculated in step S37 in the registration information storage unit 17b (step S38). In the registration information storage unit 17b, the driver ID, the driver authentication information, and the reference information of the driver's face direction or line-of-sight direction calculated in step S37 are registered in association with each other.

Next, the control unit 19 uses the information registered in the registration information storage unit 17b, that is, the driver ID, the driver authentication information, and the reference of the driver's face direction or line-of-sight direction calculated in step S37. The registration information including the information is transmitted to the server device 40 (step S39). The registration information may be transmitted to the server device 40 at another timing. The server device 40 receives the registration information and stores it in the driver information storage unit 63. The driver information storage unit 63 stores the registration information received from each on-board unit 10.

After that, the control unit 19 returns to the process of step S33, determines whether or not there is a driver ID in which the reference information of the driver's face direction or the line-of-sight direction is not registered, and determines whether or not there is a driver ID, and the driver's face direction or If it is determined that there is an unregistered driver ID in the reference information of the line-of-sight direction, the processes of steps S34 to 39 are repeated. That is, the process and registration of registering the reference information of the face direction or the line-of-sight direction for each driver ID (in other words, for each driver) in which the reference information of the driver's face direction or the line-of-sight direction is not registered. The process of transmitting the information to the server device 40 is repeated.

In step S33, if the control unit 19 determines that there is no unregistered driver ID in the reference information of the driver's face direction or the line-of-sight direction, the control unit 19 ends the process thereafter.

FIG. 8 is a flowchart showing an example of a processing operation performed by the central processing unit 51 of the server device 40 according to the embodiment.
The central processing unit 51 determines whether or not the data transmitted from the vehicle-mounted device 10 has been received (step S41).

If the central processing unit 51 determines that the data has been received from the vehicle-mounted device 10, then the data received from the vehicle-mounted device 10 is stored in the data storage unit 61 in association with the identification information of the vehicle-mounted device 10 (step). S42).

After that, the central processing unit 51 determines whether or not it is the timing to execute the evaluation of the driver's safe driving behavior (step S43). Whether or not it is the timing to execute the evaluation of the driver's safe driving behavior may be determined by, for example, whether or not a signal indicating the end of driving of the vehicle 2 is received from the vehicle-mounted device 10.

If the central processing unit 51 determines that it is not the timing to execute the evaluation of the driver's safe driving behavior, the central processing unit 51 returns to step S41. On the other hand, if the central processing unit 51 determines that it is time to evaluate the driver's safe driving behavior, the central processing unit 51 performs the evaluation processing of the safety confirmation operation at the intersection of the driver 3 of the vehicle 2 on which the on-board unit 10 is mounted. Start (step S44).

Next, the central processing unit 51 acquires data from the data storage unit 61 when the vehicle-mounted device 10 passes through the intersection (step S45). The data at the time of passing through the intersection is configured in the same manner as the data structure shown in FIG. 6, and the data of the period of passing through the intersection (about several tens of seconds before and after passing through the intersection) is acquired.

Next, the central processing unit 51 is included in the data at the time of passing the intersection acquired in step S45 in the reference information of the face direction or the line-of-sight direction of each driver registered in the driver information storage unit 63. It is determined whether or not there is reference information associated with the same ID as the driver ID (step S46).

If the central processing unit 51 determines in step S46 that there is reference information associated with the same ID as the driver ID included in the data at the time of passing through the intersection, the central processing unit 51 executes the process of step S47. In step S47, the central processing unit 51 sets the front-facing reference for detecting the driver's face swing angle (safety confirmation operation) at the intersection as the driver's face orientation or the driver's face orientation associated with the driver ID. Performs the process of setting the reference of the direction of the line of sight.

Next, the central processing unit 51 estimates the time when the vehicle 2 enters the intersection (step S48). For example, the central processing device 51 calculates the integrated value of the angular velocity data included in the data at the time of passing through the intersection, and estimates the time when the calculated integrated value reaches a predetermined integration ratio as the approach time to the intersection.

Next, the central processing unit 51 estimates the direction in which the vehicle 2 turns at the intersection (step S49). For example, the central processing unit 51 estimates the direction in which the vehicle 2 turns at an intersection based on the positive and negative of the angular velocity data. For example, a positive value is estimated to be a right turn, and a negative value is estimated to be a left turn.

Next, the central processing device 51 uses the reference of the driver's face direction or line-of-sight direction associated with the driver ID to perform a safety confirmation operation (right or left direction confirmation timing, face) before entering the intersection. (Swing angle, swing time, etc.) are compared with predetermined evaluation conditions for evaluation (step S50).

Next, the central processing unit 51 uses the reference of the driver's face direction or the line-of-sight direction associated with the driver ID to perform a safety confirmation operation after entering the intersection (confirmation timing in the right or left direction, face). (Swing angle, swing time, etc.) are compared with predetermined evaluation conditions for evaluation (step S51).

Next, the central processing unit 51 evaluates whether the traveling speed of the vehicle 2 at a predetermined time before and after the approach time to the intersection is appropriate. For example, it is evaluated whether or not the traveling speed of the vehicle 2 is equal to or less than a predetermined speed at which safe traveling is possible in the intersection (step S52).

Next, the central processing unit 51 aggregates the points evaluated or ranked scores in steps S50, S51, and S52, and calculates the operation evaluation points at the intersection (step S53).

Next, the central processing device 51 associates the operation evaluation points calculated in step S53 with the identification information of the vehicle-mounted device 10 or the information of the driver 3 (driver ID) and causes the evaluation data storage unit 62 of the storage unit 60 to associate the operation evaluation points. Store (step S54).

Next, the central processing unit 51 determines whether or not the evaluation at all the passing intersections has been completed (step S55), and if it determines that the evaluation has been completed, it determines that the subsequent processing is completed, but the evaluation has not been completed. Then, the process returns to step S48 and the process is repeated.

On the other hand, in step S46, if the central processing unit 51 determines that there is no reference information associated with the same ID as the driver ID included in the data at the time of passing through the intersection, the process of step S56 is executed. .. In step S56, the central processing unit 51 sets the frontal reference for detecting the driver's face swing angle (safety confirmation operation) at the intersection to a preset initial value of the face direction or the line-of-sight direction. Then, the processes of steps S48 to S55 are executed. In this case, in steps S50 and S51, the safety confirmation operation before and after entering the intersection is evaluated using the preset initial values of the face direction or the line-of-sight direction.

[Action / Effect]
According to the vehicle-mounted device 10 according to the above embodiment, the authentication information of each driver 3 and the reference information of each face direction or line-of-sight direction of the driver 3 are registered in association with each driver. The driver's inattentiveness determination process is performed using the reference information of the direction of each face or the direction of the line of sight of the driver 3 registered in the information storage unit 17b and registered in the registered information storage unit 17b.
Therefore, it is possible to execute the inattentive determination process in consideration of the individual difference in the direction of the face or the direction of the line of sight of the driver 3, in other words, the direction of the face or the direction of the line of sight, and the inattentiveness determination process can be executed for each driver 3. The determination process can be executed with high accuracy.

Further, according to the in-vehicle device 10, when the driver 3 is not authenticated by the authentication unit 19ac, the inattentive determination process is performed using the preset reference information of the face direction or the line-of-sight direction. Even if the driver 3 is not authenticated by 19ac, the inattentive determination process can be appropriately executed.

Further, according to the vehicle-mounted device 10, the calculation unit 19aa can efficiently calculate the reference information of the driver's face direction or the line-of-sight direction from the image of the driver 3 captured by the driver camera 21.

Further, according to the vehicle-mounted device 10, the reference information of the driver 3's face direction or line-of-sight direction is statistically processed using a plurality of images of the driver 3 captured by the driver camera 21 (mode). Since the mode, median, average, etc.) are included, the accuracy of the reference information of the direction of each face or the direction of the line of sight of the driver 3 can be improved.

Further, according to the in-vehicle device 10, since the driver camera 21 can also be used for acquiring the authentication information, it is not necessary to add equipment for acquiring the driver's authentication information, and the device cost can be suppressed. can.

Further, according to the operation evaluation system 1, the server device 40 that performs the operation evaluation acquires the information registered in the registration information storage unit 17b of the in-vehicle device 10 and stores it in the driver information storage unit 63. In the server device 40, the registration information stored in the driver information storage unit 63, that is, the reference information of the direction of each face or the direction of the line of sight of the driver 3 is used to determine the driver's safety confirmation operation at the intersection. Processing and evaluation processing are performed. Therefore, it is possible to execute the determination process and the evaluation process of the safety confirmation operation in consideration of the individual difference in the direction of each face or the direction of the line of sight of the driver 3, in other words, the direction of the face or the direction of the line of sight. It is possible to accurately execute the judgment process and the evaluation process of the safety confirmation operation for each driver 3.

[Modification example]
Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in all respects. Needless to say, various improvements and changes can be made without departing from the scope of the present invention.

FIG. 9 is an example of a functional configuration block diagram showing a feature portion of the operation evaluation system 1A according to another embodiment.
In the embodiment illustrated in FIG. 1, the vehicle-mounted device 10 is equipped with a calculation unit 19aa, a registration unit 19ab, and a registration information storage unit 17b, and the vehicle-mounted device 10 is equipped with the direction of each face or the direction of the line of sight of the driver. It is configured to calculate the reference information of the above and register the calculated reference information in the registration information storage unit 17b.

In another embodiment, as shown in FIG. 9, the server computer 50A of the server device 40A is equipped with the calculation unit 51a and the registration unit 51b, and the storage unit 60A is equipped with the registration information storage unit 64. May be good.

According to this embodiment, the server device 40A acquires image information from each in-vehicle device 10A, calculates reference information for each driver's face direction or line-of-sight direction, and registers the calculated reference information. The process of registering in the storage unit 64 is executed.

Further, the server device 40A executes a process of transmitting the information registered in the registration information storage unit 64 to each in-vehicle device 10A. The vehicle-mounted device 10A uses the registration information acquired from the server device 40A to execute the driver authentication process in the authentication unit 19ac.

According to the operation evaluation system 1A, the vehicle-mounted device 10A includes an authentication information acquisition unit 22b, an authentication unit 19ac, and a first processing unit 19ad, and the server device 40A includes a calculation unit 51a and a registration unit 51b. It has.

Therefore, the same effect as that of the above-mentioned operation evaluation system 1 can be obtained, and by equipping the server device 40A with the calculation unit 51a and the registration unit 51b, the processing load on the in-vehicle device 10A can be reduced. In addition, the equipment cost can be suppressed. Further, the server device 40A can centrally manage the authentication information of each driver and the reference information of the direction of the face or the direction of the line of sight. Even when managing it, it can be operated efficiently and appropriately.

[Additional Notes]
Embodiments of the present invention may also be described as, but are not limited to, the following appendices.
(Appendix 1)
The acquisition unit (22b) that acquires the authentication information of the driver who drives the vehicle (2), and
A calculation unit (19aa) that calculates reference information for the driver's face direction or line-of-sight direction, and
The authentication information of each of the drivers acquired by the acquisition unit (22b) and the reference information of the face orientation or the line-of-sight direction of each of the drivers calculated by the calculation unit (19aa) are described. A registration unit (19ab) that is associated and registered for each driver,
An authentication unit (19ac) that authenticates the driver based on the information registered by the registration unit (19ab) and the authentication information of the driver acquired by the acquisition unit (22b).
When the driver is authenticated by the authentication unit (19ac), the reference information of the face direction or the line-of-sight direction of the authenticated driver is acquired from the information registered by the registration unit (19ab) and acquired. The vehicle-mounted device (10) is provided with a first processing unit (19ad) that performs a predetermined process using the reference information of the driver's face direction or the line-of-sight direction.

(Appendix 2)
It is a driving evaluation system (1A) that evaluates the driving of the driver of the vehicle (2) based on the information acquired from the on-board unit (10A) mounted on the vehicle (2).
The acquisition unit (22b) that acquires the authentication information of the driver who drives the vehicle (2), and
The calculation unit (51a) that calculates the reference information of the driver's face direction or the line-of-sight direction, and
The authentication information of each of the drivers acquired by the acquisition unit (22b) and the reference information of the face direction or the line-of-sight direction of each of the drivers calculated by the calculation unit (51a) are described. A registration unit (51b) that is associated and registered for each driver, and
An authentication unit (19ac) that authenticates the driver based on the information registered by the registration unit (51b) and the authentication information of the driver acquired by the acquisition unit (22b).
When the driver is authenticated by the authentication unit (19ac), the reference information of the face orientation or the line-of-sight direction of the authenticated driver is acquired from the information registered by the registration unit (51b) and acquired. The driving evaluation system (1A) is characterized by including a processing unit (19ad) that performs a predetermined process using the reference information of the driver's face direction or the line-of-sight direction.

(Appendix 3)
Acquisition step (S7) to acquire the authentication information of the driver who drives the vehicle (2), and
The calculation step (S37) for calculating the reference information of the driver's face direction or the line-of-sight direction, and
The authentication information of each of the drivers acquired in the acquisition step (S7) and the reference information of the face orientation or the line-of-sight direction of each of the drivers calculated in the calculation step (S37) are described. A registration step (S38) in which each driver is associated and registered in the storage unit (17b), and
An authentication step for authenticating the driver based on the information registered in the storage unit (17b) in the registration step (S38) and the driver authentication information acquired in the acquisition step (S7). (S14) and
When the driver is authenticated by the authentication step (S14), the reference information of the face direction or the line-of-sight direction of the authenticated driver is acquired from the information registered in the storage unit (17b) and acquired. An information processing method including a processing step (S17) of performing a predetermined process using the reference information of the driver's face direction or the line-of-sight direction.

(Appendix 4)
On at least one computer
The acquisition step (S7) of acquiring the authentication information of the driver who drives the vehicle (2), and
The calculation step (S37) for calculating the reference information of the driver's face direction or the line-of-sight direction, and
The authentication information of each of the drivers acquired in the acquisition step (S7) and the reference information of the face orientation or the line-of-sight direction of each of the drivers calculated in the calculation step (S37) are described. A registration step (S38) in which each driver is associated and registered in the storage unit (17b), and
An authentication step (1) in which the driver is authenticated based on the information registered in the storage unit (17b) in the registration step (S38) and the driver authentication information acquired in the acquisition step (S7). S14) and
When the driver is authenticated by the authentication step (S14), the reference information of the face direction or the line-of-sight direction of the authenticated driver is acquired from the information registered in the storage unit (17b) and acquired. A program characterized by executing a processing step (S17) of performing a predetermined process using the reference information of the driver's face direction or the line-of-sight direction.

1, 1A Driving evaluation system 2 Vehicle 3 Driver 4 Communication network 10 On-board unit 11 Platform unit 12 Accelerometer 13 Angle speed sensor 14 GPS receiver 14a Antenna 15 Communication unit 16 Notification unit 17 Storage unit 17a Image information storage unit 17b Registration information storage Unit 17c Processing information storage unit 18 External interface (external I / F)
19 Control unit 19a CPU
19aa Calculation unit 19ab Registration unit 19ac Authentication unit 19ad First processing unit 19ae Second processing unit 19b RAM
19c ROM
20 Driver monitoring unit 21 Driver camera (imaging unit)
22 Image analysis unit 22a Image acquisition unit 22b Authentication information acquisition unit (acquisition unit)
22c Orientation detector 23 interface (I / F)
30 Drive recorder unit 31 External camera 32 In-vehicle camera 40, 40A Server device 41 Communication unit 50, 50A Server computer 51 Central processing device 51a Calculation unit 51b Registration unit 52 Main memory 53 Control program 60, 60A Storage unit 61 Data storage unit 62 Evaluation Data storage unit 63 Driver information storage unit 64 Registration information storage unit 80 Business terminal

Claims (10)

On-board equipment installed in at least one vehicle and
It is a driving evaluation system including at least one or more server devices that evaluate the driving of each driver using the data acquired from these in-vehicle devices.
The in-vehicle device
An acquiring unit that acquires authentication information of a driver who drives the vehicle,
An authentication unit that authenticates the driver based on the registration information of each driver acquired from the server device and the authentication information of the driver acquired by the acquisition unit.
When the driver is authenticated by the authentication unit, reference information of the authenticated driver's face orientation or line-of-sight direction is acquired from the server device, and the acquired driver's face orientation or line-of-sight direction is obtained. It is provided with a first processing unit that performs predetermined processing using the reference information of the direction.
It is configured to transmit information including the driver's authentication information authenticated by the authentication unit and time-series information of the driver's face direction or line-of-sight direction to the server device.
The server device
A calculation unit that calculates reference information for each driver's face orientation or line-of-sight direction using time-series information of each driver's face orientation or line-of-sight direction acquired from each on-board unit.
The authentication information of each driver acquired from each of the in-vehicle devices and the reference information of the face direction or the line-of-sight direction of each driver calculated by the calculation unit are registered in association with each driver. With a registration department
A driving evaluation system characterized in that the registration information of each driver registered by the registration unit is transmitted to each in-vehicle device . The in-vehicle device
When the driver is not authenticated by the authentication unit, the driver is provided with a second processing unit that performs a predetermined process using the preset reference information of the face direction or the line-of-sight direction. The operation evaluation system according to claim 1. The on-board unit includes an imaging unit that images the driver.
Said the calculating unit of the server device, and calculates the reference information of the orientation or line of sight direction of the face of each driver from said respective driver of the image captured by the imaging unit of the vehicle-mounted device The operation evaluation system according to claim 1 or 2, wherein the operation evaluation system is characterized . Statistics of the driver's face orientation or line-of-sight direction obtained by statistically processing the reference information of the driver's face orientation or line-of-sight direction using a plurality of images of the driver captured by the imaging unit. The operation evaluation system according to claim 3, wherein the value is included. The operation evaluation system according to claim 3 or 4, wherein the acquisition unit of the vehicle-mounted device acquires the authentication information from the image of the driver captured by the image pickup unit. Claims 1 to 5, wherein the reference information of the driver's face orientation or the line-of-sight direction includes information indicating a deviation from a preset face orientation or line-of-sight direction reference. The operation evaluation system described in any section. The driving evaluation system according to any one of claims 1 to 6, wherein the predetermined processing is the driver's inattentive determination processing. The operation evaluation system according to any one of claims 1 to 6, wherein the predetermined process is a process for determining a driver's safety confirmation operation at an intersection. On-board equipment installed in at least one vehicle and
It is an information processing method performed by a driving evaluation system including at least one or more server devices that evaluates the driving of each driver using the data acquired from these in-vehicle devices.
The in-vehicle device
The acquisition step of acquiring the authentication information of the driver who drives the vehicle, and
An authentication step for authenticating the driver based on the registration information of each driver acquired from the server device and the authentication information of the driver acquired in the acquisition step.
When the driver is authenticated by the authentication step, reference information of the authenticated driver's face orientation or line-of-sight direction is acquired from the server device, and the acquired driver's face orientation or line-of-sight direction is obtained. a processing step of performing a predetermined process using the reference information of direction,
A process including a step of transmitting information including the driver's authentication information authenticated in the authentication step and time-series information of the driver's face direction or line-of-sight direction to the server device is performed.
The server device
Using the time-series information of each driver's face orientation or line-of-sight direction acquired from each in-vehicle device, a calculation step of calculating the reference information of each driver's face orientation or line-of-sight direction, and a calculation step.
The authentication information of each driver acquired from each of the in-vehicle devices and the reference information of the face direction or the line-of-sight direction of each driver calculated by the calculation step are stored in association with each driver. Registration steps to register with the department and
An information processing method comprising performing a process including a step of transmitting the registration information of each driver registered in the registration step to each in-vehicle device. On-board equipment installed in at least one vehicle and
It is a program executed by a driving evaluation system including at least one or more server devices that evaluates the driving of each driver using the data acquired from these in-vehicle devices.
In the in-vehicle device
The acquisition step of acquiring the authentication information of the driver who drives the vehicle, and
An authentication step for authenticating the driver based on the registration information of each driver acquired from the server device and the authentication information of the driver acquired in the acquisition step.
When the driver is authenticated by the authentication step, reference information of the authenticated driver's face orientation or line-of-sight direction is acquired from the server device, and the acquired driver's face orientation or line-of-sight direction is obtained. a processing step of performing a predetermined process using the reference information of direction,
A step of transmitting information including the driver's authentication information authenticated in the authentication step and time-series information of the driver's face direction or line-of-sight direction to the server device is executed.
To the server device
Using the time-series information of each driver's face orientation or line-of-sight direction acquired from each in-vehicle device, a calculation step for calculating reference information of each driver's face orientation or line-of-sight direction, and a calculation step.
The authentication information of each driver acquired from each of the in-vehicle devices and the reference information of the face direction or the line-of-sight direction of each driver calculated by the calculation step are stored in association with each driver. Registration steps to register with the department and
A program characterized by executing a step of transmitting the registration information of each driver registered in the registration step to each in-vehicle device.

Images