JP2021131677A - Normalization timing control device and normalization timing control method - Google Patents

Abstract

To improve accuracy in normalization of data which indicates a state of a face of a driver.SOLUTION: A normalization timing control device 1 includes: an acquisition section 121 for successively acquiring a distance from an imaging device for imaging a face of a driver to the face of the driver from a detection device 2 for detecting data which indicates a state of a face of a driver of a vehicle; a calculation section 122 for calculating the distribution or standard deviation of a plurality of distances which are acquired by the acquisition section 121 in a predetermined period while the vehicle is traveling; and an instruction output section 123 for outputting an instruction to start normalization of data which indicates a state of a face of a driver to the detection device when the distribution or standard deviation calculated by the calculation section 122 is equal to or less than a determination threshold.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for normalizing data indicating the state of a driver's face.

Patent Document 1 discloses a technique for normalizing the degree of eye opening according to the eye shape of the driver detected from the captured image including the face of the driver of the vehicle. In this case, the degree of eye opening is normalized so that the degree of eye opening when the eyes are open is 100% and the degree of eye opening when the eyes are closed is 0%.

Japanese Unexamined Patent Publication No. 2019-74791

By the way, the driver who is driving the vehicle checks the meter of the vehicle and the surroundings according to the situation, so that the direction and posture of the face change. When the direction or posture of the driver's face changes, the shape of the eyes on the captured image may change, so there is a problem that the degree of eye opening cannot be normalized accurately.

Therefore, the present invention has been made in view of these points, and an object of the present invention is to improve the accuracy of normalization of data indicating the state of the driver's face.

In the first aspect of the present invention, an acquisition unit that sequentially acquires the distance from the image pickup device that images the driver's face to the driver's face from the detection device that detects data indicating the state of the driver's face of the vehicle. The acquisition unit calculates the variance or standard deviation of the plurality of distances acquired within a predetermined period while the vehicle is running, and the variance or standard deviation calculated by the calculation unit is the determination threshold value. In the following cases, the present invention provides a normalization timing control device including an instruction output unit that outputs an instruction to start normalization of data indicating the face state of the driver to the detection device.

For example, the instruction output unit outputs an instruction to start normalization of the degree of eye opening of the driver to the detection device when the variance or the standard deviation is equal to or less than the determination threshold value.

For example, the calculation unit calculates the variance or the standard deviation when the plurality of distances within the predetermined period are all within the range of values that can be taken when the driver takes a predetermined driving posture. ..

For example, the instruction output unit outputs an instruction to start the normalization to the detection device when the state in which the variance or the standard deviation is equal to or less than the determination threshold value continues for a predetermined time.

In the second aspect of the present invention, the computer processor determines the distance from the detection device that detects data indicating the state of the driver's face of the vehicle to the image pickup device that images the driver's face to the driver's face. A step of sequentially acquiring, a step of calculating the variance or standard deviation of a plurality of the distances acquired within a predetermined period, and a step of determining whether or not the calculated variance or the standard deviation is equal to or less than the determination threshold. When it is determined that the variance or the standard deviation is equal to or less than the determination threshold value, a step of outputting an instruction to start normalization of data indicating the state of the face of the driver to the detection device is executed. A timing control method is provided.

According to the present invention, there is an effect that the accuracy of normalization of data indicating the state of the driver's face can be improved.

It is a figure which shows the structure of the normalization timing control system which concerns on embodiment. It is a figure for demonstrating the method of calculating the standard deviation of a plurality of distances. It is a figure for demonstrating that the instruction to start the normalization is output when the state where the standard deviation is equal to or less than a determination threshold value continues for a deemed time. It is a histogram of the degree of eye opening after normalization detected by the detection device. It is a flowchart which shows an example of the process which controls the normalization timing.

[Configuration of normalization timing control system according to the embodiment]
FIG. 1 is a diagram showing a configuration of a normalized timing control system S according to an embodiment. The normalized timing control system S is mounted on a vehicle and has a normalized timing control device 1, a detection device 2, and an alarm device 3.

The detection device 2 is a device that detects data indicating the state of the face of the driver of the vehicle. For example, the detection device 2 includes an imaging device that captures the driver's face, and detects data indicating the state of the driver's face in the vehicle by analyzing the captured image including the driver's face. For example, the detection device 2 can detect the position of the driver's face, the orientation of the face, and the degree of eye opening as the state of the driver's face by analyzing the captured image, and data indicating the detected face state. Is transmitted to the alarm device 3.

Further, the detection device 2 detects the distance from the image pickup device that captures the driver's face to the driver's face by analyzing the captured image including the driver's face. Specifically, the detection device 2 detects the distance from the position where the image pickup device is installed to the position between the eyebrows of the driver's face at predetermined intervals, and the detected distance is the normalized timing control device 1 and the alarm device 3. Send to. The predetermined interval is, for example, 1/30 second.

The detection device 2 normalizes the data indicating the detected facial condition. For example, the detection device 2 normalizes the degree of eye opening of the driver. Specifically, the detection device 2 detects the distance between the eyelids from the lower eyelid to the upper eyelid of the driver a plurality of times, and sets the mode of the plurality of eyelid distances as the degree of eye opening (100) with the eyes open. Set to. Further, the detection device 2 sets a distance larger than a predetermined value or more than the eyelid distance corresponding to the eye opening degree (100) in the eye open state to the eye opening degree (120) in the eye open state. The distance between the eyelids corresponding to the degree of eye opening (0) with the eyes closed is zero. The detection device 2 is not limited to this, and the degree of eye opening may be normalized by using another known method. The detection device 2 starts normalization of data indicating the state of the face under the control of the normalization timing control device 1. Then, the detection device 2 transmits data indicating the state of the face detected after normalization to the alarm device 3.

The alarm device 3 includes, for example, a speaker or a buzzer, and outputs an alarm sound based on the state of the driver's face detected by the detection device 2. Specifically, the alarm device 3 outputs an alarm when the degree of eye opening detected by the detection device 2 as the state of the driver's face continues to be equal to or less than the alarm threshold value for a certain period of time (for example, 3 seconds). .. Further, the alarm device 3 may output an alarm when the position of the face detected by the detection device 2 continues in the alarm area for a certain period of time (for example, 3 seconds) or more. Further, the alarm device 3 may output an alarm when the state in which the angle indicating the direction of the face is within the alarm angle range continues for a certain period of time or longer.

[Configuration of Normalized Timing Control Device 1]
The normalized timing control device 1 includes a storage unit 11 and a control unit 12. The storage unit 11 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk, and the like. The storage unit 11 stores a program executed by the control unit 12.

The control unit 12 is a computing resource including a processor such as a CPU (Central Processing Unit). The control unit 12 realizes the functions as the acquisition unit 121, the calculation unit 122, and the instruction output unit 123 by executing the program stored in the storage unit 11.

The acquisition unit 121 sequentially acquires the distance from the image pickup device detected by the detection device 2 to the driver's face from the detection device 2 at predetermined intervals. For example, the acquisition unit 121 stores the distance in the storage unit 11 each time the distance is acquired in sequence.

The calculation unit 122 calculates the statistics of a plurality of distances acquired by the acquisition unit 121. The calculation unit 122 calculates, for example, the variance or standard deviation of a plurality of distances. Specifically, the calculation unit 122 calculates the variance or standard deviation of a plurality of distances acquired by the acquisition unit 121 within a predetermined period while the vehicle is traveling. The predetermined period (hereinafter referred to as the acquisition period) may be appropriately set so that the influence of a sudden change in the driver's facial condition on the variance or standard deviation of a plurality of distances can be reduced. The specific value of the acquisition period may be, for example, set between 30 seconds and 60 seconds, and more specifically, 50 seconds, but the acquisition period is not limited to this.

The calculation unit 122 calculates the variance or standard deviation when all the plurality of distances within the acquisition period are included in the predetermined range. The predetermined range is a range of values that can be taken when the driver takes a predetermined driving posture. The predetermined driving posture is, for example, a stable posture in which changes in the driver's posture and face orientation are relatively small. The stable posture is a posture taken by the driver when the vehicle travels at a constant speed of a certain level or higher. When the vehicle travels at a constant speed above a certain level, it is a case where the vehicle travels on a highway or the like.

Hereinafter, a method in which the calculation unit 122 calculates the standard deviation of a plurality of distances will be described with reference to FIG. FIG. 2 is a diagram for explaining a method of calculating the standard deviation of a plurality of distances. The horizontal axis of FIG. 2 indicates the time t, and the vertical axis indicates the distance. The solid line is a graph schematically showing the time change of the distance Z acquired by the acquisition unit 121. The range A is a range of values that can be taken when the driver takes a stable posture.

First, a case where the end point of the acquisition period is time t1 will be described. The calculation unit 122 determines whether or not the plurality of distances Z acquired by the acquisition unit 121 within the period B1 having the time t1 as the end point of the period and stored in the storage unit 11 are all included in the range A. Specifically, the calculation unit 122 determines whether or not each of the plurality of distances Z in the period B1 is included in the range A. As shown in FIG. 2, since there is a distance Z not included in the range A in the period B1, the calculation unit 122 does not calculate the standard deviation in this case.

Next, a case where the end point of the acquisition period is time t2 will be described. The calculation unit 122 determines whether or not the plurality of distances Z acquired by the acquisition unit 121 within the period B2 whose end point is the time t2 and stored in the storage unit 11 are all included in the range A. As shown in FIG. 2, since there is no distance Z not included in the range A within the period B2, in this case, the calculation unit 122 calculates the standard deviation. In the following description, it is assumed that the time t2 is the time when the plurality of distances Z are all included in the range A. That is, the calculation unit 122 starts calculating the standard deviation when all the plurality of distances Z are included in the range A.

The instruction output unit 123 determines whether or not the standard deviation calculated by the calculation unit 122 is equal to or less than the determination threshold value. When the standard deviation is equal to or less than the determination threshold value, the instruction output unit 123 outputs an instruction to start normalization of the data indicating the state of the driver's face to the detection device 2. For example, when the standard deviation is equal to or less than the determination threshold value, the instruction output unit 123 outputs an instruction to start normalization of the driver's eye opening degree to the detection device 2. The determination threshold value is a value at which the driver can be regarded as having a stable posture, and may be appropriately determined by an experiment or the like. Further, the instruction output unit 123 may determine whether or not the standard deviation is the determination threshold value on condition that the vehicle speed of the vehicle is equal to or higher than the predetermined vehicle speed. The predetermined vehicle speed is, for example, the minimum speed of a highway, and a specific value of the predetermined vehicle speed is, for example, 50 kilometers per hour.

By doing so, the instruction output unit 123 can cause the detection device 2 to start normalization of the degree of eye opening in a state where the vehicle travels on a highway or the like at a predetermined vehicle speed or higher and the driver's posture can be regarded as stable. can. Therefore, the detection device 2 can normalize the degree of eye opening in a stable state in which the orientation and posture of the driver's face do not change, so that the accuracy of normalization can be improved.

Further, the instruction output unit 123 may output an instruction to start normalization to the detection device 2 when the state in which the standard deviation is equal to or less than the determination threshold value continues for a predetermined time longer than the acquisition period. The predetermined time is a deemed time that can be regarded as traveling on a road without traffic lights or intersections such as an expressway. The specific value of the deemed time is, for example, 3 minutes.

FIG. 3 is a diagram for explaining that an instruction to start normalization is output when the state in which the standard deviation σ is equal to or less than the determination threshold value R continues for the deemed time C. The horizontal axis of FIG. 3 indicates the time t, and the vertical axis indicates the standard deviation. The solid line is a graph schematically showing the time change of the standard deviation σ calculated by the calculation unit 122. The starting point of the graph of the standard deviation σ is the time t2 when all the plurality of distances Z are included in the range A. It is assumed that the standard deviation σ is equal to or less than the determination threshold value R at the time t3 shown in FIG.

The instruction output unit 123 outputs an instruction to start normalization when the state in which the standard deviation σ is equal to or less than the determination threshold value R indicated by the broken line continues for the deemed time C. Specifically, first, when the instruction output unit 123 determines that the standard deviation σ is equal to or less than the determination threshold value R, the instruction output unit 123 stores in the storage unit 11 the determination time t3 that determines that the standard deviation σ is equal to or less than the determination threshold value R. Let me. When the determination time is already stored in the storage unit 11, the instruction output unit 123 does not update the determination time even if it newly determines that the standard deviation is equal to or less than the determination threshold value R. Further, when the instruction output unit 123 determines that the standard deviation σ is larger than the determination threshold value R when the determination time is stored in the storage unit 11, the instruction output unit 123 deletes the determination time from the storage unit 11.

After determining that the standard deviation σ is equal to or less than the determination threshold value R, the instruction output unit 123 determines whether or not the state in which the standard deviation σ is equal to or less than the determination threshold value continues for the deemed time C. For example, the instruction output unit 123 determines whether or not the deemed time C has elapsed from the determination time t3 stored in the storage unit 11. The instruction output unit 123 normalizes the state in which the standard deviation σ is equal to or less than the determination threshold R from the time t3 when the standard deviation σ becomes equal to or less than the determination threshold value to the time t4 after the deemed time C elapses. The instruction to start is output to the detection device 2.

By doing so, the instruction output unit 123 is in a state in which the driver drives the vehicle in a stable state in which the direction and posture of the face do not change, and the vehicle travels on a road such as an expressway where there is no signal or intersection. Outputs an instruction to start normalization if it can be considered to be. As a result, the detection device 2 can normalize the degree of eye opening while the driver's posture is stable.

FIG. 4 is a histogram of the degree of eye opening after normalization detected by the detection device 2. The horizontal axis of FIG. 4 indicates the output eye opening degree value, and the vertical axis indicates the frequency at which the value is output.

The broken line L1 is a histogram of the degree of eye opening detected by the detection device 2 when the degree of eye opening is normalized in a state where the direction or posture of the face changes. The peak of the broken line L1 exists at a position where the degree of eye opening is 120 (%). In this case, it can be said that the degree of eye opening of the driver is detected to be larger than the actual state. Therefore, the alarm device 3 determines the driver state using a value deviating from the actual driver state. As a result, the alarm device 3 determines that the driver is not in the doze state even though the driver is in the doze state, and cannot give an alarm in the situation where the alarm should be given.

The solid line L2 is a histogram of the degree of eye opening detected by the detection device 2 after the normalization timing control device 1 according to the present embodiment outputs an instruction to start normalization. As shown in FIG. 4, the peak of the solid line L2 exists at an eye opening degree of around 100 (%). In this case, it can be said that a value corresponding to the actual state of the driver's eye opening is detected. Therefore, the alarm device 3 can determine the driver status using the value corresponding to the actual driver status, and can output an appropriate alarm for the driver status.

[Process to control the normalization timing executed by the normalization timing control device 1]
With reference to FIG. 5, a flow of processing for controlling the normalization timing executed by the processor of the normalization timing control device 1 mounted on the vehicle will be described. FIG. 5 is a flowchart showing an example of processing for controlling the normalization timing.

First, the acquisition unit 121 acquires the distance Z from the image pickup device to the driver's face detected by the detection device 2 (step S1). Next, the calculation unit 122 sets the values that can be taken when the driver takes a stable posture for all of the plurality of distances Z within the acquisition period whose end point is the time when the acquisition unit 121 last acquired the distance Z. It is determined whether or not it is included in the range A (step S2). Specifically, the calculation unit 122 determines whether or not each of the plurality of distances within the acquisition period is included in the range of values that can be taken when the driver takes a stable posture. When the plurality of distances Z within the acquisition period are all included in the range A (Yes in step S2), the calculation unit 122 calculates the standard deviation σ of the plurality of distances Z (step S3).

The instruction output unit 123 determines whether or not the standard deviation σ calculated by the calculation unit 122 is equal to or less than the determination threshold value R that can be regarded as the driver in the stable posture (step S4). Then, when the instruction output unit 123 determines that the standard deviation σ is equal to or less than the determination threshold value R (Yes in step S4), the instruction output unit 123 stores the determination time for determining that the standard deviation σ is equal to or less than the determination threshold value R in the storage unit 11. (Step S5). When the determination time is already stored in the storage unit 11, the instruction output unit 123 does not update the determination time even if it newly determines that the standard deviation is equal to or less than the determination threshold value R. Further, when the instruction output unit 123 determines that the standard deviation σ is larger than the determination threshold value R when the determination time is stored in the storage unit 11, the instruction output unit 123 deletes the determination time from the storage unit 11.

After storing the determination time in the storage unit 11, the instruction output unit 123 determines whether or not the state in which the standard deviation σ is equal to or less than the determination threshold value R continues during the deemed time C (step S6). For example, the instruction output unit 123 determines whether or not the deemed time C has elapsed from the determination time stored in the storage unit 11.

The instruction output unit 123 issues an instruction to start normalization of data indicating the facial condition when the state in which the standard deviation σ is equal to or less than the determination threshold value R continues for the deemed time C (Yes in step S6). Is output to (step S7). For example, the instruction output unit 123 outputs an instruction to start normalization of the driver's eye opening degree. In addition, the instruction output unit 123 outputs an instruction to start normalization of data indicating the state of the face, and also outputs an instruction to normalize each of the data indicating the position of the face, the orientation of the face, and the degree of eye opening. ..

In the normalization timing control device 1, when at least one of the plurality of distances Z within the acquisition period is not included in the range A (No in step S2), the standard deviation σ is larger than the determination threshold value R (step S4). In (No), and when the state in which the standard deviation σ is equal to or less than the determination threshold value R does not continue for the deemed time C (No in step S6), the process proceeds to step S1. The normalized timing control device 1 executes the above processing while the vehicle is running. The normalized timing control device 1 may execute the above process when the vehicle speed of the vehicle is equal to or higher than a predetermined speed.

As described above, the process of controlling the normalization timing has been described as being executed by the processor of the normalization timing control device 1 mounted on the vehicle, but the subject that executes the process is limited to the normalization timing control device 1. No. For example, a server capable of communicating with the detection device 2 may sequentially acquire the distance Z and execute a process of controlling the normalization timing. Further, when a plurality of vehicles travel in a platoon, the normalization timing control device 1 mounted on any of the plurality of vehicles controls the normalization timing in the other vehicle based on the distance acquired from the other vehicle. The process may be executed.

[Effect of Normalization Timing Control Device 1 According to the Embodiment]
As described above, the normalization timing control device 1 sequentially acquires the distance Z from the image pickup device that images the driver's face to the driver's face from the detection device 2 that detects the data indicating the state of the driver's face in the vehicle. do. Then, the normalized timing control device 1 calculates the variance or standard deviation σ of a plurality of distances Z acquired within the acquisition period while the vehicle is running, and when the calculated standard deviation σ is equal to or less than the determination threshold R, the driver An instruction to start normalization of data indicating the state of the face is output to the detection device 2.

In this way, the normalization timing control device 1 detects an instruction to start normalization of data indicating the driver's face state when it can be regarded as a stable state in which the driver's face orientation and posture do not change. Send to. By doing so, the detection device 2 can start normalization of the data indicating the facial condition in a state where the orientation and posture of the driver's face are stable, so that the data indicating the facial condition can be appropriately normalized. can. Therefore, the detection device 2 can output data that correctly indicates the state of the driver's face, which is detected after being appropriately normalized. As a result, the alarm device 3 can correctly determine the driver's state by using the data indicating the face state correctly, so that it is possible to reduce unalarmed or false alarms and output an appropriate alarm.

In the above description, the calculation unit 122 calculates the standard deviation of a plurality of distances, but the present invention is not limited to this, and the same applies to the case where the calculation unit 122 calculates the variance of a plurality of distances. Method can be used. The determination threshold value when the calculation unit 122 calculates the variance may be appropriately determined by an experiment or the like, similarly to the determination threshold value of the standard deviation.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist thereof. be. For example, all or a part of the device can be functionally or physically distributed / integrated in any unit. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination also has the effect of the original embodiment. For example, the detection device 2 may use a value normalized in the past from the start of the driving force source of the vehicle until the instruction to start the normalization is received from the normalization timing control device 1. In this case, the detection device 2 may newly start the normalization when it receives an instruction to start the normalization from the normalization timing control device 1. Then, the detection device 2 updates the previously normalized value with the newly normalized value.

1 Normalization timing control device 11 Storage unit 12 Control unit 121 Acquisition unit 122 Calculation unit 123 Instruction output unit 2 Detection device 3 Alarm device

Claims (5)

From the detection device that detects the data indicating the state of the driver's face of the vehicle, the acquisition unit that sequentially acquires the distance from the image pickup device that images the driver's face to the driver's face, and
A calculation unit that calculates the variance or standard deviation of a plurality of the distances acquired by the acquisition unit within a predetermined period while the vehicle is running.
When the variance or the standard deviation calculated by the calculation unit is equal to or less than the determination threshold value, an instruction output unit that outputs an instruction to start normalization of data indicating the face state of the driver to the detection device, and an instruction output unit.
A normalized timing controller comprising. When the variance or the standard deviation is equal to or less than the determination threshold value, the instruction output unit outputs an instruction to start normalization of the eye opening degree of the driver to the detection device.
The normalized timing control device according to claim 1. The calculation unit calculates the variance or the standard deviation when the plurality of distances within the predetermined period are all within the range of values that can be taken when the driver takes a predetermined driving posture.
The normalized timing control device according to claim 1 or 2. The instruction output unit outputs an instruction to start the normalization to the detection device when the state in which the variance or the standard deviation is equal to or less than the determination threshold value continues for a predetermined time.
The normalized timing control device according to any one of claims 1 to 3. The computer processor
A step of sequentially acquiring the distance from the detection device that detects the data indicating the state of the driver's face of the vehicle to the face of the driver from the image pickup device that images the driver's face.
A step of calculating the variance or standard deviation of a plurality of the distances acquired within a predetermined period, and
A step of determining whether or not the calculated variance or standard deviation is equal to or less than the determination threshold value.
When it is determined that the variance or the standard deviation is equal to or less than the determination threshold value, a step of outputting an instruction to start normalization of data indicating the facial condition of the driver to the detection device, and
Normalization timing control method to execute.

Images