JP2000351337A - Traveling body controller - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a traveling body controller capable of controlling a traveling body accurately in accordance with changes of an operator's strain state. SOLUTION: When it is judged that a driver has his/her mind's room for avoiding danger at step S6, change speed VL of driver's pupil diameter is greatly changed. When one's own vehicle is in predetermined external environment (S10 to S12), it is judged that driver's tension is increased and his/her spiritual room for driving and operation is reduced to change over information modes for information supply and alarm informing and stop informing of information having low priority (S13 to S15).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving body control apparatus, and more particularly to a control apparatus suitable for use in a typical moving body such as an automobile.

[0002]

2. Description of the Related Art Conventionally, in an automobile which is a typical moving body, an alarm device for a driver's (operator) 's inattentiveness has been proposed as an example of a control device for controlling the automobile from the viewpoint of preventive safety. For example, JP-A-6-321
No. 1011 detects the driver's line of sight based on the result of imaging the area around the eyes of the driver's head,
A technique for controlling a display device provided in a vehicle according to the detection result has been proposed.

Japanese Patent Laid-Open Publication No. Hei 6-270711 discloses a pupil shape of a driver based on a result of photographing an area around an eye of the driver's head, and awakening of the driver based on the detection result. There has been proposed a device that determines a state and issues an alarm when the user is awake and not awake.

[0004]

However, in the above conventional example, since the control is performed based on the detected driver state at a certain moment, it is expected that the driver will feel uncomfortable. In addition, even when the driver is in the awake state, the driver may not be able to perform an appropriate driving operation when the driver is in an excessively nervous state due to fear or surprise.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a moving object control device that controls a moving object accurately in response to a change in the tension state of an operator.

[0006]

In order to achieve the above object, a moving object control apparatus according to the present invention is characterized by the following configuration.

That is, a camera for photographing the head and face of an operator of a moving body, pupil diameter detecting means for detecting a value relating to the pupil diameter of the operator based on a video signal from the camera, The external environment detected by the external environment detecting means is in a predetermined state, and the change speed of the pupil diameter detected by the pupil diameter detecting means is larger than a predetermined value. At the time, the tension state determining means for determining that the operator is in a tension state, based on the determination result by the tension state determination means,
Control means for controlling the moving body.

[0008] For example, the external environment detecting means includes, as an external environment to be detected, a traveling environment of the moving body and / or
Or a means for detecting an operating state, wherein the control means notifies a plurality of types of information with different degrees of urgency regarding the traveling environment and / or the operating state of the mobile object detected by the external environment detecting means, When the operator is determined to be in a nervous state by the nervous state determining unit, a notification unit for selectively notifying a highly urgent information as an alarm among a plurality of types of information on the driving environment and the driving state may be included. good.

A camera for photographing the head and face of the operator of the moving object; pupil diameter detecting means for detecting a value relating to the pupil diameter of the operator based on a video signal from the camera; When the change rate of the pupil diameter detected by the detection means is greater than a predetermined value, the movement is determined based on a determination result by the tension state determination means for determining that the operator is in a tension state, and the tension state determination means. Control means for controlling the body.

Alternatively, a camera for photographing the head and face of an operator of a moving body, a value relating to the pupil diameter of the operator is detected based on a video signal from the camera, and the size of the detected pupil diameter is determined. Pupil diameter detecting means for detecting that the pupil diameter is larger than a predetermined value or cannot be detected for a predetermined time, and when the detected pupil diameter is larger than a predetermined value or cannot be detected for a predetermined time. Control means for automatically causing the moving body to perform a danger avoidance operation.

[0011]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of a vehicle control apparatus according to the present invention;

[First Embodiment] FIG. 1 is a diagram showing a system configuration of an automobile equipped with a mobile control device according to a first embodiment. FIG. 2 is a diagram showing a driver's seat of the automobile shown in FIG.

In FIGS. 1 and 2, reference numeral 1 denotes a control unit for performing control described later. Reference numeral 2 denotes a laser radar for detecting a distance from a preceding obstacle. Reference numeral 3 denotes an infrared camera for photographing the front. Reference numeral 4 denotes a magnetic marker sensor that detects magnetism from a magnetic marker provided on a road to detect a predetermined traveling position or the like. Reference numeral 5 denotes a line CCD (Charge Coupled Device) that is provided on, for example, a side mirror or the rear of the vehicle body and captures images of the side and rear of the vehicle. Reference numeral 6 denotes a driving lane detection CCD camera that photographs the front of the vehicle so as to detect the driving lane by a white line or the like on the road. Reference numeral 7 denotes a vehicle speed sensor that detects a traveling speed. 8
Is a yaw rate sensor that detects a yaw angle in order to detect G in the direction and the lateral direction of the vehicle. Reference numeral 9 denotes a throttle sensor for detecting a throttle opening. Reference numeral 10 denotes a brake switch for detecting an operation state of the brake. 11
Is a steering angle sensor that detects the steering angle of the steering wheel. Reference numeral 12 denotes a turn signal sensor for detecting an operation state of the turn signal switch. Reference numeral 13 denotes a driver monitor unit that detects the position of the driver's head and the point of gaze as described later. 14 is the map information and the received GP
Based on S (Global Positioning System) signal, etc.
It is a navigation unit that detects the current position of the vehicle and guides the route to the destination. Reference numeral 15 denotes a road-to-vehicle communication unit that communicates traffic information, information on an obstacle in front, information on the presence or absence of a pedestrian walking on a pedestrian crossing, and the like with a wireless device provided on a road. Reference numeral 21 denotes a display such as a liquid crystal display (LCD) or a head-up display (HUD) for providing an output result of the navigation unit 14 and various kinds of information. Reference numeral 22 denotes a speaker that outputs various sounds and provides information and alerts to inattentive driving described below. Reference numeral 23 denotes an infrared light projection lamp which is provided on, for example, a column cover of a steering wheel and emits infrared light to the head and face of the driver. And 24
Is an infrared projection area imaging camera that is provided on a column cover of a steering wheel, for example, and captures light reflected from the head and face by the infrared projection lamp 23.

Here, a functional configuration of control in the above-described system configuration will be described.

FIG. 3 is a block diagram showing a configuration of a control function in the mobile control apparatus according to the first embodiment.
Each module described below in the present embodiment represents a functional unit of software executed by a CPU (not shown) included in the control unit 1.

As shown in FIG. 1, the external environment judgment module of the control unit 1 includes a laser radar 2, an infrared camera 3, a magnetic marker sensor 4, a line CCD 5,
Using the output signals of the traveling lane detecting CCD camera 6, the detection result of the navigation unit 14, and the information obtained by the road-to-vehicle communication unit 15, sensing of the external environment of the vehicle is performed. Note that a general method is used for sensing the external environment using each of these output signals, and detailed description in this embodiment is omitted.
In the present embodiment, as an example, sensing of the external environment is performed using a plurality of types of detection signals as described above.
The sensing is not limited to this, and sensing may be performed by appropriately using some of the plurality of types of detection signals.

In the danger determination module of the control unit 1, the output signals of the vehicle speed sensor 7, the yaw rate sensor 8, the throttle sensor 9, the brake switch 10, the steering angle sensor 11, and the turn signal sensor 12 are sensed as the vehicle state sensing. , Road-to-vehicle communication unit 15
The dangerous state is determined using the obtained information. In this danger judgment module,
In other words, when it is determined that there is no room for danger avoidance by the driver, vehicle control such as automatic braking by braking operation, throttle operation, or shifting operation of an automatic transmission (AT), or automatic steering is performed without waiting for danger avoidance by the driver. Do. Note that a general method is used for the judgment of the danger state using these output signals and the vehicle control, and a detailed description in this embodiment is omitted.

In the driver status determination module of the control unit 1, the driver monitor unit 13
The driver's gaze point, head-face position, and pupil diameter detected by the detection result and the detected pupil diameter change speed (the amount of change in pupil diameter per unit time) are used by the driver. A state determination is performed (details will be described later).

The information provision / warning notification determination module of the control unit 1 determines whether information provision / warning notification is necessary based on the determination result by the danger determination module and the determination result by the driver state determination module. The switching of the notification mode is determined, and the information is displayed on the display 21 according to the determination result and the alarm is issued by the speaker 22 (the details will be described later).

<Detection of gazing point> Next, a method of detecting the gazing point of the driver by the driver monitor unit 13 will be described.

FIG. 4 is a block diagram showing a system configuration of the driver monitor unit according to the first embodiment. Each module described below in the present embodiment includes a CP (not shown) provided in the driver monitor unit 13.
U represents a functional unit of software executed.

As shown in the figure, an infrared transmitting filter is provided in a light emitting portion of the infrared light emitting lamp 23 and a light receiving portion of the infrared light emitting area imaging camera 24. Extracts an image of the driver's head and face by performing general binarization processing and feature point extraction processing in an image processing module based on a video signal output from the infrared light projection area imaging camera 24. Then, based on the extracted image of the head and face, the gaze point detection module detects the head and face direction Dh, the gaze direction Ds, and the pupil diameter L in order to detect the gaze point of the driver.

Next, the function of the driver monitor unit 13 will be described with reference to FIGS.

FIG. 5 is a diagram showing a flowchart of the gazing point detection process in the first embodiment, and shows functions realized by the image processing module and the gazing point detection module.

In the figure, step S51: The infrared projection lamp 23 emits light to the driver's head and the analog video signal of the head and face captured by the infrared projection area imaging camera 24 is converted into an image. The video signal is captured by a processing module and subjected to general binarization processing to convert the video signal into digital multivalued image data for each pixel.

Step S52: A face image portion of the driver is extracted from the obtained multi-valued image data using a general image processing technique, and a plurality of feature points (for example, the inner and outer corners of the eye) included in the extracted face image portion are extracted. , Nostrils, etc.).

Step S53: From the extracted image data of the face image portion, the position of the reflection point and the position of the pupil generated in the cornea of the driver's eyeball by infrared projection are determined using a general image processing technique. To detect.

Step S54: The maximum value of the number of pixels in the horizontal direction of the image data corresponding to the detected position of the pupil is detected as the pupil diameter L. Here, attention is paid to the horizontal direction because, when attention is paid to the vertical direction, an accurate pupil diameter cannot be detected due to the influence of “blinking” by the driver.

Step S55: Based on the position of the feature point detected in step S52, the inclination of the driver's head face in a predetermined three-dimensional coordinate space is calculated, so that the head direction of the driver's head (head) Face direction) Ds is measured.

Step S56: The corneal reflection point detected in step S53 and the head-face direction D detected in step S55
s and the direction of the driver's line of sight (gaze direction) D
s is detected.

Step S57: On the basis of the gaze direction Ds detected in step S56 and predetermined positions inside and outside the vehicle (front position, rear-view mirror mounting position, left and right side mirror mounting positions, etc.) stored in advance, the driver's Detect the fixation point.

The above-mentioned gazing point detection processing is performed, for example, at each control cycle of the microcomputer (not shown) of the driver monitor unit 13 and the detected head face direction Dh,
The values of the line-of-sight direction Ds, the pupil diameter L, and the gazing point are stored in the RAM or the like of the microcomputer in chronological order for a predetermined number of detections.

<Control Processing by Control Unit 1> Next, control processing in the control unit 1 will be described.

In the present embodiment, the results of detection of the driver's gaze point, head and face position, and pupil diameter detected by the driver monitor unit 13 and the change speed of the detected pupil diameter (change in pupil diameter per unit time) The state of the driver is determined using the detection result of the amount. First, the relationship between the ever-changing pupil diameter of a person and the state of tension will be described.

The pupil diameter of a human generally ranges from 2.2 mm to 1 mm.
It is known that it changes within a range of about 3 mm and the pupil diameter increases for an object of interest. For example, it is determined whether there is an obstacle on the side of the vehicle before driving operation such as lane change. The pupil diameter of a driver changes greatly when the driver is watching the side mirror or the like. Here, an example of an experimental result regarding the movement of the head and the pupil of the driver will be described with reference to FIG.

FIG. 6 is a diagram showing the results of an experiment on the movement of the driver's head and face and the movement of the pupil when changing lanes.

As shown in the figure, between 30 seconds and 15 seconds before the end of the merging to determine whether or not the lane can be changed, the driver's head faces the horizontal direction many times to confirm the rear of the vehicle. It can be seen that the pupil is sharply turned left and right and is gazing at the side mirror and the room mirror during the lane change, and the movement of the head face and the pupil with respect to the vertical direction. It can be seen that there is no significant change. Accordingly, it can be determined that the driving operation performed when the subject changes lanes does not include useless visual observation such as inattentiveness. That is, even when the gazing point of the driver is directed forward, when the pupil diameter is smaller than the predetermined reference value, it can be determined that the driver's attention to the driving operation is distracted, Even when the driver's gaze is not pointed forward,
Since the pupil diameter may be larger than a predetermined reference value due to, for example, staring at a side mirror to visually check the rear side, depending on the detection result of the driver's pupil diameter alone, the driver may not be nervous about the driving operation. I cannot judge it.

Next, how the size of the pupil diameter of the driver changes according to the surrounding environment (driving environment) will be described.

FIGS. 7 and 8 show experimentally determined pupil diameters L.
It is a figure showing a situation of change of.

In FIG. 7, it can be seen that the pupil diameter (the number of pixels) of the subject is rapidly reduced in response to the sudden increase in the surroundings from the measurement time 130 to 380. This situation corresponds to, for example, a case where light of headlights of another vehicle traveling in the oncoming lane enters the driver's eyes in a dark traveling environment such as at night or in a tunnel.

In FIG. 8, it can be seen that the pupil diameter (the number of pixels) increases and changes sharply up and down because the subject is surprised after the measurement time 150. The driver may be nervous or anxious (feared) due to, for example, a sudden interruption of another vehicle ahead of the driving lane.
Equivalent to feeling.

Therefore, in the present embodiment, the driver's pupil diameter changes in accordance with the external light / dark state, which is an example of the driving environment outside the vehicle, and the driver's pupil diameter felt surprised or uneasy (fear). The control unit 1 controls the information provision and the alarm issuance, paying attention to the drastic change of.

FIG. 9 is a flowchart showing the control processing by the control unit as the moving body control device according to the first embodiment. The external environment judgment module, the danger judgment module, and the driver state judgment of the unit shown in FIG. Module, and the functions implemented by the information provision / alarm alert determination module.

In the figure, step S1: laser radar 2, infrared camera 3, magnetic marker sensor 4, line C
Using the output signals of the CD 5 and the CCD camera 6 for detecting the traffic lane, the external environment of the vehicle is sensed by a general method.

Step S2: Navigation unit 1
The current time information is obtained from 4 or the like, and the surrounding light / dark state is determined based on the obtained time. That is, if the obtained time information is from 19:00 to about 4:00, the dark environment, 5
If it is between 18:00 and 18:00, it is determined that the environment is bright. Also,
It is preferable to obtain the date information together with the time information, and to consider variations in the light and dark state depending on the season.

Step S3: Navigation unit 1
According to 4 (information obtained by the road-to-vehicle communication unit 15 may be used), the current traveling position of the vehicle, the shape of the road on the course, and the like are detected by a general method.

Step S4: vehicle speed sensor 7, yaw rate sensor 8, throttle sensor 9, brake switch 1
0, the current vehicle state (straight running, turning state, etc.) is detected by a general method using the output signals of the steering angle sensor 11, and the turn signal sensor 12.

Step S5: Information on the road (curve radius, speed regulation information, obstacle information, etc.) is obtained from the roadside equipment via the road-vehicle communication unit 15.

Step S6: Based on the results detected in steps S1 to S5, a general method (for example, converting the detection results in each of the above steps into numerical values (points) and taking the product or sum of these numerical values) By
It is determined whether or not there is a dangerous state (for example, a state in which a collision with another vehicle traveling ahead (or stopped) or a collision with a nearby obstacle may occur). At this time, if the determination result satisfies a predetermined condition, it is determined that there is no room for the driver to avoid danger, and the brake operation, throttle operation, or automatic transmission (A
Vehicle control such as automatic braking or automatic steering by the shift operation of T) is performed.

Step S7, Step S8: Step S
When it is determined from the determination result in step 6 that there is room for danger avoidance by the driver, it is determined whether information provision / warning is necessary (step S7), and if the determination result is NO (when unnecessary), Return, and if YES (if necessary), proceed to step S9 (step S9).
8).

Step S9: Detection of the driver's head-face direction Dh, detection of the line-of-sight direction Ds, pupil diameter stored in time series by the driver monitor unit 13 described above with reference to FIG. L, and the result of detection of the gazing point are obtained (in the present embodiment, at least the pupil diameter L may be obtained).

Steps S10 and S11: Changes in the pupil diameter L obtained in step S9 per unit time (change speed V
L) is calculated (step S10), and it is determined whether or not the calculated absolute value of the change speed VL is larger than a predetermined reference value VS (step S11), and YES (| VL |> VS)
In the case of, it is determined that the driver's nervous state is considerably higher than a certain reference state or that the driver is mentally upset, and that the mental margin for the driving operation is reduced, and the process proceeds to step S12. NO (| VL | ≦ VS)
In this case, it is determined that the driver's tension has not increased so much, and that there is a normal mental allowance for the driving operation, and the process proceeds to step S15. Here, as the reference value VS, the average change rate of the pupil diameter of the driver experimentally obtained at the initial timing of driving in a driving environment where the driver has a low mental burden on driving is used. I do.

Step S12: As a predetermined external environment,
For example, it is determined whether the light / dark state around the vehicle detected in step S2 is night or whether the current position of the own vehicle detected in step S3 is in a tunnel. When the determination is YES (when the host vehicle is under a predetermined external environment), the changing speed VL of the pupil diameter is determined in step S11.
Has changed significantly because the driver's eyes tried to adapt to the situation where the headlights of oncoming vehicles frequently intersected at night or in a tunnel for a short period of time, and / or It is estimated that the driver's tension has increased due to the driving operation for passing the vehicle, and the process proceeds to step S13. If NO (when the vehicle is not under a predetermined external environment), the process proceeds to step S13.
Go to 15. At this time, it is only necessary to use the detection results of the laser radar 2 and the infrared camera 3 in order to reliably determine the passing with respect to the oncoming vehicle.

Step S13: The visual information to be displayed on the display 21 among the information determined to be necessary to provide the information and the alarm issuance in the step S8 is set to be switched to the notification by the voice information by the speaker 22.

Step S14: Information having a low priority (for example, the current position information detected based on the GPS signal) is provided so that the information determined to be necessary to provide information and issue an alarm in step S8 is notified with priority. , Information such as music with high entertainment characteristics, or information with low urgency such as instantaneous fuel consumption) is set to be stopped.

Step S15: When the notification mode is set in steps S13 and S14, the set notification condition is set. On the other hand, when NO is determined in step S11 or step S12, the original notification mode is maintained and the information is set in step S8. Information to the driver that the provision / warning is necessary is notified to the driver, and the process returns.

In the control processing of FIG. 9 described above, as an example of the driving environment outside the vehicle, the light / dark state outside the host vehicle is determined in step 12, but the present invention is not limited to this. By taking into account the road conditions ahead of the host vehicle (steep curves, traffic congestion, etc.) or the change in the steering angle, which can be obtained by the processing in step S5,
It may be estimated that the driver's tension is increasing or that the driver's tension is increasing.

As described above, according to the present embodiment, when it is determined that there is room for danger avoidance by the driver based on the determination result in step S6, the change speed VL of the pupil diameter of the driver greatly changes, and the vehicle When the vehicle is in a dark state, which is an example of a predetermined external environment, it is determined that the driver's nervousness is increased and the mental margin for the driving operation is reduced, and the notification mode of information provision / warning is switched, and the priority is determined. It is possible to accurately stop the notification of information having a low level. That is, it is possible to accurately control the moving body in accordance with the change in the tension state of the operator, and it is possible to improve preventive safety.

[Second Embodiment] Next, a second embodiment based on the moving object control device according to the above-described first embodiment will be described. In the following description, the same configuration as that of the first embodiment will not be described repeatedly, and the description will focus on the characteristic portions of the present embodiment.

As described with reference to FIGS. 7 and 8,
The driver's pupil diameter changes every moment in response to changes in the surrounding environment and changes in tension.However, if the driver's tension becomes extremely high because he / she is surprised or scared by a certain subject, the tension becomes lower. It is assumed that the driver may not be able to perform a driving operation to avoid danger while keeping his / her eyes on the target or closing the eyelids.

Accordingly, in the present embodiment, the occurrence of such an external environment is estimated using various sensor outputs provided as shown in FIGS. 1 and 2, and the change in the size of the driver's pupil diameter ( If the eyelids are closed and the pupil diameter cannot be detected, the degree of tension is estimated, and if it is estimated that a driving operation to avoid danger cannot be performed, the vehicle is braked. .

<Control Process by Control Unit 1> FIG. 10 is a flowchart showing a control process by the control unit as the moving object control device in the second embodiment.

In FIG. 10, steps S21 to S25 are the same as steps S1 to S5 in FIG. 9, and a description thereof will be omitted.

Steps S26 and S27: Based on the results detected in steps S21 to S25, a general method (for example, converting the detection results in the above steps into numerical values (points) and calculating the product or sum of the numerical values It is determined whether or not the vehicle is in a state where a rear-end collision with another vehicle traveling ahead (or stopped) can occur (step S26), and there is a risk of a rear-end collision (YES in step S27). ), The process proceeds to step S28,
If there is no danger of a rear-end collision (NO in step S27), the routine returns.

Step S28: Similar to step S9 in FIG. 9, the driver monitor unit 13 detects the driver's head-to-face direction D stored in time series for a predetermined number of detections.
h, the line-of-sight direction Ds, the pupil diameter L, and the detection result of the gazing point are obtained (in the present embodiment, at least the pupil diameter L may be obtained).

Step S29: The pupil diameter L obtained from the driver monitor unit 13 is equal to a predetermined reference value S and a coefficient K.
It is determined whether it is greater than the product of.

Here, as the reference value S, the average pupil diameter of the driver when the driver is surprised or feared during the driving operation and the tension is increased is experimentally obtained and used.

The coefficient K is set in advance as a map (table) for outputting a value that changes according to the turning radius R of the traveling lane detected in step S23 or S24. In other words, the driver has to concentrate on driving as the turning radius R is smaller, so that the driver's pupil diameter is generally larger. Therefore, in such a case, by setting the coefficient K output from the map to be large, the mental burden on the driving operation of the driver that changes according to the change of the external environment is taken into consideration.

In step S29, it is determined whether or not the pupil diameter L is larger than a product of a predetermined reference value S and a coefficient K, thereby determining the driver's tension. In this determination, when YES (when the pupil diameter L is larger than the calculated product), it is determined that the driver's tension is excessively high and there is no mental margin for performing a driving operation to avoid danger. If the result is NO (when the pupil diameter L is smaller than the calculated product), the flow proceeds to step S30.

Step S30: If the pupil diameter L is smaller than the calculated product in the determination in step S29, it is determined in step S28 whether or not the state in which the pupil diameter L cannot be detected over a predetermined control cycle continues.
(When the state where the pupil diameter L cannot be detected continues), it is determined that the driver has closed his eyelids in fear so much that there is no mental room to perform a driving operation to avoid danger. Proceeding to step S32, when NO (when the pupil diameter L is smaller than the calculated product), it is determined that there is a high possibility that the driving operation for avoiding danger is performed by the driver itself, and the process proceeds to step S31.

Step S31: In order to inform the driver that a driving operation for avoiding danger is necessary, information display / alarm issuance notifying the possibility of a rear-end collision is performed by the display 21 and / or the speaker 22. I do.

Step S32: The self-vehicle is braked by automatically performing a driving operation necessary for avoiding danger.
To return. Examples of the braking method include a braking operation, a throttle operation, and automatic braking by a shift operation of an automatic transmission (AT), and vehicle control such as automatic steering may be performed.

Incidentally, in the control processing of FIG.
As a state in which the pupil diameter L cannot be detected over the predetermined control cycle in step S28, it is assumed that the driver has closed his eyelids for fear, but in this state, the driver falls asleep by falling asleep. It may include a closed state. This is similar to the case where the driver closes his eyelids in fear because it is expected that the driver will not be able to perform appropriate driving operations to avoid danger even when the driver is dozing This is because the own vehicle should be automatically braked in step S32.

<Modification of Second Embodiment> Note that the coefficient K used in step S29 is not limited to a value corresponding to the turning radius R of the traveling lane, as described in this modification. The value may be a value according to the light / dark state that can be determined in step S22.

FIGS. 11 and 12 are diagrams showing examples of characteristics of a map stored in advance in a control unit as a moving object control device according to the second embodiment. FIG. 11 shows a light environment and a dark environment, respectively. R = 1000 m, which is regarded as a substantially straight line as the turning radius increases
1 shows that the coefficient K gradually decreases.
In R2, in each of the bright environment and the dark environment, R = 100 which is regarded as a substantially straight line as the turning radius increases.
The characteristic shows that the coefficient K gradually decreases to 0 m. That is, in a dark environment, the driving operation is generally burdened and the pupil diameter becomes large. Therefore, in such a dark environment, a threshold value (the reference value S and the coefficient K) for determining the driver's tension state is used. To increase the product), the coefficient K
11 or FIG. 1 so that is larger than in a bright environment.
2 is set. Although FIGS. 11 and 12 illustrate two types of environments, a bright environment and a dark environment, a plurality of dusk environments and the like sandwiched between the two types of characteristics may be set.

In the above-described modified example, the light and dark state is considered as an example of the surrounding driving environment. However, the present invention is not limited to this. For example, when the vehicle speed is high, in the case of stormy weather,
In the case of traffic congestion or the like in which many other vehicles exist around the own vehicle, the mental load on the driving operation is high in any case, and the pupil diameter increases. Therefore, instead of the brightness characteristic, the vehicle speed detected by the vehicle speed sensor 7, the operation state of the wiper switch, or the road-vehicle communication unit 15 is added to the map of FIG.
And / or storing characteristics indicative of the condition of other surrounding vehicles available by the navigation unit 14,
The coefficient K may be obtained from the map.

As described above, according to the second embodiment and its modification, the degree of tension is estimated by a change in the pupil diameter of the driver (including a case where the eyelid is closed and the pupil diameter cannot be detected). However, when it is estimated that a driving operation for avoiding danger cannot be performed as a result of the estimation, the traveling of the vehicle can be appropriately braked. That is, it is possible to accurately control the moving body in accordance with the change in the tension state of the operator, and it is possible to improve preventive safety.

[Third Embodiment] Next, a third embodiment based on the moving object control device according to the above-described first embodiment will be described. In the following description, the same configuration as that of the first embodiment will not be described repeatedly, and the description will focus on the characteristic portions of the present embodiment.

In the present embodiment, only the state of the detected change speed VL of the pupil diameter L is used as a judgment condition and the display 2
The information display is changed according to the driver's nervous state by changing the display mode when displaying the information in 1.

<Control Process by Control Unit 1> FIG. 13 is a flowchart showing a control process by the control unit as the moving object control device in the third embodiment.

In this figure, step S51: similar to step S9 in FIG. 9, the driver monitor unit 13 detects the driver's head-face direction Dh stored in time series for a predetermined number of detections, and the line-of-sight direction Ds , The pupil diameter L, and the detection result of the gazing point are obtained (in the present embodiment, at least the pupil diameter L may be obtained).

Steps S52 and S53: Similar to steps S10 and S11 in FIG. 9, as an index indicating the driver's tension, the amount of change in the pupil diameter L obtained in step S51 per unit time (change speed V
L) is calculated (step S52), and it is determined whether or not the absolute value of the calculated change speed VL is larger than a predetermined reference value VS (step S53), and YES (| VL |> VS)
In the case of, it is determined that the driver's nervousness is considerably higher than a certain reference state or that the driver is mentally upset and the mental margin for the driving operation is reduced, and the process proceeds to step S54. NO (| VL | ≦ VS)
In this case, the driver's tension is not so high, and it is determined that there is a mental margin for making a normal judgment on the driving operation, and the routine returns.

Step S54: Since it is determined in step S53 that the driver is in a nervous state, the display mode when displaying information on the display 21 is adjusted by the reference color, brightness, or contrast used on the display screen. Therefore, when the user is not nervous (NO in step S53)
The display mode is changed to a bright display mode that draws the driver's attention as compared with the above (when it is determined that the above is the case), and the process returns. At this time, as the control of the degree of adjustment of the display mode, the display mode may be changed to a bright display mode that draws the driver's attention as the pupil diameter L obtained in step S51 increases.

As described above, according to the third embodiment, the degree of tension is estimated based on the change in the pupil diameter of the driver, and the information is displayed on the display 21 according to the estimated result. The display mode can be changed, and the preventive safety can be improved.

In step S54 described above, the display mode of the display 21 is adjusted. However, the present invention is not limited to this. The volume of the audio output to the speaker 22 may be largely changed, or a predetermined artificial voice may be output. Prior to this, a process such as issuing a predetermined alarm sound to call attention may be performed.

In each of the above-described embodiments, the present invention is applied to an automobile. However, the present invention is not limited to this. The mobile object control device according to the present invention can be applied to a mobile object such as a train or an aircraft. In this case, various types of information obtained from the outside by the external environment determination module may be converted into sensor outputs according to the moving object to which the present invention is applied.

In each of the above-described embodiments, the function of each module performed by the control unit 1 has been described as software performed by the CPU provided in the unit. However, the present invention is not limited to this. Alternatively, dedicated hardware for realizing the functions of those modules may be employed.

[0088]

As described above, according to the present invention,
According to the present invention, it is possible to provide a moving body control device that accurately controls a moving body in accordance with a change in a tension state of an operator.

That is, claim 1, claim 10, and claim 12
According to the invention, the moving body can be accurately controlled according to the change in the tension state of the operator.

According to the second aspect of the present invention, when notifying the information to the notifying means to be controlled by the moving object control device, the information which is highly necessary according to the change in the tension of the operator ( For example, information for maintaining safe driving: claim 9) can be accurately selected and notified by voice information that is easy for the operator to recognize (claim 3).

According to the invention of claim 4, the headlight of the oncoming vehicle is seen by the driver, such as when the current position is in a tunnel (claim 6) or when driving at night (claim 7). Even in a dazzling external environment where the light is incident (claim 8), it is possible to accurately control the moving body in accordance with the change in the tension state of the operator that changes due to the external environment.

According to the fifth aspect of the present invention, it is possible to detect a state in which the light of the headlamp of the oncoming vehicle enters the driver's eyes while intersecting frequently in a driving environment where the surroundings are dark. In such a situation, the moving body can be controlled accurately.

According to the eleventh aspect of the present invention, when the object to be controlled by the moving body control device is a display mode of a display, the display mode is appropriately adjusted according to a change in the tension state of the operator. Can be controlled.

According to the thirteenth aspect, when the detected pupil diameter is larger than a predetermined value or cannot be detected for a predetermined time, it is determined that the driver is in an extremely tense state (including dozing). Then, the moving body can automatically take an appropriate danger avoidance operation (for example, automatic braking: claim 14), and in other cases, an alarm can be accurately given.

[Brief description of the drawings]

FIG. 1 is a diagram showing a system configuration of an automobile equipped with a mobile object control device according to a first embodiment.

FIG. 2 is a view showing a driver seat of the automobile shown in FIG.

FIG. 3 is a block diagram illustrating a configuration of a control function in the moving object control device according to the first embodiment.

FIG. 4 is a block diagram illustrating a system configuration of a driver monitor unit according to the first embodiment.

FIG. 5 is a diagram illustrating a flowchart of a gazing point detection process in the first embodiment.

FIG. 6 is a diagram showing experimental results regarding the movement of the driver's head and face and the movement of the pupil when changing lanes.

FIG. 7 is a diagram showing how the pupil diameter L is experimentally determined.

FIG. 8 is a diagram showing how the pupil diameter L changes experimentally.

FIG. 9 is a flowchart illustrating a control process performed by a control unit serving as a moving object control device according to the first embodiment.

FIG. 10 is a flowchart illustrating a control process performed by a control unit serving as a moving object control device according to the second embodiment.

FIG. 11 is a diagram illustrating a characteristic example of a map stored in advance in a control unit as a moving object control device according to a modification of the second embodiment.

FIG. 12 is a diagram illustrating a characteristic example of a map stored in advance in a control unit as a moving object control device according to a modification of the second embodiment.

FIG. 13 is a flowchart illustrating a control process by a control unit as a moving object control device according to the third embodiment.

[Explanation of symbols]

 1: control unit, 2: laser radar, 3: infrared camera, 4: magnetic marker sensor, 5: line CCD, 6: CCD camera for detecting traffic lane, 7: vehicle speed sensor, 8: yaw rate sensor, 9: throttle sensor, 10: brake switch, 11: steering angle sensor, 12: blinker sensor, 13: driver monitor unit, 14: navigation unit, 15: road-vehicle communication unit, 21: display, 22: speaker, 23: infrared light emitting lamp, 24: infrared projection area imaging camera,

Claims (14)

[Claims] A camera for photographing the head and face of an operator of a moving object; a pupil diameter detecting means for detecting a value relating to the pupil diameter of the operator based on a video signal from the camera; External environment detection means for detecting the external environment of the above, the external environment detected by the external environment detection means is in a predetermined state, and the pupil diameter change speed detected by the pupil diameter detection means is greater than a predetermined value A movement comprising: a tension state determination unit that determines that the operator is in a tension state; and a control unit that controls the moving body based on a determination result by the tension state determination unit. Body control device. 2. The external environment detecting means includes a means for detecting a traveling environment and / or a driving state of the moving body as the external environment to be detected, and the control means detects the external environment by the external environment detecting means. In addition to notifying a plurality of types of information with different urgencies related to the traveling environment and / or driving state of the moving body, and when the tension state determination unit determines that the operator is in a tension state, 2. The mobile body control device according to claim 1, further comprising a notification unit that gives priority to an information item having a high degree of urgency as an alarm among a plurality of types of information relating to the driving state. 3. The notifying unit is capable of displaying an image and outputting a sound, and the control unit is configured to output a sound as a mode of notification by the notifying unit when notifying the information having a high degree of urgency as a warning. 3. The moving object control device according to claim 2, wherein the image display is selected when the operator is not in a tension state when the operator is selected and the tension state determination unit determines that the operator is not in a tension state. 4. The moving object is a vehicle, wherein the predetermined state detected by the external environment detecting means is an environment where a surrounding environment where the moving object is located is dark. Item 4. The moving object control device according to Item 3. 5. The predetermined state detected by the external environment detecting means is a state in which the pupil diameter detected by the pupil diameter detecting means changes abruptly in a predetermined short period in an environment where the surrounding environment is dark. The mobile object control device according to claim 4, wherein 6. The navigation device, wherein the means for detecting a traveling environment of the moving object is a navigation device for detecting a current position of the moving object based on at least a GPS signal and map information. 5. The moving object control device according to claim 4, wherein the time corresponds to a time when the device detects that the moving object is located in the tunnel. 7. The moving object control device according to claim 4, wherein said means for detecting the traveling environment of the moving object estimates the dark environment based on time information. 8. The external environment detecting means includes, as means for detecting a traveling environment of the moving body, an obstacle detecting means capable of detecting that the moving body and an oncoming vehicle pass each other in the dark environment. The moving body control device according to claim 4, wherein: 9. The means for detecting a traveling environment and / or a driving state of the moving body includes:
The mobile object control device according to claim 2, wherein information for maintaining the safe running of the mobile object is detected. 10. A camera for photographing a head and face of an operator of a moving body, pupil diameter detecting means for detecting a value relating to the pupil diameter of the operator based on a video signal from the camera, and the pupil diameter. A tension state determination unit that determines that the operator is in a tension state when a change speed of the pupil diameter detected by the detection unit is greater than a predetermined value; and the movement based on a determination result by the tension state determination unit. A moving body control device, comprising: control means for controlling a body. 11. A display for displaying a predetermined information to an operator of the moving body when the tension state determination means determines that the operator is in a tension state. 2. The method according to claim 1, wherein the aspect is adjusted in accordance with a pupil diameter detected by the pupil diameter detecting means.
0. The moving object control device according to 0. 12. A camera for photographing the head and face of an operator of a moving body, a value related to a pupil diameter of the operator is detected based on a video signal from the camera, and a size of the detected pupil diameter is determined. Pupil diameter detecting means for detecting that the pupil diameter is larger than a predetermined value or cannot be detected for a predetermined time; and when the detected pupil diameter is larger than a predetermined value or cannot be detected for a predetermined time by the pupil diameter detecting means. Control means for causing the moving body to automatically perform a danger avoidance operation. 13. An external state detecting means for detecting a traveling environment and / or a driving state of the moving body, and a traveling environment and / or a driving environment detected by the external state detecting means.
Or a notifying unit for notifying an operator of the moving body of an operating state, wherein the control unit determines that the pupil diameter detected by the pupil diameter detecting unit is smaller than a predetermined value and larger than zero. 13. The moving object control device according to claim 12, wherein the notifying unit notifies the warning that the operator performs the danger avoidance operation. 14. The danger avoidance operation by the control means,
13. The moving object control device according to claim 12, wherein the operation is an automatic braking operation.