JP2000351339A - Controller for driving device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a controller for driving device capable of performing speedy and accurate control in accordance with operator's condition. SOLUTION: When it is judged that it is necessary to supply information and inform an alarm and a driver does not see ahead at step S7, it is judged whether the driver looks aside or not by utilizing a size of his/her pupil diameter L detected at step S8 (S9, S10). When the driver looks aside, the timing for supplying information and informing the alarm is quickened or sound volume of speaker output is increased (S12).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a control device for various driving devices driven by an operator, for example, to a control device suitable for application to a typical mobile object such as an automobile.

[0002]

2. Description of the Related Art Conventionally, in an automobile which is a typical driving device, an alarm device for a driver's inattentiveness has been proposed as an example of a control device for controlling the automobile from the viewpoint of preventive safety. In JP-A-6-270712, the gaze direction of a driver is detected based on the brightness gradient of an image captured by a CCD camera, and when the detected gaze direction does not match the traveling direction of the vehicle, it is determined that the driver is looking aside. An alarm device that outputs an alarm has been proposed.

[0003]

However, in the above conventional example, when the detected line of sight does not coincide with the traveling direction of the vehicle, even when the driver is not actively gazing at the line of sight. Since it is determined to be inattentive and an alarm is output, it is expected that the driver will feel strange. In order to eliminate this discomfort, the duration of the state in which the detected gaze direction of the driver is different from the traveling direction of the vehicle is measured, and when the duration is longer than a predetermined time, "the driver is looking aside". However, in this case, when the driver actually looks aside and the alarm output is required immediately, the alarm output is not started until the predetermined time has elapsed, which is a problem. .

[0004] Therefore, an object of the present invention is to provide a control device of a driving device that performs quick and accurate control according to the state of an operator.

[0005]

In order to achieve the above object, a control device for an operating device according to the present invention has the following configuration.

That is, a camera for photographing the head and face of an operator of a driving device, a gaze direction detecting means for detecting a gaze direction of the operator based on a video signal from the camera, and an image from the camera. A pupil diameter detecting unit that detects a value related to a pupil diameter of the operator based on the signal, a gaze direction detected by the gaze direction detecting unit, and a pupil diameter detected by the pupil diameter detecting unit. And control means for controlling the driving device.

For example, in a case where the control means controls the driving device mainly based on the detected line of sight, the control means may control whether the detected pupil diameter is smaller than a predetermined value. At this time, the control of the driving device based on the gaze direction detected by the gaze direction detection means may be restricted.

[0008] Further, for example, the driving device is a moving body, and further includes a traveling direction detecting means for detecting a traveling direction of the moving body, wherein the control means includes: a gaze direction detected by the gaze direction detecting means; An alarm that warns when the inattentiveness of the operator is detected based on the traveling direction detected by the traveling direction detection means and when the pupil diameter detected by the pupil diameter detection means is larger than the predetermined value. Means should be included.

Further, for example, the apparatus further comprises detecting means for detecting an operating environment and / or an operating state of the operating device, wherein the control means performs the operation when the detection result by the detecting means becomes a predetermined state. Information providing means for providing information about the driving environment and / or driving state to a driver, the information providing means comprising: a gaze direction detected by the gaze direction detecting means; When the inattentiveness of the operator is detected based on the direction, the degree of information provision is corrected to be larger as the pupil diameter detected by the pupil diameter detecting means is larger.

[0010]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of a control device for a driving device according to the present invention applied to an automobile as a typical driving device.

[First Embodiment] FIG. 1 is a diagram showing a system configuration of an automobile equipped with a control device of a driving device according to a first embodiment. FIG. 2 is a diagram showing a driver's seat of the automobile shown in FIG.

In FIGS. 1 and 2, reference numeral 1 denotes a control unit for performing control described later. Reference numeral 2 denotes a laser radar for detecting a distance from a preceding obstacle. Reference numeral 3 denotes an infrared camera for photographing the front. Reference numeral 4 denotes a magnetic marker sensor that detects magnetism from a magnetic marker provided on a road to detect a predetermined traveling position or the like. Reference numeral 5 denotes a line CCD (Charge Coupled Device) that is provided on, for example, a side mirror or the rear of the vehicle body and captures images of the side and rear of the vehicle. Reference numeral 6 denotes a driving lane detection CCD camera that photographs the front of the vehicle so as to detect the driving lane by a white line or the like on the road. Reference numeral 7 denotes a vehicle speed sensor that detects a traveling speed. 8
Is a yaw rate sensor that detects a yaw angle in order to detect G in the direction and the lateral direction of the vehicle. Reference numeral 9 denotes a throttle sensor for detecting a throttle opening. Reference numeral 10 denotes a brake switch for detecting an operation state of the brake. 11
Is a steering angle sensor that detects the steering angle of the steering wheel. Reference numeral 12 denotes a turn signal sensor for detecting an operation state of the turn signal switch. Reference numeral 13 denotes a driver monitor unit that detects the position of the driver's head and the point of gaze as described later. 14 is the map information and the received GP
Based on S (Global Positioning System) signal, etc.
It is a navigation unit that detects the current position of the vehicle and guides the route to the destination. Reference numeral 15 denotes a road-to-vehicle communication unit that communicates traffic information, information on an obstacle in front, information on the presence or absence of a pedestrian walking on a pedestrian crossing, and the like with a wireless device provided on a road. Reference numeral 21 denotes a display such as a liquid crystal display (LCD) or a head-up display (HUD) for providing an output result of the navigation unit 14 and various kinds of information. Reference numeral 22 denotes a speaker that outputs various sounds and provides information and alerts to inattentive driving described below. Reference numeral 23 denotes an infrared light projection lamp which is provided on, for example, a column cover of a steering wheel and emits infrared light to the head and face of the driver. And 24
Is an infrared projection area imaging camera that is provided on a column cover of a steering wheel, for example, and captures light reflected from the head and face by the infrared projection lamp 23.

Here, a functional configuration of control in the above-described system configuration will be described.

FIG. 3 is a block diagram showing the configuration of a control function in the control device of the operating device according to the first embodiment. In the present embodiment, each module described below is not shown in the control unit 1. Represents a functional unit of software executed by the CPU.

As shown in FIG. 1, the external environment judgment module of the control unit 1 includes a laser radar 2, an infrared camera 3, a magnetic marker sensor 4, a line CCD 5,
Using the output signals of the traveling lane detecting CCD camera 6, the detection result of the navigation unit 14, and the information obtained by the road-to-vehicle communication unit 15, sensing of the external environment of the vehicle is performed. Note that a general method is used for sensing the external environment using each of these output signals, and detailed description in this embodiment is omitted.
In the present embodiment, as an example, sensing of the external environment is performed using a plurality of types of detection signals as described above.
The sensing is not limited to this, and sensing may be performed by appropriately using some of the plurality of types of detection signals.

In the danger determination module of the control unit 1, the output signals of the vehicle speed sensor 7, the yaw rate sensor 8, the throttle sensor 9, the brake switch 10, the steering angle sensor 11, and the turn signal sensor 12 are used for sensing the vehicle state. , Road-to-vehicle communication unit 15
The dangerous state is determined using the obtained information. In this danger judgment module,
In other words, when it is determined that there is no room for danger avoidance by the driver, vehicle control such as automatic braking by braking operation, throttle operation, or shifting operation of an automatic transmission (AT), or automatic steering is performed without waiting for danger avoidance by the driver. Do. Note that a general method is used for the judgment of the danger state using these output signals and the vehicle control, and a detailed description in this embodiment is omitted.

In the driver status determination module of the control unit 1, the driver monitor unit 13
The driver's state is determined using the detection result of the driver's gaze point, head and face position, and pupil diameter detected in (2) (details will be described later).

The information provision / warning issuance determination module of the control unit 1 determines the necessity of information provision / warning issuance and the start timing based on the judgment result by the danger judgment module and the judgment result by the driver state judgment module. At the same time as the judgment is made, information display on the display 21 according to the judgment result and an alarm is issued by the speaker 22 are performed (details will be described later).

<Detection of Gaze Point> Next, a method of detecting the gaze point of the driver by the driver monitor unit 13 will be described.

FIG. 4 is a block diagram showing a system configuration of the driver monitor unit according to the first embodiment. Each module described below in the present embodiment includes a CP (not shown) provided in the driver monitor unit 13.
U represents a functional unit of software executed.

As shown in the figure, an infrared transmitting filter is provided in a light emitting portion of the infrared light emitting lamp 23 and a light receiving portion of the infrared light emitting area imaging camera 24. Extracts an image of the driver's head and face by performing general binarization processing and feature point extraction processing in an image processing module based on a video signal output from the infrared light projection area imaging camera 24. Then, based on the extracted image of the head and face, the gaze point detection module detects the head and face direction Dh, the gaze direction Ds, and the pupil diameter L in order to detect the gaze point of the driver.

Next, the function of the driver monitor unit 13 will be described with reference to FIGS.

FIG. 5 is a diagram showing a flowchart of the gazing point detection process in the first embodiment, and shows functions realized by the image processing module and the gazing point detection module.

In FIG. 5, step S51: The headlight portion of the driver is projected by the infrared light projection lamp 23, and the analog video signal of the head face portion photographed by the infrared light projection region imaging camera 24 is converted into an image. The video signal is captured by a processing module and subjected to general binarization processing to convert the video signal into digital multivalued image data for each pixel.

Step S52: A face image portion of the driver is extracted from the obtained multi-valued image data using a general image processing technique, and a plurality of feature points (for example, the inner and outer corners of the eye) included in the extracted face image portion are extracted. , Nostrils, etc.).

Step S53: From the extracted image data of the face image portion, the position of the reflection point and the position of the pupil generated on the cornea of the driver's eyeball by infrared projection are determined using a general image processing method. To detect.

Step S54: The maximum value of the number of pixels in the horizontal direction of the image data corresponding to the detected position of the pupil is detected as the pupil diameter L. Here, attention is paid to the horizontal direction because, when attention is paid to the vertical direction, an accurate pupil diameter cannot be detected due to the influence of “blinking” by the driver.

Step S55: By calculating the inclination of the driver's head face in a predetermined three-dimensional coordinate space based on the positions of the feature points detected in step S52, the direction in which the driver's head face is directed (head) Face direction) Ds is measured.

Step S56: The corneal reflection point detected in step S53 and the head-face direction D detected in step S55
s and the direction of the driver's line of sight (gaze direction) D
s is detected.

Step S57: On the basis of the gaze direction Ds detected in step S56 and predetermined positions inside and outside of the vehicle (front position, rear-view mirror mounting position, left and right side mirror mounting positions, etc.) stored in advance, the driver's Detect the fixation point.

<Control Processing by Control Unit 1> Next, control processing in the control unit 1 will be described.

FIG. 6 is a flowchart showing a control process by the control unit as a control device of the driving device according to the first embodiment. The external environment judgment module, the danger judgment module of the unit shown in FIG.
The functions realized by the driver status determination module and the information provision / warning alarm determination module are shown.

In the figure, step S1: laser radar 2, infrared camera 3, magnetic marker sensor 4, line C
Using the output signals of the CD 5 and the CCD camera 6 for detecting the traffic lane, the external environment of the vehicle is sensed by a general method.

Step S2: Navigation unit 1
According to 4 (information obtained by the road-to-vehicle communication unit 15 may be used), the current traveling position of the vehicle, the shape of the road on the course, and the like are detected by a general method.

Step S3: vehicle speed sensor 7, yaw rate sensor 8, throttle sensor 9, brake switch 1
0, the current vehicle state (straight running, turning state, etc.) is detected by a general method using the output signals of the steering angle sensor 11, and the turn signal sensor 12.

Step S4: Information about the road (radius of curve, speed regulation information, obstacle information, etc.) is obtained from the roadside equipment via the road-vehicle communication unit 15.

Step S5: A general method (for example, converting the detection result in each of the above steps into a numerical value (point) and taking the product or sum of the numerical values, etc.) based on the results detected in steps S1 to S4 By
It is determined whether there is a dangerous state (for example, a state in which a collision with another vehicle traveling ahead or a collision with a surrounding obstacle may occur). At this time, when the determination result satisfies a predetermined condition, it is determined that there is no room for danger avoidance by the driver, and automatic braking by a brake operation, a throttle operation, a shift operation of an automatic transmission (AT), or an automatic steering is performed. Perform vehicle control.

Step S6, Step S7: Step S
When it is determined from the determination result in 5 that there is room for danger avoidance by the driver, it is determined whether information provision / warning is necessary (step S6). If the determination result is NO (when unnecessary), Return, and if YES (if necessary), proceed to step S8 (step S8).
7).

Steps S8 and S9: The results of detection of the driver's head-face direction Dh, detection of the line-of-sight direction Ds, pupil diameter L, and gazing point are obtained from the driver monitor unit 13 described above with reference to FIG. (Step S
8) Based on the obtained point of gaze, it is determined whether or not the driver is currently looking ahead. When the result of the determination is YES (when looking forward), the process returns; when NO (when looking forward) If not, the process proceeds to step S10 (step S8).

In step S9, the line-of-sight direction Ds is obtained from the driver monitor unit 13, and a deviation from the vehicle traveling direction included in the vehicle state detected in step S3 is detected. May be performed when the size is larger than the size.

Step S10: The pupil diameter L obtained from the driver monitor unit 13 is equal to a predetermined reference value S and a coefficient K.
It is determined whether it is greater than the product of. This is a judgment that pays attention to the property that the pupil diameter of a person expands when looking at an object of interest.For example, it is determined whether there is an obstacle on the side of the vehicle before driving operation such as lane change. The determined pupil diameter of the driver changes greatly when the driver is gazing at the side mirror or the like. Here, an example of an experimental result regarding the movement of the head and the pupil of the driver will be described with reference to FIG.

FIG. 7 is a diagram showing the results of an experiment on the movement of the driver's head and pupil when changing lanes.

As shown in the figure, between 30 seconds and 15 seconds before the end of the merging to determine whether or not the lane can be changed, the driver's head faces horizontally in order to confirm the back of the vehicle many times. It can be seen that the pupil is sharply turned left and right and is gazing at the side mirror and the room mirror during the lane change, and the movement of the head face and the pupil with respect to the vertical direction. It can be seen that there is no significant change. Accordingly, it can be determined that the driving operation performed when the subject changes lanes does not include useless visual observation such as inattentiveness.

It is known that the pupil of a human generally changes in the range of about 2.2 mm to 13 mm, and the pupil diameter increases for an object of interest. That is, even when the gazing point of the driver is directed forward, when the pupil diameter is smaller than the predetermined reference value, it can be determined that the driver's attention to the driving operation is distracted, Even when the driver's gazing point is not pointed forward, the pupil diameter may be larger than a predetermined reference value by staring at a side mirror or the like to visually check the rear side. You cannot judge that you are looking aside.

Therefore, in the present embodiment, in order to determine the degree of concentration on the driving operation by using the size of the pupil diameter L as a determining factor, the driving environment is set so that the driver is less mentally burdened with the driving. The average pupil diameter of the driver experimentally obtained at the initial timing of the start is used as the predetermined reference value S. In addition, the coefficient K has the value of step S
2 or the turning radius R of the traveling lane detected in step S4
Is set in advance as a map (table) from which a value that changes according to is output. In other words, the driver has to concentrate on driving as the turning radius R is smaller, so that the driver's pupil diameter is generally larger. Therefore, in such a case, the coefficient K output from the map is set to be large (for example, one of the two light and dark characteristics shown in FIG. 11 described later in the second embodiment). ), Taking into account the mental burden on the driver's driving operation that changes according to changes in the external environment. Then, in step S10, it is determined whether or not the driver is looking aside by determining whether or not the pupil diameter L is larger than a product of a predetermined reference value S and a coefficient K. In this judgment,
When YES (when the pupil diameter L is larger than the calculated product)
Since it is highly probable that the driver is looking aside, it is necessary to immediately provide information and issue an alarm, and
The process proceeds to step S11, and if NO (when the pupil diameter L is smaller than the calculated product), the process proceeds to step S11.

Step S11: If the determination in step S12 is NO, it is unlikely that the driver is looking aside, and it cannot be determined that information provision / alarm issuance is necessary immediately. And returns to step S10 even if the predetermined time has elapsed.
If there is no change in the state of the driver, it is determined that the driver is looking aside, and the process proceeds to step S12.

Step S12: Providing the driver with a warning of inattentive driving and information to notify the driver of the state detected in step S7 (ie, information on an obstacle that is not an imminent danger state but should be reported to the driver).・
Change the start timing of alarm issuance earlier. That is, the start timing of the provision of information for displaying a predetermined item on the display 21 and / or the start timing of the alarm issuance by the speaker 22 are started earlier than the predetermined timing, or the volume of the alarm issuance by the speaker 22 is set to a predetermined value. Make it louder than the volume.

According to the above-described embodiment, when the driver is not looking forward, it is determined whether the driver is looking aside by using the detected pupil diameter L, and the driver looks aside. Sometimes, the mode of information provision / alarm issuance (alarm output timing, volume) is changed to a mode in which the driver is more noticeable. If it is not immediately determined that the driver is looking aside, the state continues for a predetermined period of time. The mode of providing / warning is changed. That is, the mode of information provision / alarm issuing can be changed quickly and accurately according to the driver's condition, and the preventive safety can be improved.

[Second Embodiment] Next, a second embodiment based on the control device of the operating device according to the above-described first embodiment will be described. In the following description, the same configuration as that of the first embodiment will not be described repeatedly,
The description will focus on the characteristic portions of the present embodiment.

In the present embodiment, the control unit 1 controls information provision and alarm issuance, paying attention to the fact that the size of the driver's pupil diameter changes according to the surrounding environment (driving environment). First, how the pupil diameter changes according to the surrounding environment will be described.

FIGS. 9 and 10 show how the pupil diameter L changes experimentally.

In FIG. 9, it can be seen that the pupil diameter (the number of pixels) of the subject is rapidly reduced in response to the sudden brightening of the surroundings from the measurement time 130 to 380. This situation corresponds to, for example, a case where light from the headlights of another vehicle traveling in the oncoming lane in a dark traveling environment enters the driver's eyes.

Further, in FIG. 10, it can be seen that the pupil diameter (the number of pixels) increases and changes sharply up and down because the subject is surprised after the measurement time 150. This situation corresponds to a case where the driver feels nervous or uneasy due to, for example, a sudden interruption of another vehicle ahead of the traveling lane.

Therefore, in the present embodiment, as an example of the surrounding driving environment, control of information provision and alarm issuance according to an external light / dark state is performed.

<Control Processing by Control Unit 1> Next, control processing in the control unit 1 will be described.

FIG. 8 is a flowchart showing a control process by a control unit as a control device of the driving device in the second embodiment.

In the figure, step S1A: The current time information is obtained from the navigation unit 14 or the like, and the surrounding light / dark state is determined based on the obtained time. That is, if the obtained time information is from 19:00 to 4:00, it is determined that the environment is a dark environment and if it is from 5:00 to 18:00, it is a bright environment. Further, it is preferable to obtain the date information together with the time information, and to consider the fluctuation of the light and dark state due to the season.

Steps S2 to S4: As in Steps S2 to S4 of FIG. 6 in the first embodiment, the traveling position, vehicle state, and road-to-vehicle information are obtained.

Step S5A: FIG. 1 which is stored in advance according to the light / dark state determined in step S1A and the turning R
The coefficient K is determined by referring to 1 or the map (table) in FIG.

FIGS. 11 and 12 are diagrams showing examples of the characteristics of the map stored in advance in the control unit as the control device of the driving device in the second embodiment.
Then, in each of the bright environment and the dark environment, as the turning radius increases, it is regarded as a substantially straight line R = 1000
12 shows a characteristic in which the coefficient K gradually decreases up to m. In FIG. 12, in each of the bright environment and the dark environment, as the turning radius increases, it is regarded as a substantially straight line.
The characteristic shows that the coefficient K decreases stepwise until 00m. That is, in the present embodiment, the driving operation is generally burdened in a dark environment and the pupil diameter is increased. Therefore, in such a dark environment, the threshold value of the inattentive judgment (the reference value S and the coefficient K 11 or FIG. 12 so that the coefficient K is larger than that in a bright environment.
Is set.

Although FIGS. 11 and 12 show two examples of a bright environment and a dark environment, a plurality of dusk environments and the like sandwiched between these two types of characteristics may be set.

Steps S8 to S12: Steps S8 to S12 in FIG. 6 in the first embodiment.
Similarly to the above, when the driver is not looking forward, the inattentive judgment is made in step S10 using the coefficient K representing the surrounding light / dark state, and information provision / warning is controlled according to the judgment result.

According to the second embodiment, the mode of providing information and issuing an alarm can be quickly and accurately changed to a mode in which the driver can easily notice, according to the surrounding driving environment. Preventive safety can be improved.

In the above-described second embodiment, the light and dark state is taken into consideration as an example of the surrounding driving environment. However, the present invention is not limited to this. For example, when the vehicle speed is high, when the weather is stormy, In the case of traffic congestion or the like in which many other vehicles are present in the surroundings, the mental load on the driving operation is high in any case, and the pupil diameter increases. Therefore, step S5A
In the map of FIG. 11 (FIG. 12) referred to in, the vehicle speed detected by the vehicle speed sensor 7, the operation state of the wiper switch, or the information which can be obtained by the road-vehicle communication unit 15 and / or the navigation unit 14 instead of the light-dark characteristics. A configuration may be adopted in which characteristics representing the state of other surrounding vehicles are stored, and the coefficient K is obtained from the map.

In each of the embodiments described above, the present invention is applied to an automobile. However, the present invention is not limited to this, and the control device of the operating device according to the present invention may be used for operating a moving object such as a train or an aircraft, or operating a plant or the like. The present invention can also be applied to a monitoring device, and in that case, various types of information obtained from the outside by the external environment determination module may be converted into sensor outputs according to the driving device to which the present invention is applied.

In each of the above-described embodiments, the function of each module performed by the control unit 1 has been described as software performed by the CPU provided in the unit. However, the present invention is not limited to this. Alternatively, dedicated hardware for realizing the functions of those modules may be employed.

[0067]

As described above, according to the present invention,
It is possible to provide a control device for an operating device that performs quick and accurate control according to the state of an operator.

That is, according to the first aspect of the present invention, quick and accurate control can be performed in consideration of the line of sight of the operator and the pupil diameter of the operator.

According to the second aspect of the present invention, when the pupil diameter is smaller than the predetermined value, it is possible to assume that the operator is less interested in the direction of the line of sight at that time. An adverse effect on performing control based on the direction can be suppressed.

According to the third aspect of the present invention, when it is determined that the operator of a moving body such as a car (claim 4) is looking aside, an alarm can be output immediately, and preventive safety can be achieved. be able to.

According to the fifth and thirteenth aspects of the invention, for example, when the turning radius of the road on which the vehicle is traveling is small (claims six and fourteen), the mental burden on the driving of the vehicle is reduced. It is possible to perform an accurate control in consideration of the pupil diameter of the operator who performs the conversion in response.

According to the seventh and fifteenth aspects of the present invention, for example, an operator who converts in accordance with the driving environment of the vehicle, such as the lightness or darkness of the road on which the vehicle is traveling (claims eight and sixteen). Precise control can be performed in consideration of the pupil diameter.

Further, according to the ninth aspect of the present invention, when the operator is looking aside, accurate information provision is performed, for example, the information provision timing is advanced according to the pupil diameter (claim 10). be able to.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a system configuration of an automobile equipped with a control device of a driving device according to a first embodiment.

FIG. 2 is a view showing a driver seat of the automobile shown in FIG.

FIG. 3 is a block diagram illustrating a configuration of a control function in a control device of the driving device according to the first embodiment.

FIG. 4 is a block diagram illustrating a system configuration of a driver monitor unit according to the first embodiment.

FIG. 5 is a diagram illustrating a flowchart of a gazing point detection process in the first embodiment.

FIG. 6 is a flowchart illustrating a control process by a control unit as a control device of the driving device according to the first embodiment.

FIG. 7 is a diagram showing experimental results regarding the movement of the driver's head and face and the movement of the pupil when changing lanes.

FIG. 8 is a flowchart illustrating a control process by a control unit as a control device of the driving device according to the second embodiment.

FIG. 9 is a diagram showing how the pupil diameter L is experimentally determined.

FIG. 10 is a diagram showing how the pupil diameter L is experimentally determined.

FIG. 11 is a diagram illustrating a characteristic example of a map stored in advance in a control unit as a control device of the driving device according to the second embodiment.

FIG. 12 is a diagram illustrating a characteristic example of a map stored in advance in a control unit as a control device of the driving device according to the second embodiment.

[Description of Signs] 1: Control unit, 2: Laser radar, 3: Infrared camera, 4: Magnetic marker sensor, 5: Line CCD, 6: CCD camera for detecting traffic lane, 7: Vehicle speed sensor, 8: Yaw rate sensor, 9: throttle sensor, 10: brake switch, 11: steering angle sensor, 12: turn signal sensor, 13: driver monitor unit, 14: navigation unit, 15: road-to-vehicle communication unit, 21: display, 22: speaker, 23: red External floodlight lamp, 24: infrared projection area imaging camera,

Claims (16)

[Claims] A camera for photographing the head and face of an operator of a driving device; a gaze direction detecting means for detecting a gaze direction of the operator based on a video signal from the camera; and an image from the camera. A pupil diameter detection unit that detects a value related to a pupil diameter of the operator based on the signal; a gaze direction detected by the gaze direction detection unit; and a pupil diameter detected by the pupil diameter detection unit. And a control unit for controlling the driving device. 2. The method according to claim 1, wherein the control unit controls the driving device mainly based on the detected line-of-sight direction. When the detected pupil diameter is smaller than a predetermined value, The control device for an operating device according to claim 1, wherein the control of the operating device based on the visual line direction detected by the visual line direction detection unit at a time is regulated. 3. The driving device is a moving body, further comprising a traveling direction detecting means for detecting a traveling direction of the moving body, the control means comprising: a gaze direction detected by the gaze direction detecting means; An alarm that warns when the inattentiveness of the operator is detected based on the traveling direction detected by the traveling direction detection means and when the pupil diameter detected by the pupil diameter detection means is larger than the predetermined value. 3. The control device for an operating device according to claim 2, further comprising means. 4. The control device according to claim 3, wherein the moving body is a vehicle. 5. The vehicle according to claim 1, further comprising a traveling state detection unit configured to detect a traveling state of the vehicle, wherein the control unit changes the predetermined value according to the traveling state detected by the traveling state detection unit. The control device for a driving device according to claim 4, characterized in that: 6. The control of the driving device according to claim 5, wherein the control unit changes the predetermined value to a small value when the control unit determines that the vehicle is turning based on the detected traveling state. apparatus. 7. An operating environment detecting means for detecting an operating environment of the operating device, wherein the control means changes the predetermined value according to the operating environment detected by the operating environment detecting means. The control device for an operating device according to claim 3, wherein: 8. The control device according to claim 7, wherein the control unit changes the predetermined value greatly when the detected driving environment is a dark environment. 9. The control device further includes detection means for detecting an operation environment and / or an operation state of the operation device, wherein the control means controls the operator when a result of the detection by the detection means becomes a predetermined state. Information providing means for providing information on the driving environment and / or driving state to the vehicle, the information providing means comprising: a gaze direction detected by the gaze direction detecting means; and a traveling direction detected by the traveling direction detecting means. 3. The driving device according to claim 2, wherein when the inattentiveness of the operator is detected based on the pupil diameter, the degree of information provision is corrected to be larger as the pupil diameter detected by the pupil diameter detecting means is larger. Control device. 10. The driving according to claim 9, wherein the step of largely correcting the degree of information provision is to speed up the timing of information provision, increase the number of times, or intensify the mode of information provision. Equipment control device. 11. The driving device is a moving body, further comprising a traveling direction detecting means for detecting a traveling direction of the moving body, wherein the control means comprises: a gaze direction detected by the gaze direction detecting means; 10. The system according to claim 9, wherein when the traveling direction detected by the traveling direction detection unit does not coincide at least, the operator is determined to be looking aside.
A control device for the operating device according to the above. 12. The control device according to claim 11, wherein the moving body is a vehicle. 13. The method according to claim 1, wherein the detecting unit detects the traveling state of the vehicle, wherein the control unit changes the predetermined value according to the traveling state detected by the detecting unit. Item 13. A control device for an operating device according to item 12. 14. When the control means determines that the vehicle is turning based on the detected traveling state,
2. The method according to claim 1, wherein the predetermined value is changed to a small value.
4. The control device of the operating device according to 3. 15. When the operating environment of the operating device is detected by the detecting unit, the control unit changes the predetermined value according to the operating environment detected by the detecting unit. A control device for a driving device according to claim 9. 16. The control device according to claim 15, wherein the control unit changes the predetermined value greatly when the detected driving environment is a dark environment.