JP2005108033A - Driver condition determination device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and a method for stably determining conditions of a driver from an image of the driver. <P>SOLUTION: This driver condition determination device is provided with an image input part 101 acquiring a time series image, a face detection part 102 detecting a face from the input image, a face area determination part 104 determining whether the detected face position is a safe position or not, a driver condition determination part 106 determining the conditions of the driver from the result of the face area determination part, and a result output part 107 outputting the result of the driver condition determination part to the driver. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention detects a dangerous state such as falling asleep, looking away, or looking aside by analyzing a driver's state in an image taken by a light receiving device such as a TV camera attached to a moving body such as an automobile. In addition, the present invention relates to a driver state analysis apparatus and a driver state analysis method that perform a warning to a driver and control of a moving body.

  As a method of determining the state of the driver, such as a driver's sleep falling asleep by using an image sensor such as a camera, image analysis of the person's eyes from the image of the driver's face captured by the camera However, a technique (Patent Document 1) for determining that the eyelid is asleep when the eyelid is closed has been studied.

  However, there are individual differences in the eyes of the human subject, and in the case of a person with thin eyes or a person with long eyelashes, the open / closed state of the eyelids cannot always be accurately determined. In addition, if you fall asleep, your face may be in a state of depression, and if your eyes are hidden when you are depressed, you cannot check the open / closed state of the eyelids with the camera image, There is a problem that the eye state of the subject cannot be analyzed.

Further, as a method for monitoring the state of a subject by image processing, a method of determining whether the driver is asleep by detecting the driver's blink using a camera and examining the number and cycle thereof (Patent Document 2). There is also. However, as in the above-described method, it is difficult to accurately detect the individual difference of eyes and the open / closed state, and the blinking cycle varies depending on the individual, so it is difficult to accurately determine the dozing state.
JP 2002-279410 A Japanese Patent Application Laid-Open No. 08-225028

  As explained above, in order to detect the driver's state such as the driver's falling asleep, it is effective to take and analyze images using a non-contact camera with less burden on the driver. There is a demand for a method for accurately determining dozing without relying only on eyes that are difficult to determine the difference or state.

  The present invention detects a driver's face position using a camera and pays attention to the movement of the face position, thereby enabling analysis of the driver's state without being influenced by individual differences. An object of the present invention is to provide a state discriminating apparatus and method.

  The present invention provides a face detection unit that detects a feature point or face area of a driver's face from an input time-series image, and a position of the detected face feature point or face area at an appropriate position in the image. A face area determination unit that determines whether there is a driver state determination unit that determines a driver's state from a result of the face area determination unit or a result of the face detection unit (when a face cannot be detected by looking aside), and driving There is provided a driver situation determination apparatus characterized by having a result output unit for outputting the driver situation determination result to notify the driver.

  Further, in the present invention, a driver recognition unit that identifies whether the detected face belongs to the driver himself or herself, and the driver situation determination unit determines the state of the driver based on the recognition result. It is characterized by.

  In the present invention, the driver situation determination unit may determine the state of the driver based on information related to the state of the aircraft that the driver is driving.

  Moreover, in this invention, in order to notify to third parties other than a driver | operator, it has an external output part which outputs the said driver | operator condition determination result, It is characterized by the above-mentioned.

  Moreover, in this invention, it has an airframe control part which controls an airframe according to the result of the said driver | operator condition determination part, It is characterized by the above-mentioned.

  In addition, the present invention provides a face detection unit that detects a feature point or face area of a driver's face from an input time-series image, and the position of the detected face feature point or face area is an appropriate position in the image. A face area determining means for determining whether the driver is in a vehicle, a driver situation determining means for determining a driver's state from the result of the face area determining means or the result of the face detecting means (when the face cannot be detected by looking aside), There is provided a driver situation determination method, characterized by comprising a result output means for outputting the driver situation determination result to notify the driver.

  According to the present invention, a feature point, a face area, and the like of a driver's face detected from a camera image are detected, and the detected position, detection state, and the like are observed, so that it is stable by paying attention to the entire face. The driver's condition can be detected.

  Hereinafter, an embodiment relating to the present invention will be described with reference to the drawings.

  In this embodiment, a driver's image is taken from a camera installed on a moving body such as an automobile, a train, an aircraft, and a ship, and whether or not the face is in an appropriate region is analyzed. The driver's face is detected and the state is determined to determine whether the driver is falling asleep, looking away, or looking aside.

  FIG. 1 is a diagram showing the configuration of this embodiment. The image input unit 101 is an imaging device that captures a driver and is, for example, a camera that can capture a moving image or a still image. This camera may be a TV camera or a video camera that captures normal visible light, a special camera that can capture images other than visible light such as infrared light, or a range finder that acquires three-dimensional information of an object. The camera may have a pan / tilt zoom mechanism or a mechanism for moving the camera itself by a motor or the like.

  The face detection unit 102 detects a face region of a person who is a subject from an image taken by the image input unit. For example, Fukui, Yamaguchi, `` Face feature point extraction by combination of shape extraction and pattern matching '', Science theory (D-II) vol. There are those described in J80-D-II, No. 9, p.2170-2177, 1997. This is because facial feature points such as eyes (pupil) and nose (nasal nostrils) are extracted by using both geometric features by the separability filter and the shape of the pattern. This is a technique that can stably detect a face region. With this technique, for example, the face detection unit outputs position information of four points of eyes and nose from an image including a face. In addition, the accuracy of the face information can be increased by outputting other information such as the position of the mouth edge and the eyebrows and the contour information.

  The face area determination unit 104 analyzes the situation of the driver based on the position information of the face area output from the face detection unit. First, as shown in FIG. 2, for example, a face image detection region, that is, a range in which the driver can safely drive even if the driver's face is present during the driving of the moving body (in the subject). A range that is usually suitable for capturing a face image of a driver is set. This range may be an arbitrary shape such as a rectangle or a circle in the image. In FIG. 2, for example, a circular portion in the center of the screen, but any number may exist at any location.

  Then, as shown in FIG. 3, when the driver is photographed with a camera and the face area of the photographed driver image is within this range (for example, the circular area in FIG. 2), a temporary determination is made. The driver determines that the state is normal. On the contrary, as shown in FIG. 4, when the detected driver's face area is out of this range (for example, the circular area in FIG. 2), it is assumed that the driver's position is not in a place suitable for the operation of the moving body. It is determined that the person is not in a normal state or in an abnormal state. On the other hand, when the driver's image is taken so as to be included in the above range, but the driver's face area could not be detected from the obtained image due to reasons such as looking aside Similarly (see FIG. 5), the driver determines that the driver is not in a normal state. 2 and 3 and the like, the driver's face area detection result is expressed as a circular area, but may be any shape such as a rectangle or a circle. Further, it may be represented by point information such as coordinates of the center point of a feature point such as an eye or nose.

  The driver recognition unit 103 determines whether or not the face pattern existing in the region output from the face detection unit matches the face pattern of the driver himself (regular driving or registered in advance). Is for comparing the face image and facial feature point information acquired in advance with the currently acquired face image and facial feature point information to identify the driver. For example, as shown in FIG. 6, when a face other than the driver such as a passenger is present in the photographing range of the camera, it is possible that not only the driver but also the face other than the driver is detected. In that case, since there is a possibility that an incorrect result is output in the driving situation determination, a mechanism for identifying the driver is necessary.

  For example, Yamaguchi, Fukui, “Robust Face Recognition System 'smartface' for Facial Expression Changes,” IEICE (D-II), vol.J84-D-II. , No.6, p.1045-1052, 2001. This is a method of creating a distribution that represents individual facial features from a plurality of face patterns obtained from a plurality of still images constituting a moving image, and comparing the distributions. It can be recognized with higher accuracy than the based method. Whether the driver is a driver by examining the condition that the position where the subject's face area was detected is in the vicinity of the driver's seat in the camera frame and the face size is greater than a certain level, without needing face recognition. You may check it.

  The body information output unit 105 outputs information on the state of a moving body such as an automobile that is currently driving. The information to be output includes, for example, whether the engine is moving, how much speed and acceleration it is moving, whether the brake is stepped on, the inclination of the body, the direction of the steering wheel, and the traveling direction. As these information, data measured by a mobile body such as an automobile may be output, or data measured by various sensors provided separately from the mobile body may be output.

  The driver situation determination unit 106 determines the situation of the driver according to information output from the face area determination unit 104 and information output from the driver recognition unit 103 and aircraft information output unit 105 as necessary. is there. For example, the face area determination unit 104 obtains an output indicating that the captured face area does not exist within the safe range of the imaging area for a certain period of time or more than a certain ratio during a certain period of time. If it is determined (for example, 50 seconds or more in one minute), the driver determines that the state is not normal and outputs an abnormal state. The time interval until it is determined that this is an abnormal state may be determined in advance, or may be set as required by the user to limit the interval to suit the user, thereby reducing false detections. it can.

  Further, if the driver recognition unit 103 and the aircraft information output unit 105 exist, the information can be combined to make a more detailed determination. First, if the driver recognition unit 103 is present and the face area determination unit 104 detects the face area, the photographed subject's face cannot be identified as the driver's face. Is not output, or the face area is determined in more detail, and the recognition / collation with the driver is performed, so that the detection accuracy can be improved.

  Also, when the aircraft information output unit 105 is present, when the aircraft (moving body) is not moving at all, even if it is in an abnormal state, “not normal” is not output, and conversely, the aircraft is moving at high speed Because it is necessary to issue a warning earlier than usual, immediately after detecting an abnormal condition, it will output “not normal” or if it is determined to be abnormal, it will change the course of the aircraft. Furthermore, by performing a process such as issuing a higher-level warning, it can be expected to make a more appropriate judgment based on the situation with the actual aircraft.

  The result output unit 107 gives an output to the driver in response to a request from the driver situation determination unit 106. If the determination that the driver situation determination unit 106 is in an abnormal state or is not in a normal state continues for a long time longer than a predetermined time, a warning is output to the driver. The output may be audio information such as an alarm sound, visual information such as a monitor display or a lamp, a tactile sensation to the driver, such as giving a vibration or lightly tightening the seat belt, A scent may be emitted to stimulate the sense of smell, or a combination thereof. The external output unit 108 outputs the result of the driver situation determination unit 106 so that it can be detected by a person outside the moving body that is driving. When an abnormality is detected for a driver of a car or the like, by outputting it to the outside of the car, the surrounding people who see it can pay attention and take some measures in advance. It is thought that the surrounding safety can be secured. The output can be a visual signal such as a warning lamp or flash, or a sound such as a siren. In the event of an emergency, such as when the driver continues to be in an abnormal state for a certain period of time, use a device that can communicate with the outside, such as a phone mounted on the vehicle, a wireless device, or the driver's mobile phone. Then, you can automatically contact the police, fire department, security company, etc. to prevent accidents and to take quick measures in the event of an accident.

  The body control unit 109 directly controls mechanisms such as the engine, steering wheel, accelerator, and brake of the body according to the result of the driver situation determination unit 106. For example, when the driver situation determination unit 106 detects an abnormal state of the driver, the airframe control unit 109 performs control so that the speed of the airframe is not further increased, and if the abnormal state continues for a long time, step by step. It is thought that accidents can be prevented by slowing down and finally stopping. If the safe location is known in advance, the aircraft may be moved there. When controlling the aircraft, it is also possible to present some signal to the outside in cooperation with the external output unit, such as a hazard lamp in the case of an automobile.

  In addition, the position of the driver on the aircraft is usually determined as a driver's seat. The body information output unit 105 can physically check whether the driver is in the driver's seat using a pressure-sensitive sensor installed in the driver's seat, and can output the information. In addition to the pressure sensor, any sensor such as a sensor using infrared rays can be used as long as it can detect that a person is sitting. When the driver is absent, it can be expected that the driver status determination unit 106 does not check the driver status and suppresses unnecessary warnings. Also, if the aircraft is moving without the driver, the driver status determination unit 106 can send an output that warns.

  Depending on the aircraft, the driver's seat may not be fixed. In that case, a sensor that can specify the position, such as a pressure sensor, is installed on the floor, and the position of the driver is physically checked. The obtained position information is output by the body information output unit 105. The driver situation determination unit 106 does not give an unnecessary warning when the driver is absent.

  The obtained driver position information can also be sent to the image input unit 101. For example, in the case of a camera having a pan / tilt function or the like, pan / tilt control is performed using the position information of the driver so that the driver's face is properly within the shooting range of the camera currently shooting. Further, when a plurality of cameras are used, an image of the camera closest to the driver's position can be used so that the driver's face can be captured more appropriately. In addition, even when the camera itself does not move or a plurality of cameras are not switched, the position of the driver can be limited in the image taken from the position information of the driver. Thus, by limiting the image processing range based on the positional relationship between the camera and the driver, it is possible to reduce face misdetection and the required processing amount.

The block diagram which shows the structure of one Embodiment of this invention The figure of the area where the face of the face area judgment part exists safely Illustration when driver's face is in normal position Figure with driver's face outside the safe range Figure where the driver failed to detect the face sideways The figure for explaining the case where other than the driver is detected

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Image input part 102 ... Face detection part 103 ... Driver recognition part 104 ... Face area | region determination part 105 ... Airframe information output part 106 ... Driver situation determination part 107 ... Result output part 108 ... External output part 109 ... Airframe control part

Claims (6)

  A face detection unit that detects a feature point or face area of the driver's face from the input time-series image, and determines whether the detected face feature point or face area position is an appropriate position in the image A face area determination unit; a driver condition determination unit that determines a driver's state from a result of the face area determination unit or a result of the face detection unit; and a result output that outputs the driver condition determination result to notify the driver A driver situation determination device characterized by having a section.   A driver recognition unit that identifies whether the detected face belongs to the driver or not in comparison with the previously acquired face image or facial feature point information, and the driver situation determination unit The driver state determination apparatus according to claim 1, wherein the state of the driver is determined based on the result.   3. The driver status determination apparatus according to claim 1, wherein the driver status determination unit determines the status of the driver based on information related to a status of a body that the driver is driving.   4. The driver situation determination apparatus according to claim 1, further comprising an external output unit configured to output the driver situation determination result in order to notify a third party other than the driver.   5. The driver situation determination apparatus according to claim 1, further comprising a machine body control unit that controls the machine body according to a result of the driver situation determination unit.   Face detection means for detecting a feature point or face area of the driver's face from the input time-series image, and determining whether the detected face feature point or face area position is an appropriate position in the image A face area determination means; a driver status determination means for determining a driver's state from the result of the face area determination means or the result of the face detection means; and a result output for outputting the driver status determination result to notify the driver A driver status determination method characterized by comprising means.

Images