JP2011115450A - Apparatus, method, and program for determining aimless state - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To determine an aimless state of a determined object by easy processing with stable accuracy. <P>SOLUTION: The following processes are repeated; acquiring an image obtained by imaging a driver's face (52), computing an eye rotation angle (54), determining whether a saccade occurs or not by comparing a vector size T of the eye rotation speed with a predetermined value T<SB>θ</SB>(60), and computing and memorizing the amplitude and the direction of the detected saccade (74) while computing and memorizing (76, 78) the line position of the driver's sight. When the determination period of the aimless state arrives (a process 80 is yes), it determines in order of the following processes; the ratio of saccade with the amplitude being the predetermined value A or below is larger than the threshold Ta or not (84), and the ratio of saccade within the area where the direction is classified in the longitudinal direction is smaller than the threshold Td or not (88), and the numbers of positions where the line positions of sight gather are two or above or not (92). If any of the determinations are affirmed, the aimless state is determined and an alarm output (94) and the like is performed. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a casual state determination device, method, and program, and more particularly, a casual state determination device that determines whether or not a determination target person is in a random state, a casual state determination method that can be applied to the casual state determination device, and a computer The present invention relates to a vague state determination program for functioning as a vague state determination device.

  A technique for detecting the driver's line-of-sight direction of a vehicle and determining whether or not the driver is driving indiscriminately from the detection result of the line-of-sight direction has been proposed. Detect the gaze direction and determine the gaze frequency in the vehicle front area and the vehicle surrounding area.For example, when the gaze frequency in the vehicle front area is greater than a predetermined value and the host vehicle does not follow the preceding vehicle, A technique has been proposed in which the driver determines that the gaze frequency is greater than a predetermined value and the driver is in a loose state when there is no other vehicle in the adjacent lane.

  Further, Patent Document 2 detects a driver's gaze behavior, and sets an abrupt state determination region based on a three-dimensional object such as a guardrail or a sign beside the preceding vehicle with respect to an image obtained by imaging the front outside the vehicle. However, there is disclosed a technique for determining that the state of ambiguity is determined when the ratio of the staying time of the line of sight within the sloppy state determination region to the reference stagnation time is equal to or greater than a threshold value.

  Further, in Patent Document 3, the driver's line-of-sight behavior is detected and its variance value is calculated, and the ratio of the width of the preceding vehicle in the calculated variance value is calculated as the attention level evaluation value. Is smaller than the above, a technique for determining that the attention state of driving with respect to the preceding vehicle is a low-pitched state is disclosed.

JP 2003-80969 A JP 2007-73011 A JP 2008-137467 A

  Since the determination target person's attention state is reflected in the determination target person's gaze transition and distribution, the determination target person's casual state can also be determined from the determination target person's gaze transition and distribution. Also in the techniques described in? 3, the gaze direction and position of the driver as the determination target are detected, and the loose state is determined based on the gaze frequency of the specific area, the gaze retention, and the gaze distribution.

  However, the detection of the line of sight is performed by converting the rotation angle of the eyeball of the determination target person into a position coordinate on an image representing the front of the vehicle. However, depending on the orientation of the face of the determination target person, A large error may be added to the conversion, resulting in a state in which the determination target person cannot be determined. In addition, as in the techniques described in Patent Documents 2 and 3, when detecting a rough state based on a three-dimensional object such as a guardrail or a sign in front of the vehicle, or on a preceding vehicle, the situation in front of the vehicle is imaged. The complicated image processing of extracting an image region estimated to correspond to the above-described three-dimensional object or preceding vehicle from the obtained image is necessary, and there is also a problem that the device configuration for determining the random state becomes complicated Join.

  The present invention has been made in consideration of the above facts, and it is possible to obtain a sloppy state determination device, a sloppy state determination method, and a sloppy state determination program that can determine a sloppy state of a person to be determined with stable accuracy by simple processing. Is the purpose.

  When a person observes the surroundings, the human eyeball performs a saccade, which is a fast and linear eye movement from one gaze stop point to another gaze stop point. It has been known for a long time. The inventors of the present application pay attention to this saccade, and confirm whether there is a difference in the characteristics of saccades between humans in normal conditions and humans in random conditions, and if there are differences, what characteristics are different An experiment was conducted. In this experiment, as shown in FIG. 1 as an example, the distribution of the frequency of occurrence of saccades generated in a certain period for each amplitude is clearly different between a normal human and a human in a casual state. In the case of the state, it was found that the occurrence frequency of the small amplitude saccade is higher than that in the normal state. In addition, as shown in FIG. 2 as an example, the distribution of the occurrence frequency for each saccade direction occurring within a certain period is clearly different between the normal person and the human being in the loose state. It was also found that the frequency of occurrence of saccades in the direction including the vertical direction is lower in the illicit state than in the normal state.

  Based on the above, the random state determination device according to the first aspect of the present invention includes an eye movement measurement unit that measures the movement of the eyeball of the determination target person, and the eyeball based on a measurement result of the eyeball movement by the eye movement measurement unit. A saccade extracting means for extracting saccades in the exercise, a computing means for computing at least one of amplitude and direction in the saccades extracted by the saccade extracting means, and the saccade extracting means within an arbitrary period. Of the extracted saccades, the ratio of saccades whose amplitude calculated by the calculation means is not more than a first predetermined value is not less than a second predetermined value, and is extracted by the saccade extraction means within the period. Of the saccades, the ratio of the saccades within the range in which the direction calculated by the calculation means includes the vertical direction or the vertical direction is the third. Determination means for determining at least one of the conditions of whether or not the value is equal to or less than a predetermined value and determining that the determination target person is in a sloppy state when at least one of the determined conditions is satisfied. ing.

  In the first aspect of the present invention, the eye movement of the person to be determined is measured by the eye movement measuring means, and the saccade in the eye movement is detected by the saccade extracting means based on the measurement result of the eye movement by the eye movement measuring means. At least one of the amplitude and direction in the saccade extracted and extracted by the saccade extraction means is calculated by the calculation means. The determining means according to the invention of claim 1 is characterized in that a ratio of saccades having an amplitude calculated by the calculating means out of the saccades extracted by the saccade extracting means within an arbitrary period is equal to or less than a first predetermined value. Whether or not the second predetermined value or more, and the ratio of saccades within a range in which the direction calculated by the calculation means out of the saccades extracted by the saccade extraction means during the period is vertical or includes the vertical direction It is determined whether at least one of the three conditions is less than or equal to the third predetermined value, and the determination target person is determined to be in a loose state when at least one of the determined conditions is satisfied.

  As described above, in the invention according to claim 1, when determining whether or not the determination target person is in a state of ambiguity, the gaze position of the determination target person whose accuracy is greatly reduced depending on the orientation of the determination target person's face is required. Characteristics of the saccade that can be directly detected from an image obtained by imaging the eye of the person to be judged (the ratio of the saccade whose amplitude is equal to or less than the first predetermined value and the direction is vertical or vertical) Is determined based on at least one of the ratios of saccades within a range including It is possible to determine the casual state of the person to be determined. Also, an image region that is estimated to correspond to a three-dimensional object or a preceding vehicle from a process that converts the eyeball movement (rotation angle) of the determination target person into the line-of-sight direction or line-of-sight position of the determination target person or an image that represents the situation in front of the vehicle Therefore, it is not necessary to perform image processing such as extraction, so that the casual state of the person to be determined can be determined by simpler processing.

  In the invention described in claim 1, for example, as described in claim 2, the calculating means calculates the amplitude and direction in the saccade extracted by the saccade extracting means, and the determining means is the saccade extracting means. Among the saccades extracted by the calculation means, whether or not the ratio of saccades whose amplitude calculated by the calculation means is equal to or greater than a second predetermined value, and among the saccades extracted by the saccade extraction means Each condition is determined whether or not the saccade ratio within the range including the vertical direction or the direction calculated by the calculation means is equal to or less than a third predetermined value, and at least one of the conditions is satisfied In such a case, it is preferable that the determination target person determine that the person to be determined is in an ambiguous state.

  In the invention of claim 2, whether the ratio of the saccade whose amplitude is equal to or less than the first predetermined value is equal to or greater than the second predetermined value, and the saccade extracted by the saccade extraction means is calculated by the calculation means. Since the state is determined using both conditions of whether the direction is the vertical direction or the ratio of saccades within the range including the vertical direction is the third predetermined value or less, only one of the two conditions is used. Although the process is complicated compared to the case of determining a sloppy state, using both conditions makes it difficult to determine a sloppy state by the other condition even if it is not possible to determine a sloppy state by the influence of disturbances such as noise. Since there is a possibility that it can be determined as a state, it is possible to improve the stability of the determination accuracy of the random state.

  Further, in the invention according to claim 1 or 2, the saccade extraction means, as described in claim 3, for example, changes the rotation speed of the eyeball based on the measurement result of the eyeball movement by the eyeball movement measurement means. It is possible to calculate and extract the movement of the eyeball as a saccade during a period in which the rotation speed of the eyeball has maintained the fourth predetermined value or more.

  Further, in the invention according to any one of claims 1 to 3, for example, as described in claim 4, the determination target person is an operator who operates the device, and the determination target person operates the device. The ratio of saccades whose amplitude calculated by the calculation means is less than or equal to the first predetermined value out of the saccades extracted by the saccade extraction means during the initial period of starting the saccade, and the saccades extracted by the saccade extraction means The parameter corresponding to the determination performed by the determination unit is obtained from the ratio of the saccades within the range in which the direction calculated by the calculation unit is vertical or includes the vertical direction, and the calculated parameter is set to the second predetermined value or the second 3 It is preferable to provide setting means for setting as a predetermined value.

  The second predetermined value and the third predetermined value used by the determination unit according to the present invention for determining the sloppy state may have different optimum values for each individual determination target person. On the other hand, in the invention according to claim 3, when the determination target person is an operator who operates the device, the determination target person generally operates the device during the initial period when the determination target person starts operating the device. The parameter determined during the initial period when the determination target person started operating the device based on the fact that it is highly likely that the determination target person is not in a random state is the second predetermined value or the third Since it is set as the predetermined value, it is possible to set the optimum value for each determination target person as the second predetermined value or the third predetermined value, and it is possible to further improve the determination accuracy of the random state.

  The inventors of the present application also conducted an experiment to confirm how the distribution of the gaze position of a person differs depending on whether the person is in a normal state or a vague state. Conventionally, it is known that a human gaze position concentrates on a place where the human gazes, and the techniques described in Patent Documents 1 to 3 described above concentrate on a human (driver) gaze position. In the above experiment, the distribution of the gaze position distribution according to whether the human is in the normal state or the loose state is determined. As another difference, as shown in FIG. 3 (A) as an example, when a human is in a normal state, the human gaze position is concentrated at a single location, while as shown in FIG. 3 (B) as an example, It has been newly found that there is a difference that a plurality of places where the human gaze position is concentrated appear in the case of the state of ambiguity.

  Based on the above, in the invention according to any one of claims 1 to 4, for example, as described in claim 5, the face imaging unit that images the face of the person to be determined and the imaging by the face imaging unit The direction of the determination target person's face is determined based on the obtained image, and the line-of-sight position of the determination target person is calculated based on the measurement result of the eye movement by the eye movement measuring unit and the determined determination target person's face direction. A line-of-sight position calculating unit that repeats this, and the determining unit includes, as one of the conditions, a plurality of locations where the line-of-sight position calculated by the line-of-sight position calculating unit is concentrated within an arbitrary period. It is preferable to make a determination so that the person to be determined is determined to be in a random state when at least one of the conditions including the condition is satisfied.

  As described above, when the determination target person's random state is determined based on the line-of-sight position of the determination target person, as in the prior art, there is a large error in conversion to position coordinates depending on the orientation of the determination target person's face and the like. In addition, there is a possibility that the determination target person's sloppy state cannot be determined. However, in the invention according to claim 5, the saccade amplitude distribution and saccade direction distribution are determined based on the gaze position determination. Since it is used together with the determination of the sloppy state based on at least one, it is possible to prevent the determination target person from falling into a state where the sloppy state cannot be determined. In the invention of claim 5, the determination of the random state based on the line-of-sight position is used in combination with the determination of the random state based on at least one of the amplitude distribution of the saccade and the direction distribution of the saccade. In the case where the orientation is suitable for the determination of the random state based on the line-of-sight position, the determination accuracy of the determination target person can be improved.

  In addition, in the invention described in claim 5, for example, as described in claim 6, the line-of-sight position calculating means is configured to determine the line of sight of the determination target person based on the measurement result of the eyeball movement and the determined face direction of the determination target person. It is determined by calculating the direction and repeatedly calculating the position of the gaze direction of the determination target person on the image representing the space gazeed by the determination target person as the gaze position of the determination target person based on the calculated gaze direction In the image, the pixel corresponding to the line-of-sight position calculated by the line-of-sight position calculating unit within an arbitrary period or the frequency calculated as the line-of-sight position by the line-of-sight position calculating unit within an arbitrary period is a predetermined value or more. An area surrounded by pixels that are composed only of pixels and do not correspond to the line-of-sight position, or pixels whose frequency calculated as the line-of-sight position by the line-of-sight position calculating means within an arbitrary period is less than a predetermined value. The can be configured to recognize a position where the line of sight position is concentrated.

  According to a seventh aspect of the present invention, there is provided a method for determining a random state, which measures a movement of an eyeball of a person to be judged, extracts a saccade in the movement of the eyeball based on a measurement result of the eyeball movement, and extracts the extracted saccade Calculating at least one of the amplitude and the direction in the saccade extracted within an arbitrary period, whether or not the ratio of the saccade whose calculated amplitude is not more than a first predetermined value is not less than a second predetermined value, and Determining at least one of the conditions of whether or not the ratio of the saccades within the range including the vertical direction or the calculated direction out of the saccades extracted within the period is equal to or less than a third predetermined value; When at least one of the performed conditions is satisfied, the person to be determined is determined to be in a sloppy state. Therefore, as in the first aspect of the invention, the sloppy state of the sought-to-be-determined person can be stabilized with a simple process. In can be determined.

  According to an eighth aspect of the present invention, there is provided a computer program for determining the state of eyeball based on a result of measuring the movement of the eyeball by the eyeball movement measuring unit and the eyeball movement measuring unit. Saccade extraction means for extracting saccades in exercise, calculation means for calculating at least one of amplitude and direction in saccades extracted by the saccade extraction means, and extraction by the saccade extraction means within an arbitrary period The ratio of the saccade whose amplitude calculated by the calculation means is not more than a first predetermined value is not less than a second predetermined value and is extracted by the saccade extraction means within the period Of the saccades, the direction calculated by the calculation means is a vertical direction or a saccade within the range including the vertical direction. Function as a determination unit that determines at least one of the conditions of whether or not the result is equal to or less than a third predetermined value and determines that the determination target person is in a loose state when at least one of the determined conditions is satisfied Let

  Since the computer program for causing the computer to function as the eye movement measuring means, the saccade extracting means, the calculating means, and the judging means is described above, the computer according to claim 8. By executing the random state determination program according to the present invention, the computer functions as the random state determination device according to claim 1, and the determination is performed by simple processing as in the first and seventh aspects. It is possible to determine the moody state of the subject with stable accuracy.

  As described above, the present invention measures the eyeball movement of the determination subject, extracts the saccade in the measured eyeball movement, calculates at least one of the amplitude and direction in the extracted saccade, The ratio of saccades whose amplitude is less than or equal to a first predetermined value is greater than or equal to a second predetermined value, and the direction of saccades extracted within an arbitrary period is vertical or vertical. When the ratio of saccades within the range including the value is equal to or less than the third predetermined value, at least one of the conditions is determined, and the determination target person is in a loose state when at least one of the determined conditions is satisfied Therefore, it has an excellent effect that it is possible to determine the illness state of the person to be determined with a stable accuracy by a simple process.

It is a diagram which shows an example of the frequency distribution for every amplitude of the saccade in the normal state and the obscure state obtained by experiments conducted by the present inventors. It is a diagram which shows an example of the frequency distribution for every direction of the saccade in the normal state and the obscure state obtained by experiments conducted by the present inventors. It is a diagram which shows an example of distribution of the gaze position in the normal state and the random state obtained by the experiment which this inventor etc. implemented. FIG. 2A is a block diagram showing a schematic configuration, and FIG. 2B is a functional block diagram of a casual state determination device according to the present embodiment. It is a flowchart which shows the content of a casual state determination process.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. In the following, numerical values that do not hinder the present invention will be described. However, it goes without saying that the present invention is not limited to the numerical values described below.

  FIG. 4 (A) shows a random state determination device 10 according to the present embodiment. The random state determination device 10 is a device that is mounted on a vehicle and determines whether or not the driver of the vehicle as the determination target is in a random state, and is a face photographing camera 12 that captures a face including the eyes of the driver of the vehicle. A front shooting camera 14 for imaging the front of the vehicle, an alarm output unit 16 including a buzzer, an accelerator pedal reaction force adjustment unit 18 capable of adjusting a reaction force when the vehicle accelerator pedal is operated, and a vehicle brake A brake pedal reaction force adjusting unit 20 capable of adjusting a reaction force when the pedal is operated is provided, and these are connected to a computer 22 mounted on the vehicle.

  The accelerator pedal reaction force adjusting unit 18 is provided in the middle of a path for transmitting the operation amount of the accelerator pedal as hydraulic pressure, and adjusts the operation reaction force of the accelerator pedal by changing the hydraulic pressure in the path. The brake pedal reaction force adjusting unit 20 is also provided in the middle of a path for transmitting the operation amount of the brake pedal as a hydraulic pressure, and the brake pedal operation reaction force is changed by changing the hydraulic pressure in the path. A configuration to be adjusted can be applied.

  The computer 22 includes a CPU 22A, a memory 22B including a ROM and a RAM, a nonvolatile storage unit 22C including an HDD (Hard Disk Drive), a flash memory, and the like, and a random state determination program is stored in the storage unit 22C in advance ( Installed). The sloppy state determination program is a program for performing sloppy state determination processing described later by the computer 22, and is executed by the CPU 22A while the ignition of the vehicle is on. The above-described sloppy state determination process is a process to which the sloppy state determination method according to the present invention is applied. When the computer 22 performs the sloppy state determination process, the computer 22 performs the eye movement measurement unit shown in FIG. 26, the saccade analysis unit 28, the line-of-sight analysis unit 30, and the sloppy state detection processing unit 34 functioning as a sloppy state determination processing unit 34. The sloppy state determination device 10 according to this embodiment is a sloppy state determination device according to the present invention. 10 will function. Note that the above described sloppy state determination program corresponds to the sloppy state determination program according to the present invention.

  Next, as an operation of the present embodiment, a random state determination process realized by executing a random state determination program by the CPU 22A will be described with reference to FIG.

In this casual state determination process, first, the flag is initially set to 0 in step 50, and in the next step 52, the face image obtained by the face photographing camera 12 imaging the face including the eyes of the driver of the vehicle. Is acquired from the face photographing camera 12. In step 54, an eye region corresponding to the driver's eye is extracted from the face image represented by the face image data acquired in step 52, and a pupil corresponding to the pupil of the driver's eye is extracted from the extracted eye region. The region is extracted, and the horizontal eyeball rotation angle θ _H and the vertical eyeball rotation angle θ _V are calculated based on the horizontal position and the vertical position of the pupil region in the eye region. In the next step 56, the eye rotation angle θ _H , θ _V calculated in step 54 is differentiated with respect to time, whereby a differential value θ ′ _H , θ ′ of the eye rotation angle representing the eye rotation speed in the horizontal direction and the vertical direction. Calculate _V. Further, at step 58, the differential values θ ′ _H and θ ′ _V of the eyeball rotation angle are substituted into the following equation (1) to calculate the magnitude T of the eyeball rotation speed vector.
T = √ (θ ′ _H ² + θ ′ _V ² ) (1)
In step 60, the size T of the rotational speed vector of the eyeball calculated in step 58 it is determined whether or not larger than the predetermined value T _theta. In the present embodiment, as the predetermined value T _θ , a value (for example, a value of about 120 to 170 deg / sec) corresponding to the rotation speed of the eyeball in the saccade is set in advance, and the predetermined value T _θ is defined in claim 3. This corresponds to the fourth predetermined value described. Saccade is a fast and linear eye movements from one sight stationary point to another gaze stationary points, larger high speed eye movement also the size T is the predetermined value T _theta rotational speed vector of the eyeball There is no other than linear eye movement. Therefore, if the determination in step 60 is affirmative, it can be determined that saccade (equivalent eye movement) is being performed with the driver's eyeball. If the determination in step 60 is negative, that is, if saccade is not being performed with the driver's eyeball, the routine proceeds to step 62 where it is determined whether or not 1 is set in the flag. If this determination is also denied, the routine proceeds to step 76.

In the casual state determination process according to the present embodiment, steps 52 to 80 including the process described above are repeatedly executed. When saccade is performed with the driver's eyeball, the determination in the previous step 60 is affirmed and the process proceeds to step 64 to determine whether or not 0 is set in the flag. Since the flag is set to 0 initially when the determination in step 60 is affirmed, the determination in step 64 is affirmed and the routine proceeds to step 66 where the eyeball rotation angles θ _H and θ _V calculated in the previous step 54 are set. The saccade start point is stored in the memory 22B. In step 68, 1 is set in the flag, and the routine proceeds to step 76. Thereby, when the determination of step 60 is performed next, when the determination of step 60 is affirmed because the state where the saccade is performed with the driver's eyeball is continued, the determination of step 64 is performed. Is denied and the routine proceeds to step 76.

When the determination in step 60 is denied due to the end of the saccade on the driver's eyeball, the determination in step 62 is affirmed because 1 is set in the flag, and the routine proceeds to step 70. In step 70, the eyeball rotation angles θ _H and θ _V calculated in the previous step 54 are stored in the memory 22B as saccade end points. In the next step 72, 0 is set in the flag. Thus, when saccade is performed again with the driver's eyeball, the determination in step 64 is affirmed, and the eyeball rotation angles θ _H and θ _{V at} that time are stored in the memory 22B as the saccade start point. Will be. Further, in the next step 74, based on the saccade start point and saccade end point stored in the memory 22B, the amplitude and direction (azimuth) of the saccade performed this time with the driver's eyeball are calculated, After the saccade amplitude and direction (orientation) are stored in the memory 22B, the process proceeds to step 76.

As described above, in the random state determination process according to the present embodiment, the eyeball rotation angles θ _H and θ _V are repeatedly measured as the movement of the driver's eyeball, and whether or not the driver's eyeball has been saccaded. Each time it is detected that the saccade has been performed, its amplitude and direction (azimuth) are repeatedly calculated and stored. Of the above steps, steps 52 and 54 are processes performed by the eye movement measuring unit 26 shown in FIG. 4B, and correspond to the eye movement measuring means according to the present invention. Steps 56 to 74 are processes performed by the saccade analysis unit 28 shown in FIG. 4B, and among these, steps 56 to 72 are saccade extraction means according to the present invention (more specifically, claim 3). Step 74 corresponds to the calculation means according to the present invention (more specifically, the calculation means according to claim 2).

On the other hand, in step 76, the direction of the driver's face is calculated by performing predetermined image processing based on the face image data acquired from the face photographing camera 12 in the previous step 52. The direction of the driver's face is obtained by using a feature quantity such as a positional relationship (degree of deviation in the horizontal direction) of a pair of areas corresponding to the driver's eye pair in an area corresponding to the driver's face, for example. It can be calculated. Next, based on the face direction calculated in step 76 and the eyeball rotation angles θ _H and θ _V calculated in the previous step 54, the direction of the driver's line of sight (with respect to the state in which the driver is looking forward in front of the vehicle). Calculate the angle difference between the line of sight. The direction of the driver's line of sight is obtained by, for example, substituting the calculation result of the driver's face orientation and the eyeball rotation angles θ _H and θ _V into the following equation (2), and the line-of-sight direction vector (x, y, It can be obtained by calculating z).

However, in the above equation (2), (α, β, γ) are the roll angle, pitch angle, and yaw angle of the driver's face. In step 78, the driver's line-of-sight direction calculated in step 76 (for example, the direction represented by the line-of-sight direction vector (x, y, z)) is calculated by substituting it into a predetermined coordinate conversion formula, so that the space ahead of the vehicle is calculated. The coordinate position (line-of-sight position) of the driver's line of sight on the image representing the image (image captured by the front camera 14) is calculated and stored in the memory 22B.

  When the driver's face orientation calculated in step 76 deviates from a predetermined angle range, there is a high possibility that a large error is included in the line-of-sight direction or line-of-sight position to be calculated subsequently. Therefore, it is preferable to stop the calculation of the viewing direction and the viewing position. Since the above-described steps 76 and 78 are repeatedly executed in the casual state determination process according to the present embodiment, the driver's line-of-sight position is repeatedly calculated and stored. Steps 76 and 78 described above are processes performed by the line-of-sight analysis unit 30 shown in FIG. 4B and correspond to the line-of-sight position calculation means according to claim 5. The face photographing camera 12 corresponds to the face image pickup means described in claim 5.

  In the next step 80, it is determined whether or not it is time to determine the driver's casual state. In the present embodiment, the determination cycle of the driver's mood state is set in advance (for example, about 1 minute), and the above determination is performed by starting execution of the mood state determination process or by determining the driver's mood state last time. From this, it is determined by determining whether or not a time corresponding to the determination period has elapsed. If the determination in step 80 is negative, the process returns to step 52, and steps 52 to 80 are repeated until the determination in step 80 is affirmed. During this time, the amplitude and direction (azimuth) of the saccade and the driver's line-of-sight position are repeatedly calculated, and the calculation results are accumulated and stored in the memory 22B.

  When it is time to determine the driver's mood, the determination at step 80 is affirmed and the routine proceeds to step 82, where the soccer football stored and stored in the memory 22B within a predetermined period (a period corresponding to the determination period) is determined. The data representing the amplitude of the saccade is read, and the frequency for each amplitude of the saccade within a predetermined period is calculated based on the read data. Thereby, the frequency distribution as shown in FIG. 1 is obtained as an example. In the next step 84, the ratio of saccades whose amplitude is a predetermined value A or less among all the saccades from which data representing the amplitude in step 82 is read out from the memory 22B is calculated, and the calculated ratio is a preset threshold value. It is determined whether or not it is larger than Ta.

  As the predetermined value A, for example, “A” shown in FIG. 1, in the frequency distribution for each saccade amplitude, the amplitude at the boundary where the magnitude relationship is reversed depending on whether the driver is in a normal state or a casual state. A value can be applied. For example, a value of about 0.65 to 0.75 can be applied as the threshold Ta. The predetermined value A corresponds to the first predetermined value in the present invention, and the threshold value Ta corresponds to the second predetermined value in the present invention.

  If the determination in step 84 is negative, the process proceeds to step 86, and data representing the saccade direction (azimuth) stored and stored in the memory 22B within a predetermined period (a period corresponding to the determination period) is read and read. The frequency for each saccade direction (orientation) within a predetermined period is calculated based on the obtained data. Thereby, the frequency distribution as shown in FIG. 2 is obtained as an example. In the next step 88, the ratio of saccades within the range in which the direction (azimuth) is classified into the vertical direction is calculated among all the saccades in which the data representing the direction (azimuth) is read from the memory 22B in step 86. It is determined whether the calculated ratio is smaller than a preset threshold value Td.

  Note that the direction (azimuth) range classified in the vertical direction is, for example, in the frequency distribution for each saccade direction (azimuth) as shown in FIG. 2, depending on whether the driver is in a normal state or a casual state. It is possible to apply a direction (orientation) range (for example, a range centered on a direction (orientation) of about 90 ° or a direction (orientation) of about 300 °) or the like that is clearly different. Further, as the threshold value Td, for example, a value of about 0.60 to 0.65 can be applied although it depends on the size of the direction (orientation) range classified in the vertical direction. The direction (orientation) range classified as the vertical direction corresponds to the “vertical direction or a range including the vertical direction” in the present invention, and the threshold value Td corresponds to the third predetermined value in the present invention.

  If the determination in step 88 is negative, the process proceeds to step 90, in which data representing the driver's line-of-sight position stored and stored in the memory 22B within a predetermined period (a period corresponding to the determination period) is read, and the read data Based on the above, the frequency distribution of the line-of-sight position within a predetermined period is calculated. As a result, an image representing the (frequency) distribution of the line-of-sight position as shown in FIG. 3A or FIG. 3B is obtained as an example. In the next step 92, with respect to the image representing the (frequency) distribution of the line-of-sight position obtained in step 90, the frequency as the line-of-sight position is composed only of pixels having a predetermined value or more, and the frequency as the line-of-sight position is less than the predetermined value. The region surrounded by the pixel is extracted as a location where the line-of-sight position is concentrated. Instead of this, for example, a region that is composed only of pixels corresponding to the line-of-sight position and is surrounded by pixels that do not correspond to the line-of-sight position and whose area is equal to or larger than the threshold is extracted as a location where the line-of-sight position is concentrated. It may be. Then, it is determined whether or not the number of regions extracted as locations where the line-of-sight positions are concentrated is two or more.

  If the determination in step 92 is negative, the determinations in steps 84, 88, and 92 are all negative, and it can be determined that the driver is not in a random state. Therefore, the process returns to step 52 without performing any processing. The process after 52 is repeated. On the other hand, if the determination in any of steps 84, 88, and 92 is affirmed, the driver is likely to be in a loose state. Therefore, the process proceeds to step 94 and the alarm output unit 16 outputs an alarm. Call attention to a driver who is likely to be in a sparse state. In step 96, the accelerator pedal reaction force adjusting unit 18 increases the accelerator pedal reaction force, and in the next step 98, the brake pedal reaction force adjusting unit 20 decreases the brake pedal reaction force. Thereby, even when the alarm output from the alarm output unit 16 is not clearly heard by the driver due to the influence of ambient noise or the like, the driver can be surely cautioned. If the process of step 98 is performed, it will return to step 52 and the process after step 52 will be repeated.

  Steps 80 to 98 described above are processes performed by the random state detection unit 32 shown in FIG. 4B, and among these, steps 80 to 92 are determination means according to the present invention (more specifically, claims). This corresponds to the determination means described in 2, 5, and 6).

  As described above, in this embodiment, the ratio of saccades whose amplitude is equal to or less than the predetermined value A, the ratio of saccades within the range where the direction (azimuth) is classified in the vertical direction, and the locations where the line-of-sight positions are concentrated. "The ratio of saccades whose amplitude is less than or equal to a predetermined value A is larger than the threshold value Ta", "The ratio of saccades within the range in which the direction (direction) is classified in the vertical direction is smaller than the threshold value Td" And “the number of locations where the line-of-sight positions are concentrated is 2 or more”, the driver determines that the driver is in a random state. A simple process that does not require complex image processing, such as extracting an image area that is estimated to correspond to an object or a preceding vehicle, can determine the driver's casual state, depending on the driver's face orientation, etc. Avoiding a drastic drop in the accuracy of determination Rukoto can.

  In the above, “the ratio of saccades whose amplitude is equal to or less than the predetermined value A is larger than the threshold Ta”, “the ratio of saccades within the range in which the direction (direction) is classified in the vertical direction is smaller than the threshold Td”. In addition, although the description has been given of the mode in which the driver determines that the driver is in an indeterminate state when any of the conditions “the number of locations where the line-of-sight positions are concentrated is 2 or more” is described, the present invention is not limited to this. For example, when two or more of the above conditions are satisfied, it may be determined as a random state. For example, the condition that “the number of locations where the line-of-sight positions are concentrated is two or more” is satisfied. In this case, it is determined that the state is indiscriminate regardless of the result of other conditions, while “the ratio of saccades whose amplitude is equal to or less than a predetermined value A is greater than the threshold Ta” and “range in which the direction (azimuth) is classified in the vertical direction "The ratio of saccades is smaller than the threshold Td" For, as such determines that absentminded state if it meets also one or more other conditions, the number of conditions for determining the absentminded state may not be constant at each condition. The present invention also includes these aspects.

  Further, in the above, “the ratio of saccades whose amplitude is equal to or less than the predetermined value A is larger than the threshold value Ta”, “the ratio of saccades within the range in which the direction (direction) is classified in the vertical direction is smaller than the threshold value Td”. And when the driver satisfies any of the conditions “the number of locations where the line-of-sight position is concentrated is 2 or more”, the driver determines that the state is indeterminate, outputs an alarm, increases the reaction force of the accelerator pedal, and Although the aspect of uniformly reducing the reaction force has been described, the present invention is not limited to this. For example, as the number of conditions that satisfy the condition increases, the level of the alarm to be output increases, the accelerator pedal or the brake pedal The reaction force change amount may be increased.

  In the above description, the threshold Ta used for the determination of the ratio of saccades whose amplitude is equal to or less than the predetermined value A (step 84 in FIG. 5) and the determination of the ratio of saccades within the range in which the direction (direction) is classified in the vertical direction ( Although a mode in which preset values are used as the threshold values Td used in step 88) in FIG. 5 has been described, the present invention is not limited to this, and the initial period when the driver starts driving the vehicle. For example, during the initial period when the vehicle ignition is turned on and the execution of the sloppy state determination process is started, the ratio of saccades whose amplitude is equal to or less than a predetermined value A is calculated and set to the threshold Ta, and the direction (azimuth) The ratio of saccades within a range classified in the vertical direction may be calculated and set as the threshold value Td. For example, when the determination of step 80 shown in FIG. 5 is first affirmed, the ratio of saccades whose amplitude is a predetermined value A or less is calculated instead of the determination of steps 84 and 86. The threshold value Ta is set, and the ratio of the saccades within the range in which the direction (orientation) is classified in the vertical direction is calculated and changed to perform processing for setting the threshold value Td. The optimum values of the threshold values Ta and Td may be different for each driver. However, by performing the above processing, the optimum values for the individual drivers can be used as the threshold values Ta and Td. It is possible to improve the determination accuracy of a person's ill-mannered state. The above aspect corresponds to the invention described in claim 4.

  Furthermore, “the ratio of saccades whose amplitude is equal to or smaller than a predetermined value A is larger than the threshold Ta”, “the ratio of saccades within a range in which the direction (direction) is classified in the vertical direction is smaller than the threshold Td” and “ Although the embodiment has been described in which the driver's casual state is determined by using all the conditions that the number of locations where the line-of-sight positions are concentrated is 2 or more, the present invention is not limited to this. The driver's casual state may be determined without using the condition that the number of locations where the positions are concentrated is 2 or more, or “the ratio of saccades whose amplitude is equal to or less than a predetermined value A is greater than the threshold Ta. The ratio of the saccade within the range in which the direction (direction) is classified in the vertical direction is smaller than the threshold value Td. Good. The present invention also includes these aspects.

  Moreover, although the aspect which measures the movement of a driver | operator's eyeball based on the face image data obtained by imaging a driver | operator's face was demonstrated above, it is not limited to this, For example, a driver | operator's eyeball is not limited to this. By measuring the movement of the eyeball by applying the EOG (electro-oculograph) method based on the signals input from the electrodes attached to multiple locations on the head (for example, the left and right temples and the top and bottom of the eyeball) It is also possible to employ a configuration for measuring the movement of the eyeball.

  Moreover, although the aspect which applied the driver | operator of the vehicle as a determination subject person according to the present invention has been described above, the determination subject person according to the present invention is not limited to the vehicle driver, and the present invention is other than a car. Needless to say, it is possible to make a vague state determination using an arbitrary person such as an operator who operates an arbitrary device as a determination target.

  Further, in the above description, the mode in which the random state determination program according to the present invention is stored (installed) in advance in the storage unit 22C of the computer 22 has been described. However, the random state determination program according to the present invention can be a CD-ROM or DVD- It is also possible to provide the information recorded in a recording medium such as a ROM.

DESCRIPTION OF SYMBOLS 10 Fragile state determination apparatus 12 Face photographing camera 16 Alarm output part 18 Accelerator pedal reaction force adjustment part 20 Brake pedal reaction force adjustment part 22 Computer 26 Eye movement measurement part 28 Saccade analysis part 30 Line-of-sight analysis part 32 Mantle state detection part 34 State determination processing section

Claims (8)

Eye movement measuring means for measuring the movement of the eyeball of the determination target person;
Saccade extraction means for extracting saccade in the eye movement based on the measurement result of the eye movement by the eye movement measurement means;
A computing means for computing at least one of amplitude and direction in the saccade extracted by the saccade extracting means;
Whether or not the ratio of saccades whose amplitude calculated by the calculation means out of the saccades extracted by the saccade extraction means within an arbitrary period is not more than a first predetermined value is not less than a second predetermined value, and Whether or not the ratio of the saccade within the range including the vertical direction or the direction calculated by the calculation means out of the saccades extracted by the saccade extraction means within a period is equal to or less than a third predetermined value; Determining means for determining at least one of the conditions, and determining that the determination target person is in a loose state when at least one of the determined conditions is satisfied;
A simple state determination device. The calculation means calculates the amplitude and direction in the saccade extracted by the saccade extraction means,
The determination means is configured to determine whether a ratio of saccades having an amplitude calculated by the calculation means of the saccades extracted by the saccade extraction means is equal to or greater than a second predetermined value, and Each condition of whether or not the ratio of the saccade within the range including the vertical direction or the direction including the vertical direction among the saccades extracted by the saccade extraction means is equal to or less than a third predetermined value, respectively 2. The random state determination device according to claim 1, wherein the determination target person determines that the determination target person is in a random state when at least one of the conditions is satisfied.   The saccade extraction means calculates a transition of the rotation speed of the eyeball based on the measurement result of the eyeball movement by the eyeball movement measurement means, and the period during which the rotation speed of the eyeball maintains a fourth predetermined value or more The random state determination device according to claim 1, wherein the movement of the eyeball is extracted as the saccade. The determination target person is an operator who operates the device,
The ratio of saccades whose amplitude calculated by the calculation means out of the saccades extracted by the saccade extraction means during the initial period when the judgment subject starts operating the device, is less than or equal to a first predetermined value, And corresponding to the determination performed by the determination means out of the ratio of the saccades extracted by the saccade extraction means within the range in which the direction calculated by the calculation means includes the vertical direction or the vertical direction. 4. The casual state determination device according to claim 1, further comprising setting means for obtaining a parameter and setting the obtained parameter as the second predetermined value or the third predetermined value. Face imaging means for imaging the determination subject's face;
The orientation of the determination target person is determined based on an image obtained by imaging by the face imaging means, and the measurement result of the eye movement by the eye movement measurement means and the determined orientation of the determination target person's face Line-of-sight position calculating means for repeatedly calculating the line-of-sight position of the person to be determined based on
Further comprising
The determination means also determines whether or not there are a plurality of locations where the line-of-sight positions calculated by the line-of-sight position calculation means exist within an arbitrary period as one of the conditions, and includes the conditions The muffled state determination device according to claim 1, wherein the determination target person determines that the determination target person is in a muddy state when at least one of the conditions is satisfied. The line-of-sight position calculating means calculates the eye direction of the determination target person based on the measurement result of the movement of the eyeball and the determined orientation of the face of the determination object person, and based on the calculated line-of-sight direction, the determination target Repeatedly calculating the position of the determination target person's line of sight on the image representing the space being watched by the person as the determination target person's line of sight,
The determination unit is a pixel corresponding to the line-of-sight position calculated by the line-of-sight position calculation unit within the arbitrary period or the line-of-sight position calculated by the line-of-sight position calculation unit within the arbitrary period. Surrounded by pixels that have only a frequency that is greater than or equal to a predetermined value and that do not correspond to the line-of-sight position, or pixels that have a frequency calculated as the line-of-sight position by the line-of-sight position calculation means within the arbitrary period. The random state determination apparatus according to claim 5, wherein the region is recognized as a location where the line-of-sight position is concentrated. Measure the eye movement of the person being judged,
Extracting saccades in the eye movement based on the measurement results of the eye movement;
Calculating at least one of amplitude and direction in the extracted saccade;
Of the saccades extracted within an arbitrary period, whether or not the ratio of saccades whose calculated amplitude is less than or equal to a first predetermined value is greater than or equal to a second predetermined value, and among the saccades extracted within the period Whether or not the ratio of the saccade within the range including the vertical direction or the vertical direction is equal to or less than the third predetermined value is determined, and at least one of the determined conditions is satisfied In this case, the method for determining a sloppy state in which the determination target person determines to be a sloppy state. Computer
Eye movement measuring means for measuring the movement of the eyeball of the person to be judged,
Saccade extraction means for extracting saccade in the eye movement based on the measurement result of the eye movement by the eye movement measurement means;
A computing means for computing at least one of amplitude and direction in the saccade extracted by the saccade extracting means;
And whether or not the ratio of the saccade whose amplitude calculated by the calculation means is less than or equal to a first predetermined value among the saccades extracted by the saccade extraction means within an arbitrary period is equal to or greater than a second predetermined value, and Whether or not the ratio of the saccades within the range including the vertical direction or the direction calculated by the calculation means out of the saccades extracted by the saccade extraction means within the period is equal to or less than a third predetermined value. A random state determination program for causing at least one of the above conditions to be determined and functioning as determination means for determining that the determination target person is in a random state when at least one of the determined conditions is satisfied.