JP4756503B2 - Blink detection device and computer program - Google Patents

Description

  The present invention relates to a technique for detecting a human blink, and more particularly to a technique for stably detecting a blink state from the position of an eyelid in a human face image.

  Due to the increase in traffic volume in recent years, problems related to automobiles, particularly problems related to traffic accidents, are attracting attention. In order to operate a car safely, there are a direction of automating the driving of a car and a direction of helping a person (driver) driving the car to drive safely. Research on technologies for automating car driving has also been conducted, but it has not been put into practical use for various reasons. On the other hand, various devices for supporting automobile drivers have been proposed, and these devices support drivers, so that there is a possibility of practical application compared to technologies for automating driving of automobiles. high.

  One piece of information necessary to support the driver is information regarding the frequency of blinking of the driver, that is, information regarding the eye open / closed state. For example, when the driver's consciousness state decreases, the time for which the eyes are closed increases or the number of blinks decreases. By using such information, if it is determined that the driver's consciousness state is not normal, appropriate measures can be taken to return the consciousness state to normal.

  Information about blinking is not only used for driver assistance. For example, the blinking state of a speaker when interviewing a certain person is considered to reflect the psychological state of the speaker. Therefore, it is possible to evaluate the spoken content by detecting the blink based on the face image of the speaker.

  There are many other situations where information about blinks can be used.

Non-Patent Document 1 discloses a technique for detecting human blinks. In the technique disclosed in Non-Patent Document 1, images in the vicinity of the driver's face are continuously captured by a camera arranged in a rearview mirror. A face area and an eye area are extracted from the image using a neural network. From the extracted eye region, a combination of bright and dark edges with similar width and position is detected as upper and lower eyelids using the difference in brightness between the skin and eyes. Based on the change in the distance between the upper and lower eyelids detected in this way, it can be determined whether or not blinking has occurred.
JP 2004-185611 A JP 2003-168121 A Ito et al., “Driver Monitor Considering Individual Differences by Driver Identification”, Dynamic Image Processing Realization Workshop 2006, Proceedings, March 9-10, 2006, p. 191-195

  However, according to the above prior art, the following problems occur. For example, when capturing a face image of a subject, the light source is usually above the subject. When the light source is above the head, the upper eyelid is in a portion close to the light source and shadows are not easily generated. Therefore, the difference in brightness between the eyelid (skin) and the eyes is easy to detect, and the edge of the upper eyelid can be detected relatively easily.

  However, the lower eyelid is far from the light source and is in a position where shadows are likely to occur when light is shining over it. Therefore, it is difficult to detect a difference in brightness between eyes and skin. As a result, there is a problem that it is relatively difficult to detect the edge of the lower eyelid and it is difficult to detect the lower eyelid in a stable manner in practice.

  In the prior art, if it is difficult to detect the edge of the lower eyelid, the distance between the upper and lower eyelids cannot be measured, and as a result, the blink cannot be detected stably.

  Therefore, an object of the present invention is to provide an apparatus and a computer program that can stably detect the occurrence of blinking from a moving image of a human face.

  A blink detection apparatus according to a first aspect of the present invention is a blink detection apparatus for detecting blinks from a moving image of a face including a series of frames, for tracking the positions of both eyes of the face in the series of frames. Eye position tracking means, eyelid detection means for detecting the edge of the upper eyelid in a series of frames, and eye positions identified by eye position tracking means in the face image of the preceding frame A first distance calculating means for calculating a first distance from a predetermined reference line determined by the position to an edge of the upper eyelid detected by the eyelid detecting means; and a predetermined reference in the face image of the preceding frame A new reference line having a predetermined relationship with the line is specified based on the eye position specified by the eye position tracking unit in the face image of the frame following the preceding frame. And a second distance calculation means for calculating a second distance from the new reference line to the edge of the upper eyelid detected by the eyelid detection means in the face image of the subsequent frame And blink detection means for detecting the occurrence of blink based on the difference between the distances calculated by the first and second distance calculation means.

  According to this blink detection device, the first distance calculation means calculates the first distance from the predetermined reference line determined by the position of the eye included in the preceding frame to the edge of the upper eyelid, and the second distance The calculating means calculates a second distance from the new reference line included in the subsequent frame to the edge of the upper eyelid. The blink detection means detects the occurrence of blink based on the difference between the calculated distances. There is no need to detect the edge of the lower eyelid that becomes relatively unstable, and blinking can be detected only by the edge of the upper eyelid that can be detected stably. As a result, it is possible to provide a blink detection device that can stably detect the occurrence of blinks from a moving image of a human face.

  Preferably, the first distance calculation unit is configured to determine a line connecting the positions of the eyes and the upper eyelid detected by the eyelid detection unit based on the positions of both eyes specified by the eye position tracking unit in the preceding frame. Means for calculating the distance between the edges.

  According to this blink detection device, the means for calculating the distance calculates the distance between the line connecting the positions of both eyes and the edge of the upper eyelid. The occurrence of blinking can be detected based on the line connecting the positions of both eyes that can be detected stably and the edge of the upper eyelid. As a result, it is possible to provide a blink detection device that can stably detect the occurrence of blinks.

  Preferably, the reference line specifying means includes a reference pattern extracting means for extracting and storing a region of a prescribed shape as a reference pattern including a reference point having a predetermined relationship with the positions of both eyes in the face image of the preceding frame. Template matching means for estimating the position of the reference point of the preceding frame in the succeeding frame by template matching for the succeeding frame using the reference pattern extracted by the reference pattern extracting means as a template, and template matching Means for specifying as a new reference line a line that passes through the position of the reference point estimated in the subsequent frame by means and is parallel to a straight line connecting the centers of both eyes in the subsequent frame.

  According to this blink detection device, the reference pattern extraction unit extracts and stores a region including a reference point having a predetermined relationship with the position of both eyes in the face image of the preceding frame as a reference pattern. The template matching means estimates the position of the reference point of the preceding frame in the subsequent frame by template matching for the subsequent frame using the reference pattern as a template. A means for identifying identifies a new reference line based on the position of the reference point estimated in the subsequent frame. Using the positions of both eyes that are stably measured, it is possible to specify a reference line in the preceding frame and a new reference line in the subsequent frame. As a result, it is possible to provide a blink detection device that can stably detect the occurrence of blinks based on the reference line.

  Preferably, the reference pattern extracting means extracts and stores a defined shape region including the reference point as a reference pattern, with the midpoint position of the line segment connecting the positions of both eyes in the face image of the preceding frame as the reference point. Including means.

  According to this blink detection device, the means for saving uses the midpoint position of the line segment connecting the positions of both eyes in the face image of the preceding frame as a reference point, and uses a region having a defined shape including the reference point as a reference pattern. Extract and save. A reference pattern is extracted using the midpoint position of the line segment connecting the positions of both eyes that can be detected stably as a reference point, and template matching using the reference pattern is performed on the subsequent frames. This part is a characteristic part of the face compared to others, and the position of a new reference point can be stably identified by template matching for subsequent frames. Therefore, a new reference line can be reliably specified in the subsequent frame. As a result, it is possible to provide a blink detection device that can stably identify a reference line that is a reference for the distance to the upper eyelid in both the preceding frame and the subsequent frame, and can stably detect the occurrence of blinking. .

  Preferably, the prescribed shape is a rectangular shape.

  Preferably, the reference point is a rectangular center point.

  Preferably, the means for calculating the distance is based on the position of both eyes specified by the eye position tracking means in the preceding frame, and a line connecting the positions of both eyes and the upper eyelid detected by the eyelid detection means Means for calculating a distance between a predetermined point in the edges of the first eye and the second distance calculating means includes the new reference line and the edge of the upper eyelid detected by the eyelid detection means in the subsequent frame Means for calculating a distance from a point corresponding to the predetermined point.

  According to this blink detection device, the distance between the line connecting the eyes in the preceding frame and the predetermined point in the upper eyelid edge, and the new frame defined by the point corresponding to the reference point of the preceding frame in the following frame The distance between a simple reference line and a predetermined point in the upper eyelid edge can be calculated. Therefore, the distance to the upper eyelid in the preceding frame and the distance to the upper eyelid in the subsequent frame can be calculated according to a common criterion. As a result, it is possible to provide a blink detection device that can accurately detect the occurrence of blinks.

  Preferably, the predetermined point in the edge of the upper eyelid is a point that gives the maximum value of the distance between the line connecting the positions of both eyes and the edge of the upper eyelid.

  According to this blink detection device, the distance between the line connecting the positions of both eyes and the point giving the maximum value of the distance between the edge of the upper eyelid can be calculated. Therefore, it is possible to detect the occurrence of blink using a clear value that is the maximum value of the distance between the line connecting the positions of both eyes and the edge of the upper eyelid. As a result, it is possible to provide a blink detection device that can easily detect the occurrence of blinks.

  Preferably, the blink detection means compares the result obtained by the means for subtracting the second distance from the first distance with the predetermined positive threshold value, and the result is Means for determining that blinking has been detected when greater than the threshold value.

  According to this blink detection device, the means for subtracting subtracts the second distance from the first distance, and the means for determining determines the blink when the result of the subtraction is greater than a predetermined threshold value. It is determined that it has been detected. Based on a predetermined threshold value, the occurrence of blinking can be detected with a simple determination. As a result, it is possible to provide a blink detection device that can detect the occurrence of blinks by simple processing.

  More preferably, the blink detection device further includes an imaging unit that captures a moving image and supplies it to the eye position detection unit, the eyelid detection unit, and the reference line identification unit as a series of frames.

  According to this blink detection device, the imaging unit captures a moving image and supplies it as a series of frames to the eye position detection unit, the eyelid detection unit, and the reference line identification unit. It is possible to obtain a moving image as a premise for detecting the occurrence of blinking and immediately process it. As a result, it is possible to provide a blink detection device that can efficiently detect the occurrence of blinks.

  When the computer program according to the second aspect of the present invention is executed by a computer having a function of processing a moving image as a series of frames, the computer program operates as any one of the blink detection devices described above. The same effect as that of the blink detection device can be obtained.

  Before describing the apparatus and method according to the embodiment of the present invention, first, the basic concept regarding the eyelid edge detection method and blink detection method according to the present embodiment will be described.

<Basic principle of blink detection>
FIG. 1 is a schematic diagram illustrating a human blink state in successive frames of image signals. Referring to FIG. 1, a face 30 is a human eye state in a certain frame. It is assumed that both eyes are open in this frame. The face 30 includes an upper eyelid 40 for the right eye, an upper eyelid 42 for the left eye, a right eye 44, and a left eye 46.

  It is assumed that the eye open / close state changes in the face 32 of the next frame. That is, in this frame, both eyes are in the middle of being closed. In this case, the positions of the upper eyelid 50 and the upper eyelid 52 of the left eye of the face 32 are both lower than the positions of the upper eyelid 40 and the upper eyelid 42 of the right eye in the face 30 of the previous frame. .

  If the position of the face is fixed in the image, it is possible to easily determine whether or not there is a blink by measuring the position of the upper eyelid in the image and detecting the change. However, the human head is usually moving and its movement cannot be easily predicted. Therefore, even if the position of the upper eyelid can be detected, it is not possible to determine whether or not there has been a blink.

  In the present invention, a common reference position for detecting the position of the upper eyelid is defined between two frames, and the position of the upper eyelid with respect to this reference position is measured to determine whether or not there has been a blink.

  The frame rate of the moving image used in the blink detection device according to the present embodiment is 30 frames per second. Further, a method for extracting a face position which is a premise for detecting eyes is based on the method described in Patent Document 1. After detecting the face position, the position of both eyes is tracked and detected using the method described in Patent Document 2. According to the method described in Patent Document 2, even if a blinking frame (that is, a frame in which no eyes are present in an image), the position of both eyes is correctly tracked. Has been confirmed.

  FIG. 2 shows the basic principle of the method for monitoring the movement of the upper eyelid. For convenience, the case where blinks are detected in common for both eyes is described here, but it is actually desirable to separately perform the following processing for both the right eye and the left eye.

  Referring to FIG. 2, a straight line 80 passing through the center of both eyes is determined by tracking the positions of both eyes from an image 70 near the eyes in the preceding first frame image. Next, the upper eyelid edge of the eye is detected, and the distance d0 between the uppermost point of the edge (the position indicated by the straight line 82 in FIG. 2) and the straight line 80 is calculated.

  Further, a small area having a rectangular shape centering on the midpoint 86 of the straight line 80 connecting both eyes is extracted from the image as an inter-eyes pattern 84. The inter-eye pattern 84 is an image composed of N × M pixels.

  The interval pattern 84 is stored in a predetermined storage area. The stored interval pattern 84 is set as a reference point pattern 90. The center 92 of the reference point pattern 90 is the midpoint 86 of a straight line 80 connecting both eyes in the first frame image.

  Given a second frame that follows the first frame, the stored reference point pattern 90 is used as a template to search for an area of N × M pixels that best matches the reference point pattern 90 in the image of the next frame. To do. This is called template matching processing.

  In FIG. 2, in the second frame, an image near the eye when no blink occurs is shown as an image 100, and an example of an image near the eye when blink occurs is shown as an image 102.

  By the template matching process, the position of the pattern 120 corresponding to the reference point pattern 90 in the images 100 and 102 near the eyes is determined. Accordingly, in the images 100 and 102 near the eyes, the position of the reference point 122 corresponding to the midpoint 86 in the image 70 one frame before is also determined.

  The positions of both eyes are detected by the method described in Patent Document 2 in either case of the images 100 and 102. Based on the detected positions of both eyes, a straight line 112 passing through the center of both eyes is determined. In the present embodiment, here, a reference line 114 that passes through the reference point 122 and is parallel to the straight line 112 that passes through the center of both eyes is determined. In addition, by applying an edge filter to the image near both eyes, the upper eyelid edge of both eyes is detected. Then, the distance d1 between the edge position (for example, the position indicated by the straight line 110 in FIG. 2) and the reference line 114 is calculated.

  Here, the distances d0 and d1 are both represented by the number of pixels. If the eyelid state does not change as shown in the image 100 when moving from one frame to the next frame, the distance d0 in the certain frame and the distance d1 in the next frame also do not change. . When the blink is not performed, the position of the human eyelid is relatively stable and does not change, and as a result, d0−d1≈0.

  Similarly, even in a state where the eyelid starts to be closed (image 102), a reference line 114 passing through the reference point 122 is determined in parallel with the straight line 112 passing through the centers of both eyes. Eyelid edges are detected, and a distance d1 between these edges (represented by a straight line 130) and the reference line 114 is calculated.

  If there is a blink, d1 <d0. Therefore, a certain threshold value is set, and if d0−d1> threshold value, it is determined that there is a blink. In the embodiment described below, the image to be processed is a digital image, and it is inevitable that the eyelid detection position includes an error in units of one pixel. Accordingly, here, it is assumed that the threshold value is 2, and if the eyelid edge has moved by two or more pixels, it is estimated that blinking has occurred. Note that the threshold value depends on the number of pixels included in the image to be processed, particularly the number of pixels in the vertical direction. In the experiment, it was possible to obtain a stable result with a threshold value = 2 in an image of 320 × 240 pixels.

<Configuration>
[Realization by computer]
Hereinafter, a car navigation system (hereinafter referred to as “car navigation system”) incorporating the blink detection device according to the present embodiment will be described. This car navigation system is substantially realized by computer hardware and software executed on the computer hardware.

  FIG. 3 shows the appearance of the car navigation system 150. Referring to FIG. 3, car navigation system 150 shoots a liquid crystal screen 162 that displays a map and the like, a touch panel 164 that is stacked on liquid crystal screen 162 and is operated by touching a user, speaker 166, and the face of the driver. It includes a camera 168 and a car navigation main body 160 having a CD-ROM (Compact Disc Read Only Memory) drive 170 and substantially the same hardware as a computer.

  Referring to FIG. 4, in addition to CD-ROM drive 170, car navigation main body 160 includes a CPU (central processing unit) 190, a bus 198 connected to CPU 190 and CD-ROM drive 170, a boot-up program, and the like. A read-only memory (ROM) 192 for storing, a random access memory (RAM) 194 for storing program commands, system programs, work data, etc., all connected to the bus 198, and images taken by the camera 168 at 30 seconds per second. Connected to a camera I / F (Interface) 196 for capturing frame by frame and a speaker 166, a sound board 200 for processing sound output from the CPU 190 or the CD-ROM drive 170 and giving it to the speaker 166, and output from the CPU 190 Digital It includes the D / A (Digital Analog) converter 202 to an audio signal into an analog signal output, and a speaker 204 connected to the output of the D / A converter 202.

  Although not shown here, the car navigation body 160 may further include a network adapter board that provides a connection to a local area network (LAN) in the car.

  A computer program for causing the car navigation system 150 to operate as a blink detection device is stored in the CD-ROM 167 or the ROM 192 inserted into the CD-ROM drive 170. This program is loaded into the RAM 194 at the time of execution. The program may be loaded directly into the RAM 194 from the CD-ROM 167 or via a network.

  This program includes a plurality of instructions for causing the car navigation body 160 to operate as the blink detection device of this embodiment. Some of the basic functions necessary to perform this operation are provided by operating system (OS) or third party programs that run on the car navigation main body 160, or modules of various tool kits installed in the car navigation main body 160. The Therefore, this program does not necessarily include all functions necessary to realize the system and method of this embodiment. This program works with computer hardware by calling appropriate functions or “tools” in a controlled manner so that a desired result can be obtained. It only needs to include instructions that implement the function. Since the operation of the computer constituting the car navigation main body 160 is well known, it will not be mentioned here.

  FIG. 5 shows a flowchart when the blink detection device according to the present embodiment is realized by a computer program. Referring to FIG. 5, first, when the program is started, initialization is performed in step 210. Here, the frame number fn of the moving image of the face is set to 0.

  In step 212, 1 is added to the frame number fn, and the frame of the moving image is read from the camera I / F 196 shown in FIG.

  In step 214, processing for extracting a face area in the read frame and detecting the positions of both eyes is performed.

  In step 216, processing is performed to extract the upper eyelid edge in the frame being processed.

  In step 218, processing for determining whether or not the frame number fn is 1 is performed. If the frame number fn is not 1, the process proceeds to step 220. If the frame number fn is 1, the process proceeds to step 229. At the beginning of the process, there is no preceding frame, and there is no object to be compared in order to detect blinks, so this process is performed.

  In step 220, processing for specifying the reference point C in the image of the frame being processed is performed using the reference point pattern of the preceding frame already saved in the preceding cycle as a template.

  In step 222, a process of calculating a distance d1 to the straight line passing through the reference point C and parallel to the line segment connecting the centers of both eyes and the upper eyelid edge is performed.

  In step 224, the distance d1 calculated in step 222 for the image being processed is subtracted from the distance d0 between the line connecting both eyes and the upper eyelid edge calculated for the preceding frame image. Then, a process for obtaining the difference d is performed.

  In step 226, processing for determining whether or not the value of the difference d calculated in step 224 is greater than 1 is performed. If the difference d is greater than 1, the process proceeds to step 228. If the difference d is 1 or less, the process proceeds to step 229.

  In step 228, it is determined that blinking has been detected, and processing to be performed when blinking is detected is performed. For example, the number of blinks is added, the blink time is stored in a predetermined storage area, or a predetermined signal is output. This information is passed to another process, for example, and necessary processing is executed according to the blinking frequency. Thereafter, the process proceeds to step 229.

  In step 229, the distance between the line connecting both eyes and the edge of the upper eyelid is calculated and stored as the distance d0. Thereafter, the process proceeds to step 230.

  In step 230, the new reference point C is set at the center position of both eyes on the line connecting the eyes in the image of the frame being processed. The position of the new reference point C in the next frame is determined in the process of step 220 for the next frame.

  In step 232, an image composed of N × M pixels centered on the newly determined reference point C in the image of the frame being processed is saved in a predetermined storage area as a new reference point pattern. . This reference point pattern is used in the process of step 220 for the next frame. When this process ends, the process returns to step 212, and the subsequent processes are repeated for the next frame.

<Operation>
FIG. 6 shows a specific example of the operation of the detection of the upper eyelid edge and the blink detection using the reference point pattern.

  Referring to FIG. 5, at the start of the program, initialization is performed in step 210, and frame number fn becomes zero. In step 212, 1 is added to the frame number fn, so that fn = 1. At this time, the image of frame number 1 is read.

  In step 214, the position of both eyes is detected from the image of frame number 1, and in step 216, the edge 250 of the upper eyelid is detected.

  Since the frame number fn = 1, the process proceeds from step 218 to step 229, and the distance between the line connecting the positions of both eyes detected in step 214 and the edge 250 of the upper eyelid detected in step 216 is calculated. This value is stored as the distance d0.

  Thereafter, in step 230, the center point of the line connecting the eyes of the frame number fn = 1 is set as the reference point C. Further, N × M pixels centered on the reference point C are saved in the storage area as a reference point pattern (step 232). Thereafter, the processing returns to step 212.

  In step 212, 1 is added to the frame number fn. That is, here, the frame number fn = 2. Then, the image of the second frame is read. Thereafter, the processes of steps 214 and 216 are performed in the same manner as the first frame.

  Since the determination result in step 218 is NO, the process proceeds to step 220, where the reference point pattern in the first frame moves in the second frame by template matching using the saved reference point pattern as a template. And the corresponding area is extracted. By this process, a new reference point pattern 254 (see FIG. 6) is determined, and therefore, it is also determined where the reference point C of the reference point pattern in the first image has moved in the second image. This point is set as a new reference point C (step 220). Based on the upper eyelid edge 252 of the eye detected in step 216, the new reference point C, and the position of both eyes detected in step 214, a straight line passing through the reference point C parallel to the straight line passing through the center of both eyes. And a distance d1 between the upper eyelid edge and the upper eyelid edge (step 222).

  The distance d1 between the distance d0 and the distance d1 is calculated using the distance d1 obtained for the second frame in this way and the distance d0 calculated in the process of step 229 for the first frame. d0-d1 is calculated (step 224).

  If this difference d is greater than 1, it is determined that there has been a blink in the second frame (YES in step 226). After performing the necessary processing in step 228, the process proceeds to step 229. If the difference d is 1 or less, nothing is done and the process proceeds to step 229. Thereafter, the same processing as that performed for the first frame is performed in steps 229, 230, and 232, and the process returns to step 212.

  The above-described processing is repeatedly executed for frames having a frame number fn = 3 or less, whereby the necessary processing is performed in step 228 for the frame where blinking occurs.

  FIG. 7 shows the change in the value of d in both eyes obtained by experiments. As shown in FIG. 7, in the experiment, the change 270 in the value of the distance d of the right eye and the change 272 in the value of the distance d of the left eye are substantially synchronized. This is because blinking is generally performed at the same time for both eyes.

  With the blink detection device according to the present embodiment, the occurrence of blinks can be detected based on a predetermined reference line determined by the eye position and the distance to the edge of the upper eyelid. According to this method, it is not necessary to detect the edge of the lower eyelid that becomes relatively unstable. Therefore, the occurrence of blinking can be detected only by the edge of the upper eyelid that can be detected stably. As a result, it is possible to provide a blink detection device that can stably detect the occurrence of blinks from a moving image of a human face.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

It is a schematic diagram which shows the state of human blink. It is a figure shown about the method of monitoring a motion of an upper eyelid. 1 is an external view of a car navigation system that realizes a blink detection device according to an embodiment of the present invention. FIG. 4 is a block diagram of the car navigation system shown in FIG. 3. It is a flowchart at the time of implement | achieving the blink detection apparatus which concerns on this Embodiment with a computer. It is a photograph shown about the specific example of operation | movement with the detection of the edge of an upper eyelid, and the detection of blink using the reference point pattern. It is a photograph which shows the change of the value of d in both eyes.

Explanation of symbols

84 Inter-eyes pattern 90 Reference point pattern 150 Car navigation system 160 Car navigation system 162 Liquid crystal screen 164 Touch panel 166 and 204 Speaker 168 Camera 170 CD-ROM drive 190 CPU
192 ROM
194 RAM
196 Camera I / F
198 Bus 200 Sound board 202 D / A converter

Claims (6)

A blink detection device for detecting blink from a moving image of a face including a series of frames,
Eye position tracking means for tracking the positions of both eyes of the face in the series of frames;
Eyelid detecting means for detecting an edge of the upper eyelid in the series of frames;
In the face image of the preceding frame, a first distance from a predetermined reference line determined by the eye position specified by the eye position tracking unit to the edge of the upper eyelid detected by the eyelid detection unit First distance calculating means for calculating;
A new reference line having a predetermined relationship with the predetermined reference line in the face image of the preceding frame is identified by the eye position tracking means in the face image of the frame subsequent to the preceding frame. A reference line specifying means for specifying based on the position of
Second distance calculation means for calculating a second distance from the new reference line to the edge of the upper eyelid detected by the eyelid detection means in the face image of the subsequent frame;
Blink detection means for detecting the occurrence of blink based on the difference between the distances calculated by the first and second distance calculation means ,
The reference line specifying means includes
A reference pattern extracting means for extracting and storing a region of a prescribed shape as a reference pattern, including a reference point having a predetermined relationship with the position of both eyes in the face image of the preceding frame;
Template matching for estimating the position of the reference point of the preceding frame in the subsequent frame by template matching for the subsequent frame using the reference pattern extracted by the reference pattern extraction unit as a template Means,
A line that passes through the position of the reference point estimated in the subsequent frame by the template matching means and is parallel to a straight line connecting the centers of both eyes in the subsequent frame is specified as the new reference line. And a blink detection device.   The first distance calculation unit is configured to connect a line connecting the positions of the eyes and the upper eyelid detected by the eyelid detection unit based on the positions of both eyes specified by the eye position tracking unit in the preceding frame. The blink detection device according to claim 1, comprising means for calculating a distance between the edge of the blink. The reference pattern extracting means extracts and stores the region of the prescribed shape including the reference point as a reference pattern, with the midpoint position of the line segment connecting the positions of both eyes in the face image of the preceding frame as a reference point The blink detection device according to claim 1 , comprising the following means. The means for calculating the distance is based on the position of both eyes specified by the eye position tracking means in the preceding frame, and a line connecting the positions of both eyes and the eye detected by the eyelid detection means. Means for calculating the distance between a predetermined point in the edge of the eyelid;
The second distance calculation means calculates a distance between the new reference line and a point corresponding to the predetermined point in the upper eyelid edge detected by the eyelid detection means in the subsequent frame. The blink detection device according to claim 2, comprising means for The blink detection device according to claim 4 , wherein the predetermined point in the edge of the upper eyelid is a point that gives a maximum value of a distance between a line connecting the positions of both eyes and the edge of the upper eyelid. A computer program that, when executed by a computer having a function of processing a moving image as a series of frames, causes the computer to operate as a blink detection device according to any one of claims 1 to 5 .

Images