JP4912206B2 - Image processing method, image processing apparatus, image processing system, and computer program - Google Patents

Description

  The present invention relates to an image processing method for setting a search region in which an eye is to be searched from an image including a face and glasses, and detecting the position of the eye in the search region, an image processing apparatus for executing the image processing method, and the image The present invention relates to an image processing system including a processing device and a computer program for causing a computer to function as the image processing device.

  An operation state monitoring system that monitors the operation state of a vehicle such as an automobile has been proposed. The driving state monitoring system is configured by an imaging device, an image processing device, and the like installed at appropriate places where a driver's face can be imaged. The imaging device captures the driver's face, and the image processing device detects the driver's eyes from the captured image to detect the driver's sleep (for example, Patent Document 1). By using the driving state monitoring system, it is possible to detect a driver's drowsy driving, and it is also possible to configure a system that advises a driver who is driving a nap with an alarm.

  As a device for detecting an eye, a dozing state detection device that detects a local increase in luminance change in the vertical direction of an image and a general position such as an eye and an eyebrow on a face has been proposed (for example, Patent Document 1).

  However, since the dozing detection device according to Patent Document 1 is configured to detect an eye from a change in luminance and a general arrangement of eyes, when the driver is wearing glasses, not only the eye but also the luminance is detected. Eyeglass frames, eyeglass bridges, and the like whose changes approximate the eyes are also included, making eye position detection difficult.

An image processing system in consideration of the above problems has been proposed (for example, Patent Document 2). The imaging device of the image processing system according to Patent Literature 2 is installed diagonally forward of the driver to image the spectacle vine, and images the driver from an oblique direction. The image processing device detects a spectacle vine from an image of a driver taken from an oblique direction. Then, the image processing apparatus sets a predetermined area separated by a predetermined distance in a predetermined direction on the basis of the end portion of the spectacle vine on the butterfly plate side, and detects the eye position.
JP-A-10-040361 JP 2006-209442 A

  However, the positional relationship between the spectacle parts and the eyes varies depending on the person and varies depending on the orientation of the face. Therefore, there are cases where the eyes cannot be detected in a uniquely determined search area.

  The present invention has been made in view of such circumstances, because the relative distance of the eye to the spectacle component tends to change according to the width of the face part, the width between the plurality of face parts or the width of the face region, By setting the search area so that the relative distance to the spectacle parts becomes long / short according to the wide / narrow width, it is possible to set the search area according to the individual difference of the driver or the face direction. An image processing method capable of accurately detecting the position of an eye, an image processing device that implements the image processing method, an image processing system including the image processing device, and a computer program that causes a computer to function as the image processing device The purpose is to provide.

  Another object of the present invention is to select a search area having a size corresponding to the width of the face part, the width between a plurality of face parts, or the width of the face area, thereby allowing individual differences in the driver or the face. It is an object of the present invention to provide an image processing apparatus that can set a search area according to the orientation of the eye and accurately detect the position of the eye, and an image processing system including the image processing apparatus.

  Another object of the present invention is to exclude the detected spectacle component region from the search region, thereby preventing erroneous detection of the spectacle component as an eye and accurately detecting the eye. An image processing apparatus and an image processing system including the image processing apparatus are provided.

  Another object of the present invention includes storage means for storing the position of a face, spectacle parts, and eyes detected from one image in the past, and the position of the eye in one image and the position of the search region in another image. By configuring the search area so as to correspond to each other, it is possible to appropriately set the search area according to the individual difference of the driver and the orientation of the face, and to accurately detect the eye position And an image processing system including the image processing apparatus.

  Another object of the present invention is to search according to the individual difference of the driver or the face direction by detecting the spectacle bridge, spectacle frame or spectacle vine by focusing on the characteristics of each component and executing the above-described processing. An object of the present invention is to provide an image processing apparatus capable of setting a region and accurately detecting the position of an eye, and an image processing system including the image processing apparatus.

  An image processing method according to a first aspect of the present invention is an image processing method for setting a search area where an eye is to be searched from an image including a face and glasses, and detecting the position of the eye from the search area. Detecting a part or face region to calculate a face part, a plurality of face parts or a width of the face area, detecting a position of the spectacle part from the image, and being separated from the detected spectacle part, And a step of setting a search area so that a relative distance to the spectacle component becomes long / short according to the wide / narrow width.

  An image processing apparatus according to a second aspect of the present invention is an image processing apparatus for setting a search area where an eye is to be searched from an image including a face and glasses, and detecting the position of the eye from the search area. Region width calculating means for detecting a part or a face area to calculate a face part, between a plurality of face parts or the width of the face area, spectacle part detecting means for detecting the position of the spectacle part from the image, and the detected spectacle part And setting means for setting a search area so that a relative distance to the spectacle component becomes long / short according to the wide / narrow of the width.

  The image processing apparatus according to a third aspect is characterized in that the setting means sets a search area having a dimension corresponding to the width.

  According to a fourth aspect of the present invention, there is provided an image processing apparatus comprising: means for excluding a spectacle part area detected by the spectacle part detection means from the search area when the spectacle part is included in the search area. .

  An image processing apparatus according to a fifth aspect of the present invention is configured to acquire images in time series, and includes a facial part detection unit that detects a facial part other than an eye from each image, and a facial part detected from one image. Storage means for storing eyeglass parts and eye positions, and the setting means includes the positions of face parts and eyeglass parts detected from other images, and the face parts and eyeglass parts stored in the storage means. And a means for comparing the positions of the eyes and setting the search area so that the position of the eye in one image corresponds to the position of the search area in another image.

An image processing apparatus according to a sixth aspect of the invention is characterized in that the spectacle component includes a spectacle bridge, a spectacle frame, or a spectacle vine.
According to a seventh aspect of the present invention, there is provided an image processing apparatus comprising a face direction detecting means for detecting a face direction, wherein the eyeglass component detecting means detects a spectacle vine when detecting that the detecting means is in a landscape orientation. It is comprised by these.

An image processing system according to an eighth invention includes the image processing device according to any one of the second to sixth inventions, and an imaging device that images a driver of the vehicle. The imaging device is configured to detect the position of the eye from an image obtained by imaging.

A computer program according to a ninth invention causes a computer to execute the steps according to the first invention.

In the first, second, eighth , and ninth inventions, the position of the spectacle component is detected, and the search area is set based on the position of the spectacle component. The positional relationship between the eyeglass component and the eye varies depending on individual differences, face orientation, etc., and the relative position of the eye with respect to the eyeglass component also depends on the width of the face part, the width between the face parts, or the width of the face area. There is a tendency to be long / short. Therefore, by setting the search area so that the relative distance to the spectacle component becomes longer / shorter according to the wide / narrow width, the search area including the eyes can be set more appropriately, and eye detection can be performed. Accuracy can be improved. That is, the search area is set so that the relative distance becomes longer when the width of the face part, the width between the plurality of face parts or the width of the face area is wide, and the relative distance becomes shorter when the width is narrow. Thus, the eye detection accuracy can be improved.

In the third and eighth inventions, since the size of the eye tends to change according to the width of the face part, the width between the plurality of face parts, or the width of the face area, the setting means has the above width. A search area having a corresponding size is set. Therefore, the search area including the eyes can be set more appropriately, and the eye detection accuracy can be improved.

In the fourth and eighth inventions, when a spectacle part is included in the search area, for example, when a spectacle bridge or spectacle frame is included, the spectacle part may be erroneously detected as an eye. The spectacle parts detected by the detecting means are excluded from the search area. Accordingly, erroneous eye detection can be prevented and eye detection accuracy can be improved.

In the fifth and eighth inventions, the image processing apparatus acquires image data in time series, and the face part detection means, the spectacle part detection means, etc. detect the face part, spectacle parts, and eyes from each image. And a memory | storage means memorize | stores the position of the face part, spectacles components, and eyes which were detected from one image, ie, the past image. Then, the setting means compares the positional relationship between the face part, the spectacle part, and the eye in the image at the past position with another image, that is, the face part and the spectacle part detected from the current image. A search area is set such that the position of the eye at and the position of the search area in another image correspond to each other.
Therefore, it is possible to set the search area including the eyes more accurately and detect the eyes with high accuracy irrespective of the driver's posture, face orientation, and the like.

In the sixth , seventh, and eighth inventions, a spectacle bridge, a spectacle frame, or a spectacle vine is detected as a spectacle component, and a search area is set and an eye is detected. Note that the spectacle bridge, the spectacle frame, or the spectacle vine can be detected separately from the eye by extracting the feature.

In the first, second, eighth, and ninth inventions, the distance from the spectacle component is the distance between the spectacle component, the width of the face portion, the width between the face portions, or the width of the face region. Accordingly, by setting the search area so as to be longer and shorter, it is possible to set the search area according to the individual difference of the driver or the direction of the face, and to accurately detect the eye position.
In particular, a means for calculating a luminance average value in a predetermined area with reference to the spectacle part, a means for calculating a luminance dispersion value in the predetermined area, and the presence / absence of sunglasses based on the calculated luminance average value and the luminance dispersion value. If it is configured to detect the position of the eye from the search area when it is determined that there is no sunglasses, it effectively prevents false detection caused by trying to detect the position when the eye cannot be detected. can do.

According to the third and eighth aspects of the present invention, it is possible to select a search area having a size corresponding to the width of the face part, the width between the plurality of face parts, or the width of the face area, so that the individual difference of the driver Alternatively, a search area can be set according to the orientation of the face, and the eye position can be accurately detected.

According to the fourth and eighth aspects of the invention, it is possible to accurately detect the eye by excluding eyeglass parts that are likely to be erroneously detected as eyes from the search area.

According to the fifth and eighth aspects of the present invention, the face part, the spectacle position and the eye position stored for the past image are compared with the face part and spectacle component positions detected from the image of the current frame. By setting the search area to be searched, the search area can be appropriately set according to the individual difference of the driver and the orientation of the face, and the eye position can be accurately detected.

According to the sixth , seventh, and eighth inventions, the search area corresponding to the individual difference of the driver or the face direction is set by detecting the spectacle bridge, the spectacle frame, or the spectacle vine and executing the above-described processing. The position of the eye can be detected accurately.
In addition, it is provided with a face direction detecting means for detecting the face direction, and when detecting that the detecting means is in landscape orientation, the spectacle component detecting means is configured to detect the spectacle vine. Since the position of the eyeglass vine can be detected, it is possible to prevent erroneous detection of the position of the eyeglass vine and the eye by omitting useless detection processing.
Means for storing a threshold value for detecting the spectacle component; means for calculating a detection rate of the spectacle component by the spectacle component detection means; and means for increasing or decreasing the threshold according to the calculated calculation rate of the spectacle component. In this case, it is possible to improve the accuracy of detection of spectacle parts and eyes by setting an appropriate threshold according to the characteristics of the spectacle parts.

Hereinafter, the present invention will be described in detail with reference to the drawings illustrating embodiments thereof.
FIG. 1 is a block diagram illustrating a configuration example of an image processing system according to an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes an imaging device mounted on a vehicle, and the imaging device 1 is connected to an image processing device 2 that performs image processing via a communication network. The communication network is configured by, for example, a communication line such as a dedicated cable, or a wired or wireless in-vehicle LAN (Local Area Network).

The imaging device 1 is disposed in front of a driver such as a steering wheel or a dashboard in a vehicle, and can capture an image so that the horizontal and vertical directions of the driver's face are the horizontal and vertical directions of the image. Has been adjusted.
The imaging apparatus 1 includes an MPU (Micro Processor Unit) 11 that controls the entire apparatus, various computer programs executed based on the control of the MPU 11 and a ROM (Read Only Memory) 12 that records data, and a computer recorded in the ROM 12. By a RAM (Random Access Memory) 13 for recording various data temporarily generated during the execution of the program, an imaging unit 14 configured using an imaging device such as a CCD (Charge Coupled Device), and the imaging of the imaging unit 14 An A / D converter 15 that converts the obtained analog image data into digital data, a frame memory 16 that temporarily records the image data digitally converted by the A / D converter 15, and an image processing device 2 And a communication interface 17 used for communication.

  In the imaging apparatus 1, the imaging unit 14 performs imaging processing continuously or intermittently, generates, for example, 30 pieces of image data (image frames) per second based on the imaging processing, and sends the image data to the A / D converter 15. Output. The A / D converter 15 converts each pixel constituting the image into digital image data indicated by a gradation such as 256 gradations (1 Byte) and records the digital image data in the frame memory 16. The image data recorded in the frame memory 16 is output from the communication interface 17 to the image processing apparatus 2 at a predetermined timing.

Each pixel constituting the image is two-dimensionally arranged, and the image data includes the position of each pixel indicated by a plane rectangular coordinate system, a so-called XY coordinate system, and the luminance of each pixel indicated as a gradation value. Data (luminance value) to be shown is included. The horizontal direction of the image corresponds to the X-axis direction of the image data, and the vertical direction of the image corresponds to the Y-axis direction of the image data.
The coordinates may be shown in the order in which they are arranged in the data, instead of showing the coordinates in the XY coordinate system for each pixel. In this embodiment, image processing based on grayscale image data will be described. When color image data is used, similar image processing may be executed on the luminance data of the pixels constituting the image data.

  The image processing apparatus 2 is mounted with a computer, a CPU (Central Processing Unit) 21 that controls the entire apparatus, a computer program 31 according to the embodiment of the present invention, various threshold values for detecting spectacle parts and eyes, A recording medium 41 that records various information such as predetermined values, for example, an auxiliary recording unit 22 that reads information from a CD-ROM, a hard disk (hereinafter referred to as HD) 23 that records various information read by the auxiliary recording unit 22, and HD 23 A RAM 24 for recording various data temporarily generated when the recorded computer program 31 is executed, a frame memory 25 composed of a volatile memory, and a communication interface 26 used for communication with the imaging apparatus 1 are provided. ing.

The computer program 31 of the present invention includes at least a computer that detects a face area from an image including a face and glasses G, calculates a face width, detects a position of a spectacle component from the image, and detects detected glasses And a program for executing a process of setting a search region so that the relative distance to the spectacle part becomes long / short according to the wide / narrow face width. A computer program 31 is recorded on the recording medium 41 so as to be readable by a computer.
Needless to say, the computer program 31 includes one distributed from an external computer via a wired or wireless communication network, and the image processing apparatus 2 acquires the computer program 31 via the communication network and records it in the HD 23. You may do it.

  The CPU 21 reads various information such as the computer program 31 and data of the present invention from the HD 23, records the information in the RAM 24, and executes various processing procedures included in the computer program 31. It operates as the processing device 2.

  The image processing device 2 receives the image data output from the imaging device 1 through the communication interface 26, records the received image data in the frame memory 25, reads out the image data recorded in the frame memory 25, and performs various images. Process.

  Next, processing contents of various devices used in the image processing system according to the embodiment of the present invention will be described.

  FIG. 2 is a flowchart showing a processing procedure of the CPU 21 for detecting eyes from an image including a face and glasses G. First, the CPU 21 of the image processing apparatus 2 acquires image data from the frame memory 25 in time series (step S11). And CPU21 detects a driver | operator's face area from the acquired image data (step S12). Further, the CPU 21 detects a facial part other than the eyes, for example, a nostril (step S13).

Next, the CPU 21 executes a first spectacle component recognition process (step S14). In the first spectacle component recognition process, the spectacle bridge H, the spectacle upper frame I, and the spectacle vine J are detected (see FIG. 8). And CPU21 performs the 2nd spectacles component recognition process (step S15). In the second spectacle component recognition process, the lower spectacle frame K and the upper spectacle frame I are detected (see FIGS. 26 and 29), and it is determined whether or not it is sunglasses. When the eyeglass component recognition process is finished, the CPU 21 executes the eyeglasses learning process and the eyeglass determination process using the recognition result (step S16). In the spectacle learning process, the acquisition rate of each spectacle part and the position of the spectacle part are stored. It should be noted that the acquisition rate and the like of each spectacle part may be stored separately according to the orientation of the face. In addition, various threshold values and predetermined values for detecting each spectacle part or face part may be changed according to the acquisition rate of each spectacle part. The shape of each spectacle part or face part, brightness change, and the like vary depending on the type of spectacle G and the person, but the accuracy of spectacle part or face part detection can be improved by setting an optimum threshold with reference to the acquisition rate. it can.
For example, when the acquisition rate of the spectacles bridge H is 95% and the acquisition rate of the spectacles frame I is 60%, the spectacles G are present and the threshold value for detecting the spectacles frame I is appropriate. Therefore, there is a possibility that the frame I on the glasses is not detected. In such a case, the spectacles frame I can be detected more accurately by changing the threshold value for detecting the spectacles frame I so as to facilitate detection.
Conversely, when the acquisition rate of the spectacles bridge H is 0% and the acquisition rate of the spectacles frame I is 15%, the spectacles G are not present and the threshold value for detecting the spectacles frame I is detected. Is not appropriate, there is a possibility that the frame I on the glasses is erroneously detected. In such a case, the spectacles frame I can be detected more accurately by changing the threshold value for detecting the spectacles frame I so that erroneous detection does not occur.
In the eyeglass determination process, the presence or absence of the eyeglasses G is comprehensively determined based on the acquisition rate of each eyeglass component.

  Next, the CPU 21 determines whether or not sunglasses are present (step S17). If the eye is concealed by sunglasses, the eye cannot be detected, and the upper or lower part of the sunglasses lens may be erroneously recognized as an eye, but the problem of erroneous recognition may be avoided by the processing in step S17. it can.

  If it is determined that there is sunglasses (step S17: YES), the CPU 21 ends the process related to eye detection. When it is determined that there is no sunglasses (step S17: NO), the CPU 21 sets a search area excluding the spectacle part area (step S18), and detects the eye position from the search area (step S19). Then, the CPU 21 executes eye re-detection processing (step S20). In the eye re-detection process, a search area that is separated in a predetermined direction from the position of the detected spectacle component is set. In particular, the CPU 21 sets the search area so that the relative distance of the search area with respect to the spectacle parts increases or decreases according to the face width and the size of the search area also increases or decreases in consideration of individual differences, face orientation, and the like. In addition, a search area is set using past recognition results regarding eyeglass parts, a face part, and an eye position. Then, the eyes are detected again from the set search area.

  Finally, the CPU 21 determines a more reliable eye position from among the eye positions detected in steps S19 and S20, and detects the face part detected in step S13 and steps S14 and S15. The positional relationship between each eyeglass component and the eye position is registered in the HD 23 (step S21). Note that the registration process may be appropriately executed at another timing. The CPU 21 updates the positional relationship recorded in the HD 23 when detecting the face part, each spectacle part, and the eye position in a state where the face direction is closer to the front side. The method for detecting the face orientation is a known technique, and for example, it is determined by the position of the nose with respect to the face area in the horizontal direction. Moreover, you may comprise so that it may judge with the shape of an eye. Further, a positional relationship having a high appearance frequency may be recorded as a positional relationship at the time of front.

Hereinafter, the processing content described using the flowchart of FIG. 2 will be described more specifically.
FIG. 3 is an explanatory diagram conceptually showing a face area detection method. In step S12, the CPU 21 integrates the luminance values of the pixels arranged in the horizontal direction for each Y coordinate of the image, and compares the luminance integrated value obtained by the integration with a predetermined threshold value to thereby determine the face area in the vertical direction. The range of R0 is detected. Fy1 indicates the Y coordinate of the upper end of the face region R0 in the vertical direction, and Fy2 indicates the Y coordinate of the lower end of the face region R0 in the vertical direction. Since the face region R0 is imaged brighter than the hair region and the background region, the face region R0 can be detected by comparing the luminance integrated value with a threshold value.

  Similarly, the CPU 21 integrates the luminance values of the pixels arranged in the vertical direction with respect to the X coordinates of the image, and compares the luminance integrated value obtained by the integration with a predetermined threshold value to thereby determine the face area in the horizontal direction. The range of R0 is detected. Fx1 indicates the left end of the face region R0 in the horizontal direction, and Fx2 indicates the right end of the face region R0 in the horizontal direction. Note that “left” means the left as viewed from the imaging apparatus 1, that is, the left side of the sheet of FIG. 3, and “right” means the right side of the sheet of FIG. Therefore, when viewed from the side of the imaged driver, the left and right are reversed.

  FIG. 4 is an explanatory diagram conceptually showing a nose position detection method. In FIG. 4, an outer rectangular frame indicated by a solid line indicates the entire image, and a rectangular frame drawn by a broken line surrounding the nose inside the rectangular frame indicates a detection processing range R <b> 1 for detecting the nose. ing.

  In step S13, the CPU 21 integrates the luminance values of the pixels arranged in the horizontal direction so as to detect the detection processing region R1 for detecting the nose, and derives the change in the luminance integrated value in the vertical direction from the integrated result. Then, the CPU 21 detects a plurality of Y coordinate positions indicating the minimum value from the change in the derived luminance integrated value in the vertical direction. By this processing, a plurality of detection target candidates including eyebrows, eyes, nose, and mouth having a low average luminance in the horizontal direction are detected. Next, the CPU 21 detects both ends of the detection target candidate from the change in the luminance value of the pixels arranged in the horizontal direction in the Y coordinate indicating the detected minimum value, and detects the minimum point in the horizontal direction. In the periphery of the nose, as shown by “x” marks, both ends of the nose and two local minimum points corresponding to the nostrils are detected. The CPU 21 specifies the detection processing range around the nose based on the ratio of the face width Fw = Fx2−Fx1 to the lengths of both ends of the nose in the horizontal direction, the number of minimum points, and the like. Next, the CPU 21 selects the vicinity of the minimum point as a nostril search region from the change in luminance value of the pixels arranged in the horizontal direction, and detects the nostril position using a black region extraction filter or the like.

Next, the processing contents of the first spectacle component recognition according to step S14 will be specifically described.
FIG. 5 is a flowchart showing a processing procedure of the CPU 21 relating to the first spectacle component recognition. First, the CPU 21 detects the spectacle bridge H (step S31), and detects the spectacle upper frame I (step S32). Then, the CPU 21 determines whether or not the detection process for the left and right eyeglass parts has been completed (step S33). If it is determined that the left or right eyeglass component detection process has not been completed (step S33: NO), the CPU 21 returns the process to step S32. If it is determined that the left and right eyeglass parts detection processing has been completed (step S33: YES), the CPU 21 detects the eyeglass vine J (step S34), and ends the processing procedure relating to the first eyeglass part recognition. Next, the processing contents of steps S31 to S34 will be described more specifically.

  6 and 7 are flowcharts showing the processing procedure of the CPU 21 related to the detection of the eyeglass bridge H. The CPU 21 determines whether or not the left and right eye positions detected in the past are stored (step S51). In the description, the left and right directions are directions viewed from the imaging device 1, that is, the right side in FIG. If it is determined that the left and right eye positions are stored (step S51: YES), a detection processing range of the eyeglass bridge H is calculated based on the left and right eye positions (step S52). When it is determined that the left and right eye positions are not stored (step S51: NO), the CPU 21 calculates a detection processing range of the eyeglass bridge H based on the nose position (step S53).

  FIG. 8 is an explanatory diagram conceptually showing the detection processing range of the eyeglass bridge H. FIG. 8A shows the detection processing range R2 calculated based on the left and right eye positions. In step S52, as shown in FIG. 8A, the midpoint of the left and right eye positions in the horizontal direction is set as a reference point P1, N1 pixels vertically above the reference point P1, N2 pixels below, and horizontally A region having a spread of N3 pixels on both sides is calculated as a detection processing range R2. N1, N2, and N3 are determined by a fixed ratio with respect to the face width Fw. By configuring the detection processing range R2 to be wider or narrower depending on the face width Fw, an appropriate detection processing region R2 can be calculated regardless of the distance between the imaging device 1 and the driver. Specifically, when the driver is close to the imaging device 1, the detection processing range R2 is expanded, and when the driver is far from the imaging device 1, the detection processing range R2 is narrowed.

  FIG. 8B shows a detection processing range calculated based on the position of the nostril. In step S53, as shown in FIG. 8B, the position of the N4 pixel is set to the reference position P1 from the midpoint P0 of the left and right nostrils in the horizontal direction to the upper side in the vertical direction. A detection processing range R2 is calculated.

FIG. 9 is an explanatory diagram conceptually showing a method for detecting the upper and lower positions of the eyeglass bridge H. FIG. 9A shows an example of the image area of the eyeglass part.
When the process of step S52 or step S53 is finished, the CPU 21 detects each pixel arranged in the horizontal direction in the detection processing range R2 as shown in FIG. Are integrated (step S54). The vertical axis of the graph in FIG. 9B indicates the Y coordinate in the image data, and the horizontal axis indicates the luminance integrated value obtained by integrating the luminance values in the horizontal line of each X coordinate. The luminance integrated value indicates the brightness of the horizontal line at each Y coordinate. When the eyeglass bridge H exists, as shown in FIG. 9B, the integrated luminance value of the spectacle bridge H portion in the vertical direction Y falls lower than the other portions.

Next, the CPU 21 calculates an upper and lower difference value of the luminance integrated value as shown in FIG. 9C (step S55). The vertical axis of the graph shown in FIG. 9C indicates the Y coordinate in the image data, and the horizontal axis indicates the vertical difference value at each Y coordinate. The vertical difference value is calculated by, for example, the following formula 1.
Vertical difference value = M1 (y + predetermined pixel) −M1 (y).
However, y represents the Y coordinate in one horizontal line, and M1 (y) represents the luminance integrated value in the horizontal line.

  Then, the CPU 21 detects the Y coordinate having the maximum vertical difference value as the upper position of the glasses bridge H (step S56), and detects the Y coordinate having the minimum vertical difference value as the lower position of the glasses bridge H (step S57). .

  FIG. 10 is an explanatory diagram conceptually showing a method for detecting the left and right position candidates of the eyeglass bridge H. FIG. 10A shows an example of the image area of the eyeglass part, as in FIG. When the processing of step S57 is completed, the CPU 21 integrates the luminance values of the pixels arranged in the vertical direction in the detection processing range R2 as shown in FIG. Step S58). The horizontal axis of the graph of FIG. 10B represents the X coordinate in the image data, and the vertical axis represents the luminance integrated value obtained by integrating the luminance value in the vertical direction of each X coordinate. The integrated luminance value indicates the brightness of the vertical line at the X coordinate. When the spectacles bridge H exists, as shown in FIG. 10B, the integrated luminance value of the connection portion between the spectacles bridge H and the spectacles upper frame I falls lower than the other portions.

Next, the CPU 21 calculates the left-right difference value of the luminance integrated value as shown in FIG. 10C (step S59). The horizontal axis of the graph shown in FIG. 10C indicates the X coordinate in the image data, and the vertical axis indicates the left-right difference value at each X coordinate. The left-right difference value is calculated by, for example, the following formula 2.
Left-right difference value = M2 (x + predetermined pixel) −M2 (x) Equation 2
However, x represents the X coordinate of one vertical line, and M2 (x) represents the luminance integrated value at the pixel position x in the vertical line.

  Then, the CPU 21 detects the X coordinate having the maximum left-right difference value on the left side in the horizontal direction from the reference point P1 as a left position candidate of the eyeglass bridge H (step S60), and the left-right difference on the right side in the horizontal direction from the reference point P1. The X coordinate having the smallest value is detected as the right position candidate of the eyeglass bridge H (step S61).

  Next, the CPU 21 determines the presence or absence of the eyeglass bridge H based on the detection results of steps S56, 57, 60, 61 (step S62). Specifically, the sum of the absolute value of the upper and lower difference values of the upper position of the spectacle bridge H and the absolute value of the upper and lower difference values of the lower position is equal to or greater than a first predetermined threshold value, and the right position candidate of the spectacle bridge H is selected. The sum of the absolute value of the left / right difference value and the absolute value of the left / right difference value of the left position candidate is equal to or greater than a second predetermined threshold, and the distance between the left position candidate of the glasses bridge H and the right position candidate is the face width Fw. Is within the threshold range, it is determined that there is a glasses bridge H. That is, it is determined whether or not the upper position, the lower position, and the left and right position candidates of the eyeglass bridge H are not in an unnatural positional relationship as the eyeglass bridge H.

  When it is determined in step S62 that the glasses bridge H is present (step S62: YES), the CPU 21 stores the position coordinates of the middle point H3, the left end point H1, and the right end point H2 of the eyeglass bridge H (step S63). The processing related to the detection of is finished. The first threshold value and the second threshold value may be changed by learning the detection result of the eyeglass bridge H. Although the ease of detection of the spectacle bridge H differs depending on the spectacles G, an optimal threshold value corresponding to the type of spectacles G is obtained by increasing or decreasing the first and second threshold values based on the acquisition rate of the spectacle bridge H. And the spectacles bridge H can be accurately detected.

  FIG. 11 is a schematic diagram showing the middle point H3, the left end point H1, and the right end point H2 of the eyeglass bridge H.

  The X coordinate of the middle point H3 of the eyeglass bridge H is the coordinate of the middle point between the left position candidate and the right position candidate of the eyeglass bridge H in the horizontal direction, and the Y coordinate is the coordinate of the upper position of the eyeglass bridge H. The X coordinate of the left end point H1 of the spectacle bridge H is the coordinate of the left position candidate of the spectacle bridge H, and the Y coordinate is the midpoint between the upper position coordinate and the lower position coordinate of the spectacle bridge H. The X coordinate of the right end point H2 of the eyeglass bridge H is the coordinate of the right position candidate of the eyeglass bridge H, and the Y coordinate is the midpoint between the upper position coordinate and the lower position coordinate of the eyeglass bridge H.

  FIGS. 12 and 13 are flowcharts showing the processing procedure of the CPU 21 related to the first spectacles frame detection, and FIG. 14 is an explanatory diagram conceptually showing the left end point candidate and the right end point candidate of the spectacles upper frame I. . First, the CPU 21 calculates the left and right end point candidates I10 and I20 of the upper frame I based on the positions of the middle point H3, the left and right end points H1 and H2 and the face region R0 of the eyeglass bridge H (step S71).

  As shown in FIG. 14, the X coordinate of the right end point candidate I20 of the spectacles upper frame I is the X coordinate of the left end point H1 of the spectacles bridge H, and the Y coordinate is the Y coordinate of the midpoint H3 of the spectacles bridge H. The X coordinate of the left end point candidate I10 of the spectacles upper frame I is the X coordinate Fx1 of the left end point of the face region R0, and the Y coordinate is the Y coordinate of the midpoint H3 of the spectacle bridge H.

  Next, the CPU 21 calculates the position coordinates of the left end point I1 and the right end point I2 obtained by correcting the position coordinates of the left end point candidate I10 and the right end point candidate I20 of the upper spectacle frame I (step S72).

  Next, the CPU 21 sets a detection processing range for detecting the vertex of the upper frame I (step S73).

  FIG. 15 is an explanatory diagram conceptually showing a detection processing range R3 for detecting the vertex I3 of the upper frame I. As shown in FIG. 15, the detection processing range R3 is a rectangular region, and is a region having the left end point I1 and the right end point I2 of the spectacles upper frame I as both ends in the horizontal direction and appropriately extending in the vertical direction above and below. is there. More specifically, the X coordinate of the upper left end R31 of the detection processing range R3 is the X coordinate value of the left end point I1 of the upper frame, and the Y coordinate is the uppermost point of the left end point I1 and the right end point I2, that is, This is a coordinate value obtained by subtracting the predetermined value N5 from the Y coordinate value having the smallest value. Further, the X coordinate of the lower right end R32 of the detection processing range R3 is the X coordinate of the right end point I2 of the upper spectacle frame I, and the Y coordinate is the lowest point of the left end point I1 and the right end point I2, that is, the most. This is a coordinate value obtained by adding a predetermined value N6 to the Y coordinate value having a large value. N5 and N6 are fixed ratios with respect to the face width Fw.

  Next, the CPU 21 performs predetermined image processing on the image data, for example, the image data of the face region R0, and creates a processed image (step S74). Specifically, a filter process using a Laplacian filter is performed on the image data, and the filtered image is binarized. The threshold used for binarization is set so that the ratio between the number of pixels having a luminance value equal to or higher than the threshold in the processing range and the number of pixels having a luminance value less than the threshold is a predetermined ratio. Then, noise removal processing such as isolated point removal is executed. By the processing in step S74, a binary image in which a strong edge is 1 and a weak area is 0 can be obtained.

Next, the CPU 21 calculates a glasses bridge score for evaluating the presence or absence of the glasses bridge H (step S75).
FIG. 16 is an explanatory diagram for explaining a re-evaluation processing method for the eyeglass bridge H. The CPU 21 determines the content rate of the luminance value 1 in the image region having the predetermined width N7 up and down from the straight line connecting the left end point H1 and the right end point H2 of the spectacle bridge H, which is the image processed in step S74. Calculate as a score. The content rate is the ratio of pixels having a luminance value of 1 to the number of pixels constituting the image area.

Next, the CPU 21 determines whether or not the eyeglass bridge H is present based on the eyeglass bridge score (step S76). It is re-evaluated whether the spectacles bridge H exists by judging whether a content rate is more than a predetermined threshold value. If the eyeglass bridge H exists, an edge component exists in the vicinity of the eyeglass bridge H and appears as a pixel having a luminance value of 1 in the image processed in step S74. By the process of step S76, the presence or absence of the eyeglass bridge H can be accurately determined by the above-described eyeglass bridge H detection process and the above-described re-evaluation process.
The presence or absence of the eyeglass bridge H is evaluated using not only the eyeglass bridge score of the current image data but also the eyeglass bridge score of the past image data. Specifically, the eyeglass bridge score of the current image data is equal to or greater than a threshold value, and the number of image data having the eyeglass bridge score equal to or greater than the threshold value among the image data of the first predetermined number of frames is equal to or greater than the second predetermined number. In this case, it is determined that there is a glasses bridge H.

  FIG. 17 is an explanatory diagram conceptually showing a method for searching for the vertex position of the frame I on the eyeglasses. If it is determined in step S76 that the eyeglass bridge H is present (step S76: YES), a vertex score for evaluating the vertex I3 of the eyeglass bridge H is calculated while scanning the detection processing range R3 (step S77). Specifically, the luminance sum of the glasses upper frame I on the line segment connecting the left end I1 and the right end I2 of the glasses upper frame I and the vertex I3 is calculated as a vertex score. Then, the CPU 21 specifies the position of the vertex I3 having the maximum vertex score (step S78), and determines whether or not the maximum value of the vertex score is equal to or greater than a predetermined threshold (step S79). If it is determined that the threshold is equal to or greater than the threshold (step S79: YES), the CPU 21 stores the position coordinates of the left end I1, the right end I2, and the vertex I3 of the upper spectacle frame I (step S80), and relates to the detection of the upper spectacle frame I. Finish the process. Further, when it is determined in step S76 that there is no spectacles bridge H (step S76: NO), and when it is determined that the maximum value of the vertex score is less than the threshold value (step S79: NO), the CPU 21 determines that the frame I on the spectacles is The process ends as if it was not detected.

  FIG. 18 is a flowchart showing a processing procedure of the CPU 21 relating to the detection of the spectacle vine J. The CPU 21 determines whether or not a predetermined detection processing condition for detecting the spectacle vine J is provided (step S91). If the spectacle vine J flag indicating whether or not the spectacle vine J can be acquired is 1, and the face direction is right or left, it is determined that the detection processing condition is satisfied. For example, the spectacle vine J flag is set to 1 when the acquisition rate of the spectacle bridge H is equal to or greater than a predetermined value in the spectacle learning process, and is set to 0 when the acquisition rate is less than the predetermined value. Even if the acquisition rate of the spectacle bridge H is equal to or higher than a predetermined value, if the acquisition rate of the spectacle vine J is poor, the spectacle vine J flag is set to 0. By configuring so as to determine whether or not it is necessary to detect the spectacle vine J according to the orientation of the face, useless detection processing can be omitted and the detection accuracy of the spectacle vine J can be improved.

  When it is determined that the detection processing condition is satisfied (step S91: YES), the CPU 21 calculates a detection processing range R6 of the spectacle vine J (step S92).

FIG. 19 is an explanatory diagram conceptually showing a method of calculating the spectacle vine J detection processing range R6. First, the CPU 21 calculates a reference range R4 as shown by a solid rectangular frame in FIG. The reference area R4 is the upper half of the face area R0. That is, the left and right ends and upper end of the reference region R4 coincide with the left and right ends and upper end of the face region R0, and the lower end of the reference region R4 is an intermediate position between the upper and lower ends of the face region R0.
Next, the CPU 21 limits the vertical range to a certain range as shown by a solid rectangular frame in FIG. Specifically, the CPU 21 limits the range to a range R5 that has an N81% spread of the face width Fw up and down around the Y coordinate axis of the midpoint of the eye position detected in the past. When there is no eye point midpoint detected in the past, the upper and lower ranges may be set with reference to the midpoint H1 of the eyeglass bridge H.

  Then, the CPU 21 limits the horizontal range to a certain range as shown by a solid rectangular frame in FIG. Specifically, when the face is facing left as shown in FIG. 19C, the CPU 21 determines the range having the width of N82% of the face width Fw from the right end of the range R5 as the detection processing range R6 of the spectacle vine J. And When the face is directed to the right, a range having a width of N82% of the face width Fw from the left end of the range R5 is set as the spectacle vine J detection processing range R6.

  When the processing in step S92 is completed, the CPU 21 calculates a Y position table in which each X coordinate in the detection processing range R6 is associated with the Y coordinate of the pixel estimated as the spectacle vine J (step S93).

FIG. 20 is an explanatory diagram conceptually showing a method of calculating the Y position table. First, the CPU 21 specifies a pixel estimated as the spectacle vine J from a plurality of pixels arranged in the Y direction at the left end of the detection processing range R6, and stores the Y coordinate of the pixel. Thereafter, the CPU 21 sequentially scans the same processing in the X direction from the left end to the right end of the detection processing range R6, and executes it on each Y direction line.
More specifically, as shown in FIG. 20, the vertical luminance difference and the straight direction differential value are calculated for each pixel in the Y direction line whose X coordinate is x1. For example, with respect to the target pixel whose Y coordinate is y1, the luminance difference between the upper and lower sides, that is, the pixel whose Y coordinate is y1 + Δy and the pixel whose Y coordinate is y1−Δy, and the differential value in the vertical direction are calculated. Then, the target pixel having the luminance difference equal to or greater than the threshold value and the maximum differential value in the vertical direction is specified as the pixel estimated as the spectacle vine J, and the X coordinate and Y coordinate (hereinafter referred to as the Y position) of the pixel. Is stored in the Y position table. The Y position table is information in which a plurality of consecutive X coordinate values are associated with Y positions corresponding to the respective X coordinates.

When calculating the Y position table, it is preferable to execute a noise removal process.
FIG. 21 is an explanatory diagram conceptually showing the noise removal method according to the Y position table. The three vertical lines indicate the processing lines L1, L2, and L3 for extracting the spectacle vine J, and the CPU 21 extracts the pixels of the spectacle vine J in order from the left processing line L1 to the right processing line L3. The black square marks indicate the pixels calculated as the positions of the spectacle vines J in the respective lines L1, L2, and L3, and the white square marks indicate the pixels after the noise removal processing. Specifically, the CPU 21 determines that the difference between the Y position in the processing line L2 and the Y position in the processing line L3 is equal to or greater than a first predetermined value, and the Y position in the processing line L1 and the Y position in the processing line L1. When the difference is less than the second predetermined value, the Y coordinate of the spectacle vine pixel in the processing line L2 is corrected to an intermediate value between the Y position in the processing line L1 and the Y position in the processing line L3 as shown in FIG. .

FIG. 22 is an explanatory diagram conceptually showing a method for detecting the left end and the right end of the spectacle vine J. The CPU 21 refers to the Y position table created in step S93, identifies spectacle vine pixels having continuity, and calculates the positions of the left end point J1 and right end point J2 of the longest continuous line (step S94).
Specifically, the CPU 21 refers to the Y position table and determines that there is continuity when the difference between the Y positions in adjacent X coordinates is equal to or smaller than a predetermined threshold. Conceptually, when the difference between the Y positions of the spectacle vine pixels adjacent in the horizontal direction, for example, the difference in Y coordinate between the spectacle vine pixels whose X coordinate is x1 and the spectacle vine pixels whose X coordinate is x1 + 1 is less than a predetermined threshold value. Each eyeglass vine pixel is determined to be continuous.

Then, when there are a plurality of spectacle vine pixel groups having continuity, the CPU 21 specifies the spectacle vine pixel group having the maximum number of consecutive pixels, and sets the minimum X coordinate and the maximum X coordinate in the spectacle vine pixel group. The X coordinate of the left end point candidate and right end point candidate of the spectacle vine J is stored. However, when the difference between the X coordinate of the left end point candidate and the X coordinate of the right end point candidate is equal to or smaller than a predetermined threshold, that is, when the length as the spectacle vine J is too short, it is determined that there is no spectacle vine J.
When the X coordinate of the left end point candidate and the right end point candidate is calculated without determining that there is no spectacle vine J, the CPU 21 refers to the Y position table and determines the X coordinate and Y position between the left end point candidate and the right end point candidate. The approximate straight line of the spectacle vine J is calculated using the least square method (step S95).
Then, the CPU 21 calculates the Y coordinate of the left end point candidate using the approximate line from the X coordinate of the left end point candidate, and calculates the Y coordinate of the right end point candidate using the approximate line from the X coordinate of the right end point candidate (step). S96).

  Next, the CPU 21 evaluates the spectacle vine J candidates, and determines that the spectacle vine J is the left end point J1 and the right end point J2 when the left end point candidate and the right end point candidate of the spectacle vine J satisfy the predetermined condition. The coordinates of the left and right end point candidates are determined as the coordinates of the left end point J1 and the right end point J2, and stored (step S97). The first predetermined condition is that the horizontal angle of the left end point candidate and the right end point candidate is within a predetermined angle range, and the second predetermined condition is that the average of residuals of the approximate curve is not more than a predetermined threshold value. That is, it is determined whether or not the spectacle vine J is substantially linear and the angle with respect to the horizontal direction is within a natural range as the spectacle vine J.

Next, the processing contents of the second spectacle component recognition according to step S15 will be specifically described.
FIG. 23 is a flowchart illustrating a processing procedure of the CPU according to the second spectacle component recognition. The CPU 21 detects the lower spectacles frame K using the differential filter (step S111), and detects the upper spectacles frame I (step S112). Then, a luminance average value and a luminance dispersion value are calculated to determine whether or not it is sunglasses (step S113).

  Next, the CPU 21 determines whether or not the detection process has been completed for the left and right frames (step S114). If it is determined that the detection process has not been completed for the left and right frames (step S114: NO), the CPU 21 returns the process to step S111. When it is determined that the detection process has been completed for the left and right frames (step S114: YES), the CPU 21 determines whether the sunglasses are worn using the result of the sunglasses determination for the past image data. Is executed (step S115), and the processing relating to the second spectacle component recognition is completed.

  FIG. 24 is a flowchart illustrating the processing procedure of the CPU according to the second glasses lower frame detection. The CPU 21 calculates a detection processing range R7 for detecting the lower spectacles frame K (step S131).

FIG. 25 is an explanatory diagram conceptually showing the detection processing range of the glasses lower frame K. As shown in FIG. As shown in FIG. 25A, the horizontal range of the detection processing range R7 is a range of N9% of the face width Fw to the left and right with the eye position detected in the past as the center. That is, the X coordinate of the left end of the detection processing range R7 is Ex−Fw × N9 / 100, and the X coordinate of the right end of the detection processing range R7 is Ex + Fw × N9 / 100, where X is the X coordinate of the eye position. .
The lower end of the detection processing range R7 is a position N10% below the face width Fw from the midpoint P0 of the nostril. That is, the Y coordinate of the lower end of the detection processing range R7 is the Y coordinate of the middle point P0 + Fw × N10 / 100. The upper end of the detection processing range R7 is the Y coordinate Ey of the eye position E.

  When the detection processing range R7 is calculated, the CPU 21 performs a differential filter extraction process using a differential filter in the detection processing range R7 (step S132). FIG. 25B shows an example of the coefficients of the differential filter. The differential filter is a filter that extracts an image region having a high luminance from the top to the bottom in the vertical direction, an image region having a low luminance, and a lower frame portion that changes to an image region having a high luminance. The CPU 21 calculates differential filter values in order from the lower side to the upper side of the detection processing range R7 in one Y-direction line, and detects a pixel that first becomes a predetermined threshold value or more. The differential filter value is a numerical value obtained by multiplying eight neighboring pixels centered on the target pixel by the coefficients of the differential filter and adding the values obtained by multiplication.

  In addition, during the differential filter process, when the pixel one pixel above the target pixel, for example, the Y coordinate of the target pixel is y1, the luminance of the pixel whose Y coordinate is y1-1 is equal to or higher than the first predetermined luminance, an external light source Is determined to be a portion where the spectacle lens is reflecting, and the pixel (hereinafter referred to as a reflection partial pixel) is detected. The first predetermined luminance is 240 with 255 gradations, for example. The external light source is an LED light source installed in the vehicle to illuminate the driver for various purposes such as face recognition.

  In addition, when the luminance of the target pixel at the start of the differential filter processing, that is, the luminance of the pixel one pixel above or below the target pixel in the lower portion of the detection processing range R7 is at least the second predetermined luminance, It is determined that the brightness tends to be bright, and the first predetermined luminance is increased. For example, the second predetermined luminance is 210 at 256 gradations, and the first predetermined luminance is increased from 240 to 250. The above processing enables more accurate detection of the reflection portion.

  When the above-described processing is completed for one Y-direction line, the CPU 21 sequentially executes similar processing for other Y-direction lines from the left side to the right side in the detection processing region R6. Note that, in accordance with the horizontal width of the detection processing range R7, the Y-direction line that is the target of the differential filter processing is thinned out, and a maximum of n results are calculated. n may be set as appropriate in consideration of the processing speed and the determination accuracy.

FIG. 26 is an explanatory diagram conceptually showing the lower position of the glasses lower frame K. FIG.
When the differential filter extraction processing is finished, the CPU 21 detects the lower position of the glasses lower frame K (step S133), and finishes the processing relating to the glasses lower frame detection. The lower position of the glasses lower frame K is a position indicated by a cross in FIG. Specifically, when a plurality of pixels are detected in step S132, the CPU 21 stores the intermediate value of the Y coordinates of each pixel among the detected n pixels as the Y coordinate of the lower position. In addition, the CPU 21 stores the X coordinate of the substantially horizontal center of the detection processing range R8 as the X coordinate of the lower position.

If there is no pixel detected in step S132, it is determined whether or not a reflection partial pixel has been detected. When it is determined that the reflection partial pixel is detected, the Y coordinate of the reflection partial pixel is stored as the Y coordinate of the lower position. When there are a plurality of reflection partial pixels, the maximum Y coordinate is stored as the lower Y coordinate.
In the case of the glasses G without the lower frame, the glasses lower frame K is not detected, but when the reflection part is detected, it is considered that the glasses G are present in the reflection part. Remember as.
If there is no pixel detected in step S132 and no reflection partial pixel is detected, the CPU 21 determines that there is no under-glasses frame K and ends the process.

  Note that an average value of the differential filter values in the n detected pixels is calculated and stored as a frame score under the glasses. Based on the lower spectacle frame score, it can be determined whether the detected lower position is the spectacle G or the eye. When the spectacles lower frame score is larger than the first threshold, the CPU 21 determines that the spectacles are G, and when the spectacles lower frame score is less than the second threshold less than the first threshold, the CPU 21 determines that the eyes are spectacles.

  FIG. 27 is a flowchart illustrating a processing procedure of the CPU according to the second frame on eyeglasses detection. The CPU 21 calculates a detection processing range R8 for detecting the upper frame I (step S151).

  FIG. 28 is an explanatory diagram conceptually showing the detection processing range R8 for the upper frame I. In the CPU 21, the horizontal range of the detection processing range R8 is the same as the detection processing range R8 of the lower frame as shown in FIG. The lower end of the detection processing range R8 is the lower position of the glasses lower frame K. The upper end of the detection processing range R8 is a position that is a predetermined number% above the face width Fw from the upper end of the detection processing range R8 of the glasses lower frame K.

  When the detection processing range R8 is calculated, the CPU 21 executes differential filter extraction processing using a differential filter in the detection processing range R8 (step S152). FIG. 28B shows an example of the coefficients of the differential filter. The differential filter is a filter that extracts the upper frame I portion of the spectacles that changes from an image area with high luminance to an image area with low luminance from the lower side to the upper side in the vertical direction. The CPU 21 calculates the differential filter value in order from the lower side to the upper side of the detection processing range R8 in one Y-direction line, and detects a pixel that first becomes a predetermined threshold value or more. The differential filter value is a numerical value obtained by multiplying eight neighboring pixels centered on the target pixel by the coefficients of the differential filter and adding the values obtained by multiplication.

When the above-described processing is finished for one Y-direction line, the CPU 21 executes the same processing for other Y-direction lines. It should be noted that, depending on the horizontal width of the detection processing range R8, the Y direction lines to be subjected to differential filter processing are thinned out, and a maximum of n results are calculated.
FIG. 29 is an explanatory diagram conceptually showing the upper position of the spectacles upper frame I.
When finishing the differential filter extraction process, the CPU 21 detects the upper position of the spectacle upper frame I (step S153), and ends the process related to the spectacle upper frame detection. The upper position of the spectacles upper frame I is a location indicated by a cross in FIG. Specifically, when a plurality of pixels are detected in step S152, the CPU 21 stores the intermediate value of the Y coordinates of each pixel among the detected n pixels as the Y coordinate of the upper position. In addition, the CPU 21 stores the X coordinate of the approximate center in the horizontal direction of the detection processing range R8 as the X coordinate of the upper position.

  FIG. 30 is a flowchart showing the processing procedure of the CPU related to the determination of sunglasses, and FIG. 31 is an explanatory diagram conceptually showing the determination method of sunglasses. CPU21 turns off the sunglasses flag which shows the determination result of whether it is sunglasses in the present flame | frame (step S171). Then, the CPU 21 calculates the luminance average and luminance dispersion value within the sunglasses processing range (step S172). The upper and lower sides of the sunglasses processing range are the upper position I4 and the lower position K1 of the upper spectacle frame I, and the left and right of the sunglasses processing range are the same as the processing range of the lower spectacle frame K.

  Next, the CPU 21 determines whether or not the average brightness value is equal to or less than a threshold value (step S173). When it is determined that the average brightness value is equal to or less than the threshold value (step S173: YES), the CPU 21 determines whether the variance value of the brightness values is equal to or less than the threshold value (step S174). When it determines with it being below a threshold value (step S174: YES), CPU21 turns on a sunglasses flag and memorize | stores the value of a sunglasses flag (step S175).

  When the process of step S175 is finished, when it is determined that the luminance average value is larger than the threshold value (step S173: NO), when it is determined that the variance value is larger than the threshold value (step S174: NO), the CPU 21 It is determined whether the left and right sunglasses determination has been executed (step S176). If it is determined that the left / right or either one of the sunglasses determination is not executed (step S176: NO), the CPU 21 returns the process to step S171. When it is determined that the left and right sunglasses determination has been executed, the CPU 21 ends the process related to the sunglasses determination.

FIG. 32 is a flowchart showing the processing procedure of the CPU according to the sunglasses comprehensive determination. CPU21 reads the sunglasses flag of the past predetermined number of frames (step S191). Then, the CPU 21 determines whether or not the sunglasses flag is on in the predetermined number of frames in the number of frames equal to or greater than the threshold (step S192). If it is determined that the frame is ON in the frame equal to or greater than the threshold (step S192: YES), the comprehensive sunglasses flag indicating the result of comprehensively determining the presence or absence of sunglasses is turned on (step S193), and the processing related to the sunglasses comprehensive determination is completed. . When it is determined that the frame number is less than the threshold value and is ON (step S192: NO), the CPU 21 turns off the general sunglasses flag (step S194) and ends the process related to the sunglasses general determination.
In the result of only one frame indicated by the sunglasses flag, there is a possibility that the sunglasses may be erroneously recognized due to instantaneous luminance change, hand approach, etc., but by comprehensively determining using the sunglasses determination result in a plurality of frames, Such erroneous recognition can be prevented.

When the process related to the second spectacle component recognition is finished, as shown in FIG. 2, the CPU 21 executes the spectacle learning and the spectacle determination process (step S16).
Specifically, the CPU 21 records the success or failure of detection of the spectacle vine J, the spectacle upper frame I, the spectacle lower frame K, the spectacle bridge H, and the positions of the spectacle components.
In addition, the CPU 21 calculates an acquisition rate of each spectacle part in a predetermined number of frames in the past, that is, a detection success rate, and records it in the HD 23. The acquisition rate of the spectacle parts is calculated for each spectacle part. This is because the spectacles lower frame K is not detected depending on the spectacles G, but the spectacles bridge H can be detected.
And CPU21 determines the presence or absence of the spectacles G based on the acquisition rate of each spectacles part. For example, when there is one or more spectacle parts with an acquisition rate equal to or higher than a predetermined acquisition rate, it is determined that there is spectacle G. Further, when the face is sideways, the presence / absence of the glasses G is determined using the acquisition rate of the glasses vine J, and when the face is frontward, the presence / absence of the glasses G is determined using the acquisition rate of the glasses bridge. You may do it.

Next, the processing content of steps S18 and S19 shown in FIG. 2 will be described.
If it is determined in step S16 that the glasses G are present, the CPU 21 sets the search area by excluding the area of the eyeglass parts.

FIG. 33 is an explanatory diagram conceptually showing a state in which the spectacle part region is excluded from the eye search region. For example, when the face region R0 is set as the eye search region, the CPU 21 sets an image region obtained by excluding the region of each spectacle part from the search region as the search region. The region of the eyeglass bridge H is, for example, an image region having a predetermined width up and down from a straight line connecting the left end point H1 and the right end point H2 of the eyeglass bridge H. The region of the spectacles upper frame I has a predetermined width in the vertical direction around the upper position I4 of the spectacles upper frame I, and the horizontal range is the same as the detection processing region R7. The region of the glasses lower frame K has a predetermined width in the vertical direction with the lower position of the glasses lower frame K as the center, and the horizontal range is the same as the detection processing region R7. The region of the spectacle vine J is a rectangular region having the left end point J1 and the right end point J2 of the spectacle vine J as vertices.
The region of the upper spectacle frame I is calculated based on the coordinates detected by the second spectacle component recognition process, but is configured to be calculated based on the coordinates detected by the first spectacle component recognition process. May be.

And CPU21 detects eyes by template matching etc. in the search area | region where the spectacles component was excluded. Alternatively, the eye may be detected by detecting an upper eyelid or a lower eyelid whose brightness changes in the vertical direction by edge filter processing.
By excluding the spectacle part region from the eye search area, it is possible to prevent erroneous detection of the spectacle part as an eyelid.

Next, the eye re-detection process in step S20 will be described.
FIG. 34 is a flowchart illustrating the processing procedure of the CPU related to eye redetection. First, the CPU 21 sets a search area S based on the position of the spectacle vine J, and executes eye re-detection processing (step S211). The relative distance of the search area S with respect to the spectacle vine J is set to be proportional to the face width Fw, and the dimension of the search area S is also set to be proportional to the face width Fw.

Hereinafter, the processing content of step S211 is demonstrated concretely.
FIG. 35 is a flowchart showing the processing procedure of the CPU related to eye re-detection using the detection result of the spectacle vine J, and FIG. 36 conceptually shows a search area set based on the position of the spectacle vine J. FIG. The CPU 21 calculates the face width Fw (step S231), and calculates the vertical width Sh of the eye estimation area based on the face width Fw (step S232). Further, the CPU 21 calculates the lateral width Sw of the eye estimation area based on the face width Fw (step S233). The horizontal width Sw and the vertical width Sh of the eye estimation area are proportional to the face width Fw.

Then, the CPU 21 sets the position of W11% left side of the lateral width Sw of the eye estimation area from the left end point J1 of the spectacle vine J as the X coordinate of the right end of the search area S (step S234). When the X coordinate of the left end point J1 of the spectacle vine J is J1x, the X coordinate of the right end of the search area S is represented by J1x−Sw × W11 / 100.
Next, the CPU 21 sets, as the X coordinate of the left end of the search area S, the position on the left side W12% of the lateral width Sw of the eye estimation area from the left end point J1 of the spectacle vine J (step S235). The X coordinate of the left end of the search area S is represented by J1x-Sw × W12 / 100.

Next, the CPU 21 sets a position 11% below the vertical width Sh of the eye estimation area from the left end point J1 of the spectacle vine J as the X coordinate of the upper end of the search area S (step S236). When the Y coordinate of the left end point J1 of the spectacle vine J is J1y, the Y coordinate of the upper end of the search area S is represented by J1y + Sh × H11 / 100.
Next, the CPU 21 sets a position 12% lower than the left end point J1 of the eyeglass vine J in the vertical direction Sh of the eye estimation region as the Y coordinate of the lower end of the search region S (step S237). The Y coordinate of the lower end of the search area S is represented by J1y + Sh × H12 / 100.

  Then, the CPU 21 detects an eye from the search area S set as shown in FIG. 36 (step S228), and ends the process related to eye redetection using the detection result of the spectacle vine J.

  When the eye re-detection process related to the spectacle vine J is completed as shown in FIG. 34, when the spectacle bridge H is detected, the CPU 21 stores the spectacle parts, the face part and the eye position recorded in the HD 23 in the past, A search area is set based on the eyeglass bridge H detected in the current frame and the positional relationship between the facial parts, and eye re-detection processing is executed (step S212).

Hereinafter, the processing content of step S212 is demonstrated concretely.
FIG. 37 is a flowchart of the CPU related to eye re-detection using the detection result of the eyeglass bridge H, and FIG. 38 is an explanatory diagram conceptually showing a search area set using the detection result of the eyeglass bridge H. .

  First, the CPU 21 reads the position coordinates of the eyeglass bridge H, both eyes, and the nostril registered in the past, that is, recorded in the HD 23 (step S251).

  Next, the CPU 21 registers the positional relationship between the eyeglass bridge H, both eyes and nostril registered as shown in FIG. 38A, and the positional relationship between the eyeglass bridge H and nostril detected from the image data of the current frame. Then, the coordinates (Ex, Ey) of the eye redetection position E in the current frame are calculated (step S252). Even when the position, orientation, etc. of the face are changing, there is a certain relationship between the registered positional relationship between each part and each part in the current frame. Therefore, the CPU 21 calculates a conversion formula for converting the coordinates of each registered part into the coordinates of each part in the current frame. The CPU 21 calculates the eye position in the current frame as the coordinates of the re-detection position using the calculated conversion formula and the registered positions of both eyes.

  Next, the CPU 21 calculates the face width Fw (step S253), and calculates the vertical width Sh of the eye estimation region based on the face width Fw (step S254). Further, the CPU 21 calculates the lateral width Sw of the eye estimation area based on the face width Fw (step S255). The horizontal width Sw and the vertical width Sh of the eye estimation area are proportional to the face width Fw.

  Next, as shown in FIG. 38B, the CPU 21 sets the position 13 W to the right of the lateral width Sw of the eye estimation area from the redetection position E as the X coordinate of the right end of the search area (step S <b> 256). ), The position on the left side of W13% is set as the X coordinate of the left end of the search area (step S257). The X coordinate at the right end of the search area is expressed as Ex + Sw × W13 / 100, and the X coordinate at the left end is expressed as J1x−Sw × W13 / 100.

  Next, the CPU 21 sets the position of H13% above the vertical width Sh of the eye estimation area from the redetection position Ey as the Y coordinate of the upper end of the search area (step S258), and sets the position below H13% of the search area. The Y coordinate of the lower end is set (step S259). The Y coordinate of the upper end of the search area is represented by Ey−Sh × H13 / 100, and the Y coordinate of the lower end of the search area is represented by Ey + Sh × H13 / 100.

  Then, the CPU 21 detects an eye from the set search region as shown in FIG. 38B (step S260), and ends the process related to eye redetection using the detection result of the eyeglass bridge H.

  As shown in FIG. 34, when the eye re-detection process related to the spectacle bridge H is completed, when the upper frame I is detected, the CPU 21 detects the learning result of the spectacle parts and the facial part and the current frame. The search area is set based on the frame I on the eyeglasses and the positional relationship between the face parts, and eye re-detection processing is executed (step S213).

  FIG. 39 is an explanatory diagram conceptually showing a search area set using the detection result of the frame I on the glasses. The processing content is the same as the eye redetection processing related to the eyeglass bridge H, and the same processing as steps S251 to S259 is executed using the position of the spectacles upper frame I instead of the position of the eyeglass bridge H.

  First, the CPU 21 reads the positions of the glasses upper frame I, both eyes, and the nostril registered in the past. In particular, the CPU 21 reads the positions of the eyeglass bridge H, both eyes, and the nostril when the driver is facing the front side.

  Then, as shown in FIG. 39 (a), the positional relationship between the registered glasses upper frame I, both eyes and nostrils, and the positional relationship between the glasses upper frame I and nostrils detected from the image data of the current frame. Based on the above, the coordinates (Ex, Ey) of the eye redetection position E in the current frame are calculated.

  Next, an area having a width of Sw × W13 on the left and right and Sh × H13 on the top and bottom with respect to the redetection position is set as a search area. Then, the CPU 21 redetects the eyes from the set search area.

  When the eye re-detection process related to the spectacles upper frame I is finished, as shown in FIG. 34, the CPU 21 determines whether or not the eye is re-detected (step S214). If it is determined that the eye has not been redetected (step S214: NO), the CPU 21 ends the process relating to eye redetection. When it is determined that the eye has been re-detected (step S214: YES), the CPU 21 evaluates the eye positions detected in steps S19, 211, 212, and 213, respectively, and is estimated to be more correct as the eye position. Is selected (step S215), and the processing relating to eye redetection is completed. For example, the quality of the eye position is determined from past learning results, the positional relationship between the eyes, and the like.

  According to the image processing method, the image processing apparatus 2, the image processing system, the computer program 31, and the recording medium 41 according to the embodiment configured as described above, the CPU 21 is likely to be erroneously detected as an eye in steps S14 and S15. Since the configuration is such that a spectacle part is detected and a search area excluding the spectacle part is set in steps S18 and S19 and eyes are detected from the search area, erroneous detection of the spectacle part as an eye is effectively prevented. Thus, the eyes can be detected accurately.

  In steps S211 to S213, the CPU 21 is separated from the spectacle component so that the relative distance to the spectacle component becomes longer / shorter according to the width / narrowness of the face width, and the dimensions are wide / short. Since the search area is set to be large / small according to the narrowness, the search area can be set according to the individual difference of the driver or the face direction, and the eye position is accurately detected. be able to.

  Further, in steps S212 and 213, the CPU 21 compares the face part, glasses G position and eye position previously registered in the HD 23 with the face part and eyeglass component positions detected from the current frame image. Since the eye is detected by setting the search area so that the position of the eye in one image in the past corresponds to the position of the search area in the other image at the present, the individual difference of the driver, the orientation of the face And the search area can be set appropriately, and the eye position can be accurately detected.

  In the embodiment, the relative distance with respect to the spectacle component is long / short according to the wide / narrow face width, and the size is large / small according to the wide / narrow face width. The search area is set so that the relative distance to the spectacle component is long depending on the width of a face part such as the eyes and nose, the width between a plurality of face parts, for example, the width between two eyes. The search area may be set so as to be shorter / shorter and the size to be larger / smaller depending on the width. Even in such a configuration, it is possible to set a search area corresponding to the individual difference of the driver or the face direction, and to accurately detect the position of the eyes.

(Appendix 1)
In an image processing method for setting a search area where an eye should be searched from an image including a face and glasses, and detecting an eye position from the search area,
Detecting one or a plurality of face parts or face regions from an image and calculating a face part, between a plurality of face parts or a width of the face area;
Detecting the position of the spectacle part from the image;
And a step of setting a search area so as to be long / short relative to the spectacle component according to the wide / narrow width of the spectacle component. Method.

(Appendix 2)
In an image processing apparatus for setting a search area where an eye is to be searched from an image including a face and glasses, and detecting the position of the eye from the search area,
Area width calculating means for detecting one or a plurality of face parts or face areas from an image and calculating a face part, between a plurality of face parts or a width of the face area;
Spectacle component detection means for detecting the position of the spectacle component from the image;
A setting means for setting a search area so as to be long / short relative to the spectacle part according to the width / narrowness of the spectacle part. Processing equipment.

(Appendix 3)
The setting means sets the search area so that the relative distance to the spectacle part is long when the width is wide, and sets the search area so that the relative distance to the spectacle part is short when the width is narrow. The image processing apparatus according to Supplementary Note 2, which is a feature.

(Appendix 4)
The setting means includes
The image processing apparatus according to appendix 2 or appendix 3, wherein a search area having a size corresponding to the width is set.

(Appendix 5)
The image processing according to appendix 2 or appendix 3, further comprising means for excluding the spectacle part area detected by the spectacle part detection means from the search area when the spectacle part is included in the search area apparatus.

(Appendix 6)
It is configured to acquire images in time series,
Face part detection means for detecting a face part other than the eye from each image;
Storage means for storing the face part, spectacle parts and eye position detected from one image,
The setting means includes
The position of the face part and eyeglass parts detected from another image and the face part, eyeglass parts and eye position stored in the storage means are compared, and the eye position in one image and the other image are compared. The image processing apparatus according to any one of appendices 2 to 5, further comprising means for setting the search region so that the position of the search region corresponds.

(Appendix 7)
The image processing apparatus according to any one of appendix 2 to appendix 6, wherein the spectacle component includes a spectacle bridge, a spectacle frame, or a spectacle vine.

(Appendix 8)
A face direction detecting means for detecting the face direction;
The spectacle component detection means includes
The image processing device according to appendix 7, wherein the image processing device is configured to detect a spectacle vine when it is detected that the detection means is in the horizontal direction.

(Appendix 9)
The spectacle component detection means includes spectacle bridge detection means for detecting a spectacle bridge,
The eyeglass bridge detection means includes:
Means for calculating a luminance integrated value obtained by integrating the luminance values of the pixels arranged in the first direction, and calculating a position of the eyeglass bridge in the first direction based on a change in the integrated value in the second direction different from the first direction; ,
Means for calculating a luminance integrated value obtained by integrating the luminance values of the pixels arranged in the second direction, and calculating a position of the eyeglass bridge in the second direction based on a change in the luminance integrated value in the first direction. The image processing apparatus according to appendix 7 or appendix 8.

(Appendix 10)
The spectacle component detection means includes spectacle frame detection means for detecting spectacle frames,
The spectacle frame detection means includes:
Means for extracting the edge of the spectacle frame with a Laplacian filter;
A means for calculating a correlation between the extracted edge and an approximate straight line or approximate curve including the detected spectacles bridge and the end of the face region, and means for specifying the approximate straight line or the curve based on the correlation. The image processing apparatus according to appendix 7 to appendix 9.

(Appendix 11)
The spectacle component detection means includes spectacle frame detection means for detecting spectacle frames,
The spectacle frame detection means includes:
Means for calculating a luminance difference between pixels arranged adjacent to each other in a predetermined direction;
A comparison means for comparing the calculated luminance difference and threshold;
The image processing apparatus according to any one of appendices 7 to 9, further comprising: a unit that calculates a position of the spectacle frame in a predetermined direction based on a comparison result of the comparison unit.

(Appendix 12)
The spectacle component detection means includes spectacle vine detection means for detecting spectacle vines,
The spectacle vine detection means comprises:
Means for calculating a luminance difference between pixels arranged adjacent to each other in a predetermined direction;
The image processing apparatus according to appendix 7 to appendix 11, comprising means for calculating a position of the spectacle vine based on a change in the calculated luminance difference.

(Appendix 13)
Means for storing a threshold for detecting spectacle parts;
Means for calculating a detection rate of spectacle parts by the spectacle part detection means;
The image processing apparatus according to any one of supplementary notes 2 to 12, further comprising means for increasing or decreasing the threshold according to a detection rate of the detected spectacle component.

(Appendix 14)
Means for calculating a luminance average value in a predetermined region with reference to the spectacle part;
Means for calculating a luminance dispersion value in the predetermined region;
Means for determining the presence or absence of sunglasses based on the calculated luminance average value and luminance variance value;
The image processing apparatus according to appendix 2 to appendix 13, wherein the image processing device is configured to detect the position of the eye from the search area when it is determined that there is no sunglasses.

(Appendix 15)
The image processing apparatus according to any one of Supplementary Notes 2 to 14, and
An imaging device for imaging a vehicle driver,
The image processing apparatus includes:
An image processing system configured to detect an eye position from an image obtained by imaging by the imaging device.

(Appendix 16)
In a computer program for setting a search area to search for an eye from an image including a face and glasses on a computer and detecting the position of the eye from the search area,
On the computer,
Detecting one or a plurality of face parts or face regions from an image and calculating a face part, between a plurality of face parts or a width of the face area;
Detecting the position of the spectacle part from the image;
And a step of setting a search area so as to be long / short relative to the spectacle part according to the wide / narrow width of the spectacle part. program.

(Appendix 17)
In a computer-readable recording medium in which a computer program for setting a search area to search for an eye from an image including a face and glasses on a computer and detecting the position of the eye from the search area is recorded.
On the computer,
Detecting one or a plurality of face parts or face regions from an image and calculating a face part, between a plurality of face parts or a width of the face area;
Detecting the position of the spectacle part from the image;
A computer program is recorded that is separated from the detected spectacle component and executes a step of setting a search area so that a relative distance to the spectacle component becomes long / short according to the width / narrowness of the width. A computer-readable recording medium characterized by the above.

It is a block diagram which shows the structural example of the image processing system in embodiment of this invention. It is a flowchart which shows the process sequence of CPU which detects eyes from the image containing a face and glasses. It is explanatory drawing which shows the detection method of a face area notionally. It is explanatory drawing which shows the nose position detection method notionally. It is a flowchart which shows the process sequence of CPU which concerns on 1st spectacles component recognition. It is a flowchart which shows the process sequence of CPU which concerns on the detection of a spectacles bridge. It is a flowchart which shows the process sequence of CPU which concerns on the detection of a spectacles bridge. It is explanatory drawing which shows notionally the detection process range of a spectacles bridge. It is explanatory drawing which shows notionally the detection method of the upper position of a spectacles bridge, and a lower position. It is explanatory drawing which shows notionally the detection method of the right-and-left position candidate of a spectacles bridge. It is a schematic diagram which shows the middle point of a spectacles bridge, a left end point, and a right end point. It is a flowchart which shows the process sequence of CPU which concerns on the 1st frame on spectacles detection. It is a flowchart which shows the process sequence of CPU which concerns on the 1st frame on spectacles detection. It is explanatory drawing which shows notionally the left end point candidate of a spectacles upper frame, and a right end point candidate. It is explanatory drawing which shows notionally the detection process range for detecting the vertex of a spectacles upper frame. It is explanatory drawing for demonstrating the reevaluation processing method of a spectacles bridge. It is explanatory drawing which shows notionally the search method of the vertex position of the spectacles upper frame. It is a flowchart which shows the process sequence of CPU which concerns on the detection of a spectacle vine. It is explanatory drawing which shows notionally the calculation method of the detection process range of a spectacles vine. It is explanatory drawing which shows notionally the calculation method of a Y position table. It is explanatory drawing which shows notionally the noise removal method which concerns on a Y position table. It is explanatory drawing which shows notionally the detection method of the left end of a spectacles vine, and a right end. It is a flowchart which shows the process sequence of CPU which concerns on 2nd spectacles component recognition. It is a flowchart which shows the process sequence of CPU which concerns on 2nd spectacles lower frame detection. It is explanatory drawing which shows notionally the detection process range of a spectacles lower frame. It is explanatory drawing which shows notionally the lower position of a spectacles lower frame. It is a flowchart which shows the process sequence of CPU which concerns on the 2nd frame on spectacles detection. It is explanatory drawing which shows notionally the detection processing range of the spectacles upper frame. It is explanatory drawing which shows notionally the upper position of a spectacles upper frame. It is a flowchart which shows the process sequence of CPU which concerns on determination of sunglasses. It is explanatory drawing which shows the determination method of sunglasses notionally. It is a flowchart which shows the process sequence of CPU which concerns on sunglasses comprehensive determination. It is explanatory drawing which shows notionally the state which excluded the area | region of the spectacles part from the search area | region of eyes. It is a flowchart which shows the process sequence of CPU which concerns on re-detection of eyes. It is a flowchart which shows the process sequence of CPU which concerns on the redetection of the eye using the detection result of a spectacle vine. It is explanatory drawing which shows notionally the search area | region set on the basis of the position of a spectacle vine. It is a flowchart of CPU which concerns on redetection of the eye using the detection result of a spectacles bridge. It is explanatory drawing which shows notionally the search area | region set using the detection result of the spectacles bridge. It is explanatory drawing which shows notionally the search area | region set using the detection result of the spectacles upper frame.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Image processing device 21 CPU
G Glasses H Glasses bridge I Glasses frame J Glasses vine K Glasses frame

Claims (9)

In an image processing method for setting a search area where an eye should be searched from an image including a face and glasses, and detecting an eye position from the search area,
Detecting one or a plurality of face parts or face regions from an image and calculating a face part, between a plurality of face parts or a width of the face area;
Detecting the position of the spectacle part from the image;
And a step of setting a search area so as to be long / short relative to the spectacle component according to the wide / narrow width of the spectacle component. Method. In an image processing apparatus for setting a search area where an eye is to be searched from an image including a face and glasses, and detecting the position of the eye from the search area,
Area width calculating means for detecting one or a plurality of face parts or face areas from an image and calculating a face part, between a plurality of face parts or a width of the face area;
Spectacle component detection means for detecting the position of the spectacle component from the image;
A setting means for setting a search area so as to be long / short relative to the spectacle part according to the width / narrowness of the spectacle part. Processing equipment. The setting means includes
The image processing apparatus according to claim 2, wherein a search area having a dimension corresponding to the width is set. 4. The apparatus according to claim 2, further comprising means for excluding a region of the spectacle part detected by the spectacle component detection unit from the search region when the spectacle component is included in the search region. 5. Image processing device. It is configured to acquire images in time series,
Face part detection means for detecting a face part other than the eye from each image;
Storage means for storing the face part, spectacle parts and eye position detected from one image,
The setting means includes
The position of the face part and eyeglass parts detected from another image and the face part, eyeglass parts and eye position stored in the storage means are compared, and the eye position in one image and the other image are compared. The image processing apparatus according to any one of claims 2 to 4, further comprising means for setting the search area so that the position of the search area corresponds to the position. The image processing device according to claim 2, wherein the spectacle component includes a spectacle bridge, a spectacle frame, or a spectacle vine. A face direction detecting means for detecting the face direction;
  The spectacle component detection means includes
  When detecting that the detection means is in landscape orientation, it is configured to detect a spectacle vine.
  The image processing apparatus according to claim 6. An image processing apparatus according to any one of claims 2 to 7 ,
An imaging device for imaging a vehicle driver,
The image processing apparatus includes:
An image processing system configured to detect an eye position from an image obtained by imaging by the imaging device. In a computer program for setting a search area to search for an eye from an image including a face and glasses on a computer and detecting the position of the eye from the search area,
On the computer,
Detecting one or a plurality of face parts or face regions from an image and calculating a face part, between a plurality of face parts or a width of the face area;
Detecting the position of the spectacle part from the image;
And a step of setting a search area so as to be long / short relative to the spectacle part according to the wide / narrow width of the spectacle part. program.

Images