JP2006038689A - Nose detection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a nose detection apparatus which improves accuracy in detecting a nose. <P>SOLUTION: The nose detection apparatus 1 photographs a face of a driver and performs edge detection for an image of the photographed face in longitudinal and lateral directions in the image. The nose detection apparatus 1 then detects a nose candidate from common points between detected edges in the longitudinal and lateral directions in the image and detects an eye candidate from edges in the lateral direction in the image. The nose detection apparatus 1 then combines the detected nose candidate and eye candidate, selects a combination which is valid as an eye-nose candidate from these combinations, and estimates the coordinates of the nose through statistical processing from the selected eye-nose candidate. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a nose detection device.

  2. Description of the Related Art Conventionally, there has been proposed an apparatus that detects a driver's doze, aside look, or abusive driving from a face image obtained by imaging a driver's face. In these devices, a driver's eyes are mainly detected and tracked from a face image, thereby detecting dozing, looking aside, or abusive driving.

  Further, in detecting and tracking the eyes, these devices detect the driver's nose, eyebrows, mouth, or facial contours, etc., and detect the eyes accurately from the relative positional relationship between these and the eyes. That is, not only the eyes but also the nose is detected, so that the eyes are correctly detected from the positional relationship between them.

Here, as a device for detecting a facial part other than the eye, for example, a device that detects a nostril by converting a gray value of an image and then floating and binarizing the nostril (patent) Reference 1). In addition, for each candidate point (black and round place in the face area) extracted by the skin color determination process, the candidate point is divided into the left and right from the center of the face area using the geometric arrangement condition for the positional relationship between both pupils. There is known a technique for detecting nostrils by narrowing down a combination of left and right (one set on the left and right) and calculating similarity with a nostril dictionary and a non-nasal nostril dictionary (see Patent Document 2).
JP-A-8-300978 JP 2001-67459 A

  However, all the conventional nose detection devices detect the nostrils as black circles, so if the lighting environment is not stable, such as when one day is plugged in, the nostrils will not be photographed as black circles and the nose It will interfere with the detection. Even if the light environment is stable, for example, if near-infrared illumination is installed below the front of the driver, the near-infrared light is inserted into the nostril, so the nostril is not photographed as a black circle, which is useful for detecting the nose. It will cause trouble. As described above, the conventional nose detection device still has room for improvement in the detection accuracy of the nostrils.

  The present invention has been made to solve such a conventional problem, and an object of the present invention is to provide a nose detection device capable of improving the detection accuracy of the nose.

  The nose detection device of the present invention includes an imaging unit, an edge detection unit, a nose candidate detection unit, an eye candidate detection unit, an eye nose candidate selection unit, and a nose coordinate estimation unit. Then, the photographing means photographs the driver's face, and the edge detecting means detects the edge of the face image photographed by the photographing means in the vertical and horizontal directions. The nose candidate detecting means detects a nose candidate from the common point of the vertical and horizontal edges detected by the edge detecting means, and the eye candidate detecting means detects the horizontal direction of the image detected by the edge detecting means. Eye candidates are detected from the edges. The eye / nose candidate selecting means combines the nose and eye candidates detected by the nose candidate detecting means and the eye candidate detecting means, and selects what is established as an eye / nose candidate from these combinations. The nose coordinate estimating means The coordinates of the nose are estimated by statistical processing from the candidates for the eye and nose selected by the eye nose candidate selecting means.

  According to the present invention, by selecting what can be established as an ocular nose as an ophthalmic nose candidate, the number of ophthalmic nose candidates including the actual nose is increased, and statistical processing is performed from such ophthalmic nose candidates. Since the nose is detected, the accuracy can be improved as compared with the case where only the nostril is detected alone. Therefore, it is possible to improve the detection accuracy of the nose.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described with reference to the drawings. In the drawings, the same or similar elements are denoted by the same reference numerals, and description thereof is omitted. In the following, an example in which the nose detection device according to the present embodiment is mounted on an automobile will be described.

  FIG. 1 is a configuration diagram of a nose detection device according to the present embodiment. As shown in the figure, the nose detection device 1 includes a camera (imaging means) 10, an image acquisition unit 20, and a nose detection unit 30.

  The camera 10 photographs the driver's face. This camera 10 has a structure in which a lens is combined with a CCD element or a CMOS element, and is installed slightly below the front of the driver. The angle of view of the camera 10 is about 20 to 40 degrees, and the driver's face can be captured within the angle of view. Further, the camera 10 is configured to transmit a video signal Sa obtained by photographing to the image acquisition unit 20 when photographing an image including the driver's head.

  The image acquisition unit 20 stores the video signal Sa from the camera 10 as a digital image in a storage area. Specifically, the image acquisition unit 20 converts the video signal Sa into, for example, digital data indicating grayscale data of 256 gradations per pixel with a horizontal width of 640 pixels and a vertical width of 480 pixels, and stores the digital data. Here, the digital data stored in the storage area is called face image data. The image acquisition unit 20 is configured to transmit face image data to the nose detection unit 30.

  The nose detection unit 30 detects from the face image data from the image acquisition unit 20 where the nose (nasal passage) is present in the face image by image processing and image recognition, and outputs it as nose coordinate data. is there.

  FIG. 2 is a configuration diagram illustrating another example of the nose detection device 1 according to the present embodiment. As shown in the figure, the nose detection device 1 may further include a near-infrared illumination 40 in addition to the camera 10, the image acquisition unit 20, and the nose detection unit 30. This near-infrared illumination 40 is used in dark places such as at night or in tunnels. When a dark shadow is produced on a part of the face under the strong environment of the western sun, the light environment may be affected by the sunbeams or entering or leaving the building. This is used to improve the light environment when changing in a short time.

  Further, at least one near infrared illumination 40 is arranged so that the entire face of the driver can be illuminated evenly. Specifically, in the example illustrated in FIG. 2, the near-infrared illumination 40 is installed on each side of the camera 10 to illuminate the entire driver's head.

  Next, a detailed configuration of the nose detection unit 30 will be described. FIG. 3 is a data flow diagram showing details of the nose detection unit 30 shown in FIG. As shown in the figure, the nose detection unit 30 includes a preprocessing unit (edge detection unit) 31, a nose candidate detection unit (nose candidate detection unit) 32, an eye candidate detection unit (eye candidate detection unit) 33, and an eye nose candidate. A selection unit (eye nose candidate selection unit) 34 and a nose coordinate estimation unit (nose coordinate estimation unit) 35 are provided.

  The pre-processing unit 31 receives face image data from the image acquisition unit 20 and detects edges in the image vertical direction and horizontal direction. The preprocessing unit 31 inputs processing area data set in advance for face image data, and performs edge detection on an image in the processing area of the face image. The preprocessing unit 31 is configured to output data of the detected vertical edge and horizontal edge.

  The nose candidate detection unit 32 detects nose candidates (nostril candidates) from the data of the vertical edge and the horizontal edge detected by the preprocessing unit 31. Here, the nose candidate does not need to be a nose, and the nose candidate detection unit 32 detects what is predicted to be a nose as a nose candidate. For this reason, the nose candidate detecting unit 32 detects a nose pad portion of the glasses in addition to the nose, for example, when the driver is wearing glasses. And the nose candidate detection part 32 becomes a structure which outputs the data of the detected nose candidate.

  The eye candidate detection unit 33 detects eye candidates from the horizontal edges detected by the preprocessing unit 31. Here, the eye candidate does not need to be an eye, and the eye candidate detection unit 33 detects what is predicted to be an eye, for example, a frame of eyeglasses or an eyebrow as an eye candidate. . The eye candidate detection unit 33 is configured to output detected eye candidate data.

  The eye / nose candidate selection unit 34 inputs the data of the nose candidate detected by the nose candidate detection unit 32 and the data of the eye candidate detected by the eye candidate detection unit 33, and combines these candidates. A candidate that can be established as an eye-nose is selected as a candidate for the eye-nose. That is, the eye-nose candidate selection unit 34 is configured to execute a first process for creating a combination from the nose and eye candidates and a second process for selecting what is established as an ocular nose from the combination.

  Specifically, the eye-nose candidate selecting unit 34 performs, for example, a combination of an actual nose and an eye, a combination of an eyeglass frame upper end and a nose, a combination of an eyebrow and an eyeglass nose pad, Create a combination of the lower end of the spectacle frame and the nose pad of the spectacles.

  In addition, as a second process, the eye / nose candidate selection unit 34 selects a combination that is established as a candidate for the eye / nose from the above combination based on a predetermined condition. That is, the eye / nose candidate selection unit 34 selects a candidate that can be a candidate for the eye / nose based on the condition that the eye is above the nose. Thereby, among the above combinations, the combination of the eye and nose, the combination of the upper end of the spectacle frame and the nose, and the combination of the eyebrow and the nose pad of the spectacles are selected as candidates for the eye nose. On the other hand, the combination of the lower end of the spectacle frame and the nose pad part of the spectacles is not selected as an eye nose candidate because it does not meet the above conditions.

  In this way, the eye / nose candidate selection unit 34 eliminates, as much as possible, the eye nose candidate that does not include the actual nose in the second process (in the above, the combination of the lower end of the spectacle frame and the nose pad part of the spectacles is excluded). A candidate that is predicted to have a high probability of including an actual nose is selected as an eye nose candidate. Thereby, the eye-nose candidate selection unit 34 performs processing so that the number of eye-nose candidates including the actual nose increases when a plurality of eye-nose candidates are selected. The eye / nose candidate selection unit 34 is configured to output data of the selected eye / nose candidate.

  The nose coordinate estimation unit 35 estimates the nose coordinates by statistical processing from the eye nose candidates selected by the eye nose candidate selection unit 34. Here, the statistical process refers to a process of quantitatively grasping the eye nose candidates and estimating the nose coordinates from the result.

  The statistical process will be specifically described. For example, it is assumed that a combination of an eye and a nose, a combination of an upper end of a spectacle frame and a nose, and a combination of an eyebrow and a nose pad of a spectacle are selected as eye nose candidates. In this case, statistically, there is one ocular nose candidate for the nose pad of the glasses, and there are two ocular nose candidates for the nose. Therefore, the nose can be specified with high probability by specifying the larger number as the nose by statistical processing, and the nose coordinates can be estimated. In particular, as described above, since the eye-nose candidate is selected from the condition that the eye is above the nose (due to the second process), these eye-nose candidates are selected when the eye-nose candidate is selected. The number of things that include the actual nose has increased, and the nose can be identified effectively by statistical processing.

  Furthermore, since the number of ophthalmic nose candidates that include actual noses tends to increase, an averaging process is performed as a statistical process, and the nose coordinates are averaged by averaging the coordinate values of the nose parts of the ocular nose candidates. Even if it is estimated, a value close to the actual nose can be easily obtained.

  The nose coordinate estimation unit 35 is configured to output the nose coordinates obtained as described above as nose coordinate data. In addition, the nose coordinate estimation unit 35 is configured to output average nose data. Here, the average nose data is average value data of nose coordinates estimated in the past. In the present embodiment, as will be described later, when estimating the nose coordinates from the current face image, the nose coordinates are more preferably estimated by using the average nose data.

  FIG. 4 is a data flow diagram showing details of the preprocessing unit 31 shown in FIG. As shown in the figure, the preprocessing unit 31 includes a low resolution image generation unit (low resolution image generation unit) 31a, a horizontal edge detection unit (horizontal edge detection unit) 31b, and a vertical edge detection unit (vertical edge detection unit). 31c.

  The low resolution image generation unit 31a generates a low resolution image. Here, the low resolution image is obtained by reducing the resolution of the face image taken by the camera 10. Further, the low resolution image generation unit 31a is configured to input the processing area data and generate a low resolution image only for a predetermined area of the face image set in advance. The low resolution image generation unit 31a is configured to output data of the generated low resolution image.

  The horizontal edge detector 31b detects an edge in the image vertical direction for the low resolution image generated by the low resolution image generator 31a. Further, the vertical edge detection unit 31c detects an edge in the horizontal direction of the image with respect to the low resolution image generated by the low resolution image generation unit 31a. As described above, since the detection units 31b and 31c perform edge processing on the low resolution image, the processing speed is improved. The detection units 31b and 31c are configured to output horizontal edge data or vertical edge data obtained by edge detection in the image vertical direction or horizontal direction, respectively.

  FIG. 5 is a data flow diagram showing details of the nose candidate detection unit 32 shown in FIG. As shown in the figure, the nose candidate detection unit 32 includes a common point extraction unit 32a and a nose candidate selection unit 32b.

  The common point extraction unit 32a inputs vertical edge data and horizontal edge data, and extracts common points of edges in the vertical and horizontal directions of the image. Moreover, the nose candidate selection part 32b combines the common points extracted by the common point extraction part 32a, and selects what can be established as a nostril. In other words, the nose candidate detection unit 32 can extract the common point of the image vertical and horizontal edges by the common point extraction unit 32a, combine the common points by the nose candidate selection unit 32b, and can be established as a nose among the combinations. Is selected.

  Here, the nose candidate detection unit 32 obtains the common point of the edge in order to detect the nostril candidate. This is due to the following reason. That is, since the eye has a horizontally long shape, it can be detected simply by obtaining edge data in the image vertical direction line and obtaining horizontal edge data. However, the nostril usually exists as a circle and does not have features such as a long side. For this reason, if processing is performed in the same manner as for eyes, nose candidates cannot be detected. Therefore, the nose candidate detection unit 32 inputs both edge data in the vertical and horizontal directions of the image, and extracts common points by the common point extraction unit 32a.

  In addition, the nose candidate detection unit 32 selects a combination of common points that can be established as a nose by the nose candidate selection unit 32b in order to detect nostril candidates, for the following reason. That is, generally, the nose has two nostrils adjacent to each other at an appropriate distance. For this reason, by detecting what can be established as a nose from this condition or the like, those which are not possible as noses are excluded.

  The above is the configuration of the nose detection device 1 according to the present embodiment. Next, operation | movement of the nose detection apparatus 1 which concerns on this embodiment is demonstrated. First, as shown in FIGS. 1 and 2, in the present apparatus 1, the camera 10 captures the driver's face and transmits a video signal Sa to the image acquisition unit 20. The image acquisition unit 20 stores the video signal Sa in the storage area as a digital image. Next, the image acquisition unit 20 transmits the face image data to the nose detection unit 30.

  Then, as shown in FIGS. 3 and 4, the face image data is input to the low resolution image generation unit 31 a of the preprocessing unit 31. Next, the low resolution image generation unit 31a generates a low resolution image from the face image data. Here, the face image data is high-resolution image data having a size of 640 × 480 pixels (256 gradations per pixel) and a size generally called VGA, and the low-resolution image generation unit 31 a A low resolution image shown in FIG. 6 is generated from the face image data.

  FIG. 6 is an explanatory diagram showing an example of a low resolution image, where (a) shows an example of a low resolution image and (b) shows another example of a low resolution image. First, the low-resolution image shown in FIG. 6A is not generated by thinning out pixels, but an average value is obtained for the pixels of the corresponding face image, and this average value is used as the gray value of the pixels of the low-resolution image. Is generated by That is, when generating the low resolution image, the low resolution image generation unit 31a obtains an average of the gray values for a plurality of pixels of the face image corresponding to each pixel of the low resolution image, and calculates the average value of the low resolution image. The gray value of the pixel.

More specifically, if the density value of arbitrary coordinates (x, y) of the low resolution image data is d ′ and the density value of the arbitrary coordinates of the face image is d,

Thus, the gray value of the pixel of the low resolution image is obtained. Thus, the gray value of one pixel of the low resolution image in the present embodiment is an average value of 10 × 10 pixels of the face image data. Thus, by generating a low-resolution image and reducing the face image data to 1/100, the processing time in the subsequent image processing can be shortened. Also, instead of generating a low-resolution image by thinning out pixels, a low-resolution image is generated by obtaining an average value for the pixels of the corresponding face image, so that it is possible to smooth the face image data and remove noise. it can.

  In the above, from the viewpoint of processing speed, a low resolution image of 1/10 in both vertical and horizontal directions was generated. However, the present invention is not limited to this, and the low resolution image cannot be recognized as a face by human eyes. If the resolution is not set, low-resolution images with other ratios (for example, 1/8 vertical and horizontal) may be generated.

  In addition, the low resolution image generation unit 31a inputs processing area data. Therefore, the low resolution image generation unit 31a may generate a low resolution image only for a predetermined area of the face image as shown in FIG.

  Reference is again made to FIG. When a low-resolution image is generated as described above, the low-resolution image generation unit 31a outputs low-resolution image data. Then, the horizontal edge detection unit 31b detects an edge in the image vertical direction for a low resolution image. Further, the vertical edge detection unit 31c detects an edge in the horizontal direction of the image with respect to the low resolution image.

  7A and 7B are explanatory diagrams showing detailed operations of the horizontal edge detection unit 31b and the vertical edge detection unit 31c. FIG. 7A shows an example of a face image, and FIG. 7B shows the grayscale value of the pixel column Ya in the image vertical direction. (Light quantity) is shown, and (c) shows the gray value (light quantity) of the pixel row Xa in the horizontal direction of the image. In this embodiment, the horizontal edge detection unit 31b and the vertical edge detection unit 31c detect edges for a low-resolution image. For convenience, FIG. 7A shows a face image that is not set to low resolution. To do.

  First, the horizontal edge detection unit 31b inputs an image as shown in FIG. Then, the horizontal edge detection unit 31b detects the gray value of the pixel for each line in the vertical direction of the image. As a specific example, the horizontal edge detection unit 31b detects a gray value as shown in FIG. 7B for the vertical line Ya shown in FIG.

  At this time, the horizontal edge detecting unit 31b has a minimum point at which the difference between the gray values of adjacent pixels turns from decreasing to increasing (for example, points turning from black to white on the image at p1 to p4 shown in FIG. 7B). Explore.

  Next, the horizontal edge detection unit 31b deletes the point where the change amount of the light and shade value is less than the set value from among the minimum points. Further, when the local minimum point is independent in the vicinity of 8, the horizontal edge detecting unit 31b deletes the local minimum point. As described above, the horizontal edge detection unit 31b deletes a local minimum point with almost no change in the gray value or a local minimum point detected as only one point as noise or the like.

Here, the change amount of the above-described gray value will be described. FIG. 8 is an explanatory diagram of the change amount of the gray value. As shown in the figure, the variation t of the shade value is obtained from the following equation (2).

Note that a is the coordinate of the maximum value, and d (a) indicates the maximum value. Further, b is the coordinate of the minimum value, and d (b) indicates the minimum value. The maximum value is a value opposite to the minimum value, and the difference between the gray values of adjacent pixels is changed from increasing to decreasing.

  As described above, the horizontal edge detector 31b detects a horizontal edge. Then, as shown in FIG. 4, the horizontal edge detection unit 31 b outputs detected horizontal edge data.

  The vertical edge detection unit 31c performs the same process. That is, when an image as shown in FIG. 7A is input, the vertical edge detection unit 31c detects the gray value of the pixel for each line in the horizontal direction of the image. As a specific example, the vertical edge detection unit 31c detects a gray value as shown in FIG. 7C for the horizontal line Xa shown in FIG. Further, the vertical edge detection unit 31c searches for a minimum point (for example, p5 to p7 shown in FIG. 7C) at which the difference in the gray value of adjacent pixels starts from decreasing to increasing.

  Next, the vertical edge detection unit 31c deletes a point where the change amount of the gray value is less than the set value from among the minimum points. As a result, the vertical edge detection unit 31c detects a vertical edge. Then, as shown in FIG. 4, the vertical edge detection unit 31c outputs the detected vertical edge data.

  Here, a specific image example of the minimum point detected by the horizontal edge detection unit 31b and the vertical edge detection unit 31c and edge data is shown. FIG. 9 is an explanatory diagram of the minimum point and edge data detected by the horizontal edge detection unit 31b. (A) shows the minimum point, and (b) shows the edge data. FIG. 10 is an explanatory diagram of the minimum point and edge data detected by the vertical edge detection unit 31c, where (a) indicates the minimum point and (b) indicates the edge data.

  First, as shown in FIG. 9A, the horizontal edge detection unit 31b extracts all the minimum points that change from black to white and then change from white to black for each vertical line of the image. Then, the horizontal edge detection unit 31b deletes those local minimum points that have a change in the gray value that is less than the set value and those that do not have other local minimum points in the vicinity of 8 and are independent. Then, the horizontal edge detection unit 31b detects horizontal edge data as shown in FIG.

  Similarly, the minimum edge shown in FIG. 10A is also extracted for the vertical edge detection unit 31c, and among these minimum points, the change in the gray value is less than the set value is deleted. Then, the vertical edge detector 31c finally detects the vertical edge data shown in FIG.

  Please refer to FIG. 3 and FIG. The vertical edge data and the horizontal edge data detected as described above are input to the common point extraction unit 32 a of the nose candidate detection unit 32. Then, the common point extraction unit 32a extracts a common point where the vertical edge and the horizontal edge overlap.

  FIG. 11 is an explanatory diagram showing common points extracted by the common point extraction unit 32a. In the figure, common points are shown as white dots. Specifically, the data of the common points for FIG. 9B and FIG. 10B is as shown in FIG. The common point extraction unit 32a extracts such common point data and transmits it to the nose candidate selection unit 32b.

  Reference is again made to FIG. The nose candidate selection unit 32b inputs the data of the common points from the common point extraction unit 32a, combines the data of the common points, and selects a nose candidate that is established as a nostril by combination. More specifically, the nose candidate selection unit 32b performs a labeling process, a deletion process, and a selection process on the common point data.

  That is, first, the nose candidate selection unit 32b attaches a label to each common point. Here, the nose candidate selection unit 32b attaches the same label to the common point and the common point adjacent to the common point in the vicinity of 8 (labeling process).

  And the nose candidate selection part 32b deletes a label larger than 4 pixels. That is, since one pixel of the low-resolution image corresponds to, for example, 100 pixels in the face image, the nose candidate selection unit 32b deletes a label larger than 40 × 40 = 1600 pixels in the face image (deletion process). .

  Next, the nose candidate selection unit 32b leaves a label that can be paired in the horizontal direction and deletes other labels. Specifically, the nose candidate selection unit 32b leaves a label whose distance in the horizontal direction of the image is 3 pixels or more and 6 pixels or less and whose distance in the image vertical direction is 0 pixels or more and 2 pixels or less as a pair, and puts other labels on the labels. delete. And the nose candidate selection part 32b of the nose candidate detection part 32 outputs the remaining pair as data of a nose candidate (selection process).

  The nose candidate data obtained as described above is shown in FIG. FIG. 12 is an explanatory diagram showing the nose candidates detected by the nose candidate detection unit 32. As shown in the figure, the nose candidate selection unit 32b leaves only a pair of common points that are 3 pixels to 6 pixels in the horizontal direction of the image and 0 pixels to 2 pixels in the vertical direction of the image. As a nose candidate. On the other hand, common points that do not meet the above conditions have been deleted.

  Please refer to FIG. While the nose candidate detection unit 32 detects nose candidates, the eye candidate detection unit 33 detects eye candidates. At this time, the eye candidate detection unit 33 performs labeling processing, deletion processing, division processing, and selection processing on the horizontal edge. That is, first, the eye candidate detection unit 33 labels each horizontal edge (labeling process).

  Then, the eye candidate detection unit 33 deletes a label smaller than 4 pixels. That is, a label having less than 4 pixels is less likely to be an eye due to its size. For this reason, the complement detection unit 33 deletes labels smaller than 4 pixels (deletion process).

  Further, the eye candidate detection unit 33 divides a label that is horizontally longer than a certain pixel (for example, 12 pixels or more) by deleting a point having a minimum change amount of the gray value in the label. For example, when a horizontal edge is detected from a driver's face image, depending on the light environment, the left temple and the right temple may be detected as one long horizontal edge from the left temple. Here, the long horizontal edge has a small change in gray value between the left temple and the left eyebrow. Similarly, the amount of change in the gray value is small between the left eyebrow and the right eyebrow, and between the right eyebrow and the right temple. For this reason, for a horizontally long label, the accuracy of eye candidate detection is improved by dividing the label by deleting the point having the smallest change amount of the gray value (division processing).

  Next, the eye candidate detection unit 33 deletes other labels while leaving a label that can be paired in the horizontal direction. In other words, the nose candidate selection unit 32b deletes other labels while leaving a label whose distance in the horizontal direction of the image is 9 to 22 pixels and whose distance in the vertical direction of the image is 0 to 5 pixels as a pair. To do. The eye candidate detection unit 33 then outputs the remaining pairs as eye candidate data (selection process).

  FIG. 13 shows eye candidate data obtained as described above. FIG. 13 is an explanatory diagram showing eye candidates detected by the eye candidate detection unit 33. As shown in the figure, the eye candidate detection unit 33 leaves only a pair of horizontal edges that are 9 pixels or more and 22 pixels or less in the horizontal direction of the image and that are 0 pixels or more and 5 pixels or less in the vertical direction of the image. Are detected as eye candidates. On the other hand, lateral edges that do not meet the above conditions are deleted.

  FIG. 3 will be referred to again. The nose candidate data and eye candidate data output as described above are input to the eye nose candidate selection unit 34. Then, the eye / nose candidate selection unit 34 combines the nose candidate and the eye candidate, and selects what is established as the eye / nose from these combinations.

  Here, it is assumed that the nose and the nose pad of glasses are detected as nose candidates. Further, it is assumed that eyes, eyeglass frames, and eyebrows are detected as eye candidates. At this time, the eye / nose candidate selection unit 34 creates three combinations by combining the nose, the eye, the eyeglass frame, and the eyebrow. Similarly, the eye / nose candidate selection unit 34 also creates three combinations by combining the eye, the spectacle frame, and the eyebrow for the nose pad portion of the glasses. Then, the eye / nose candidate selection unit 34 creates a total of six combinations.

  Next, the eye-nose candidate selection unit 34 obtains the center coordinates of both eyes for the eye candidate and the center coordinates of both nostrils for the nose candidate. Then, for the above six combinations, the ocular nose candidate selection unit 34 i) the center of both eyes is present on the upper end side of the image from the center of both nostrils, ii) the distance between the center of both eyes and the center of both nostrils is It must be within the market value of the distance to the nose (for example, within 3 to 12 pixels), iii) The angle between the line connecting the center of both eyes and the center of both nostrils, and the line connecting between both eyes is almost perpendicular (For example, 70 degrees or more and 110 degrees or less), iv) It is established as an ocular nose based on four criteria that a line segment connecting both eyes is almost horizontal (for example, an angle with the horizontal line is less than ± 15 degrees). Determine whether. For example, the combination of the spectacle nose pad (nose candidate) and the spectacle frame lower end (eye candidate) conforms to the above condition i) because the nose candidate is closer to the upper end of the image than the eye candidate. Therefore, it is not selected as a candidate for nose.

  Then, the eye / nose candidate selection unit 34 selects a candidate that meets all the above conditions as a candidate for the nose. FIG. 14 is an explanatory diagram showing the eye-nose candidates selected by the eye-nose candidate selection unit 34. In the figure, the eye and nose candidates are indicated by triangles. As shown in the figure, the ocular nose candidate selection unit 34 selects an ophthalmic nose candidate that satisfies the above conditions i) to iv). Specifically, in addition to an eye nose candidate consisting of both eyes and nose, an eye nose candidate consisting of a spectacle frame (lower end) and a nose, a left eye, a right frame (lower end) of glasses, and an eye nose candidate consisting of a nose , And a total of four eye nose candidates comprising both eyebrows and a nose pad.

  In the above description, for the convenience of explanation, a total of six combinations were created by combining the nose and eyes, the spectacle frame, and the eyebrow, and further combining the nose pad portion of the spectacles and the eye, the spectacle frame, and the eyebrow. However, in practice, the left eye, the right frame (lower end) of the glasses and the nose are combined, or the right eyebrow, the left frame (upper end) of the glasses and the nose pad are combined. An eye nose candidate consisting of the right frame (lower end) of the glasses and the nose is selected.

  In addition, the ocular nose candidate selection unit 34 excludes those that are not established as ophthalmic nose candidates according to the conditions i) to iv), and thus the ocular nose candidates are likely to include an actual nose. Then, the eye-nose candidate selection unit 34 outputs the data of the eye-nose candidate obtained as described above to the nose coordinate estimation unit 35 as shown in FIG.

  Next, the nose coordinate estimation unit 35 estimates the coordinates of the nose from the eye nose candidates selected by the eye nose candidate selection unit 34 by statistical processing.

  Here, the nose coordinate estimation part 35 performs the process which forms a frequency distribution as a statistical process, specifies a nose from the frequency, and estimates a coordinate. FIGS. 15A and 15B are explanatory diagrams when the nose coordinate estimation unit 35 forms a frequency distribution as a statistical process. FIG. 15A shows the frequency distribution, and FIG. 15B shows an image example. In FIG. 15A, it is assumed that the frequency becomes higher as it becomes closer to white.

  First, the nose coordinate estimation unit 35 forms a frequency distribution as shown in FIG. 15A based on the nose candidates selected by the eye nose candidate selection unit 34. For example, as shown in FIG. 15B, the actual combination of both eyes and nose, the combination of the upper end of the spectacle frame and the nose, the combination of both eyebrows and the nose pad of the spectacles, the lower end of the right frame of the spectacles and the left Assume that a total of four combinations of eyes and nose are selected as candidates for the eyes and nose. Then, the frequency distribution based on the nose candidate is the frequency “1” for the nose pad portion of the glasses and the frequency “3” for the nose.

  Next, the nose coordinate estimation unit 35 specifies the nose candidate having the highest frequency of the eye nose candidate among the formed frequency distribution as the nose, and estimates the coordinates. That is, according to the example shown in FIG. 15A, the nose coordinate estimation unit 35 specifies the frequency “3”, that is, the white portion as the nose.

  Here, as described above, the ophthalmic nose candidate is selected based on the above conditions i) to iv), and therefore, the ophthalmic nose candidate is likely to include an actual nose. The number of things is increasing. And since a nose is detected by statistical processing from such an eye nose candidate, an improvement in precision can be aimed at compared with the case where only a nostril is detected independently.

  In addition, since the number of ocular nose candidates includes the actual nose is large, a value close to the actual nose can be obtained even if the averaging process is performed as a statistical process to estimate the nose coordinates. It will be easy.

  Moreover, the nose coordinate estimation unit 35 may estimate the nose coordinates by statistical processing using the past face image. In other words, the nose coordinate estimation unit 35 forms a frequency distribution of the eye-nose candidates obtained from the face images of a certain period in the past with reference to the nose coordinates.

  And the nose coordinate estimation part 35 makes the position of a nose candidate with the highest frequency of an eye nose candidate among the formed frequency distributions an average coordinate position. Next, the nose coordinate estimation unit 35 identifies the nose candidate closest to the average coordinate position as the nose in the current face image, and estimates the coordinates. Even in this case, it is possible to estimate the nose coordinates with high accuracy because the past nose candidates are used.

  The nose coordinate estimation unit 35 outputs the average coordinate position as average nose data to other elements to be stored. And the nose coordinate estimation part 35 will read and process average nose data in performing the next process.

  The operation of the nose detection device 1 has been described above. The following technical effects have been confirmed by performing such operations. FIG. 16 is a graph showing the detection rate of the nose. In the figure, the vertical axis indicates the detection rate, and the horizontal axis indicates the past number of frames. Further, in the figure, the number of detection target persons is 50.

  First, the face image was taken when the lighting environment was not stable, such as when one day was inserted, or when the nostril was not photographed as a black circle because the near infrared ray was inserted into the nostril. Under this circumstance, when the nose was detected from the face image obtained first, the detection rate was about 90%. That is, the nose coordinates could be accurately estimated from 45 out of 50 people.

  Further, when detection is repeated and the number of past image frames reaches a certain number, processing using the average coordinate position becomes possible. Here, when the number of past image frames is 60 (about 2 seconds), the average coordinate position is obtained, and when the nearest nose candidate is identified as the nose and the coordinates are estimated, the detection rate is about 100%. became. That is, it can be said that the nose can be reliably detected in about 2 seconds.

  On the other hand, according to the method of directly detecting the nostril and obtaining the nose coordinates, in a shooting environment where one day is inserted, the detection rate when the past image is 60 frames was 54%.

  As described above, the present apparatus 1 increases the number of ophthalmic nose candidates including the actual nose by selecting those that can be established as ophthalmic noses as statistical candidates. Since the nose is detected by processing, the detection accuracy of the nose is improved as compared with the case where only the nostril is detected alone.

  In this way, according to the nose detection device 1 according to the present embodiment, the edge is detected from the face image, and the nose candidate is detected from the common point of the edge in the vertical and horizontal directions of the image. Here, since the nose (nasal passage) usually exists as a black circle, it can be suitably detected by detecting not only from the edge in the vertical or horizontal direction of the image but also from the common point of both edges. Also, since nose candidates are detected at this stage, the nose and other objects (for example, the nose pad of glasses) may be detected. It does not matter if it is detected.

  Then, eye candidates are detected from edges in the horizontal direction of the image. Here, since the eyes are generally long in the horizontal direction, they are preferably detected from the edges in the horizontal direction of the image. In addition, since eye candidates are detected at this stage, for example, a frame part of eyeglasses or an eyebrow may be detected.

  Then, the detected nose and eye candidates are combined, and those that are established as candidates for the eye and nose are selected from these combinations. That is, a process of creating a combination and then selecting it is performed.

  Specifically, first, the combination includes, for example, an actual nose and eye, an eyeglass frame upper end and nose, an eyebrow and eyeglass nose pad, and an eyeglass frame lower end. A thing composed of a nose pad of glasses is created.

  Next, from those combinations, a candidate that is established as a candidate for the eye and nose is selected based on a predetermined condition. Specifically, for example, a candidate that can be a candidate for an eye and nose is selected based on the condition that the eye is above the nose. Thereby, the combination of eyes and nose among the above combinations is selected as a candidate for nose. Further, for example, a combination of the upper end of the spectacle frame and the nose and a combination of the eyebrows and the nose pad part of the spectacles are selected as candidates for the eye nose. However, the combination of the lower end of the spectacle frame and the nose pad part of the spectacles does not meet the above conditions and is not selected as an ophthalmic nose candidate.

  In this way, the ocular nose candidate is preferably selected from the positional relationship between the eyes and the nose, and thus tends to include the actual nose. That is, the number of eye nose candidates includes an actual nose.

  The nose coordinates are then estimated by statistical processing from the selected ocular nose candidates. Here, in addition to the combination of the actual eye and nose, the combination of the upper end of the spectacle frame and the nose and the combination of the eyebrow and the nose pad of the spectacles are selected as candidates for the eye nose. To do. Then, statistically, there is one combination for the nose pad portion of the glasses and two combinations for the nose. Therefore, the nose can be specified with high probability by specifying the nose which has a larger number in the statistical processing, and the nose coordinates can be estimated. In particular, as described above, the eye-nose candidates are preferably selected from the positional relationship between the eyes and the nose, and the number including the actual nose increases, so the nose is detected from the eye-nose candidates by statistical processing. Thus, the detection accuracy of the nose can be increased.

  In addition, since the number of eye nose candidates includes the nose, a value close to the actual nose can be easily obtained even if the averaging process is performed as a statistical process to estimate the nose coordinates. .

  In this way, by selecting those that can be established as eye nose candidates, the number of eye nose candidates including the actual nose is increased, and the nose is extracted from such eye nose candidates by statistical processing. Therefore, the accuracy can be improved as compared with the case where only the nostril is detected alone. Therefore, it is possible to improve the detection accuracy of the nose.

  In addition, since a low-resolution image in which the resolution of the captured face image is reduced is generated and edge detection is performed on the low-resolution image, the processing speed can be improved.

  In addition, when generating a low-resolution image in which the resolution of the captured face image is reduced, a low-resolution image is generated only for a predetermined area of the face image that is set in advance. By setting the predetermined area, the generation speed of the low resolution image can be improved, and further, the processing speed can be improved for other processes using the low resolution image.

  Further, when generating a low-resolution image in which the resolution of the captured face image is reduced, an average gray value is obtained for a plurality of pixels of the face image corresponding to each pixel of the low-resolution image, and this average value is calculated. The gray value of the pixel of the low resolution image is used. For this reason, it is possible to obtain the average of the gray value and remove (smooth) the noise from the face image data, and to improve the detection accuracy of the nose as compared with the case where the pixels are thinned out.

  In addition, the horizontal edge of the low-resolution image is detected by searching for a minimum point where the difference in the gray value of adjacent pixels in each vertical line of the low-resolution image turns from decreasing to increasing. Further, the vertical edge of the low-resolution image is detected by searching for the minimum point where the difference in the gray value of the adjacent pixels in each horizontal line of the low-resolution image turns from decreasing to increasing. As described above, since the minimum point of the change in shading is searched for the edge detection, the edge can be suitably detected even in an optical environment in which the shading change of the image is reduced.

  Further, common points with the vertical and horizontal edges of the image are combined, and a combination that can be established as a nostril by this combination is detected as a nose candidate. For this reason, prior to performing statistical processing, it is possible to remove in advance what is not possible as a nose, and it is possible to improve the detection accuracy of the nose.

  In addition, the vertical edges of the image are combined, and those that can be established as eyes by this combination are detected as eye candidates. For this reason, prior to performing statistical processing, items that cannot be used as eyes in advance are removed, and the presence of an inappropriate eye candidate in selecting an ophthalmic nose candidate causes a drop in the accuracy of selecting an ophthalmic nose candidate. Can be avoided.

  In addition, as statistical processing, a frequency distribution is formed with respect to the nose candidate based on the nose candidate, and the nose candidate with the highest frequency of the eye nose candidate is identified as the nose, and the coordinates are determined. Estimated. Here, the eye nose candidates tend to include the actual nose. For this reason, an actual nose can be suitably specified by creating a frequency distribution and specifying a high frequency location as a nose.

  In addition, the eye nose candidate obtained from the face image of a certain period in the past is formed a frequency distribution with reference to the nose coordinates, and the position of the nose candidate with the highest frequency of the eye nose candidate among the formed frequency distribution is averaged Coordinate position. Further, in the current face image, the nose candidate closest to the average coordinate position is identified as the nose, and the coordinates are estimated. Here, for example, the driver basically looks forward while driving, and the coordinates of the nose are stable. For this reason, generating a frequency distribution from a certain past image is the same as learning the position of the nose in the driver's face image. To identify the nose. Therefore, the detection accuracy of the nose can be improved.

  As described above, the present invention has been described based on the embodiment, but the present invention is not limited to the above embodiment, and may be modified without departing from the gist of the present invention. For example, in the above-described embodiment, an example in which the nose detection device is mounted on an automobile has been described. However, the present invention is not particularly limited thereto, and may be mounted on a vehicle other than a vehicle or may be used in a device other than a vehicle.

It is a lineblock diagram of a nose detection device concerning this embodiment. It is a block diagram which shows the other example of the nose detection apparatus 1 in this embodiment. It is a data flow diagram which shows the detail of the nose detection part shown in FIG. It is a data flow diagram which shows the detail of the pre-processing part shown in FIG. It is a data flow diagram which shows the detail of the nose candidate detection part shown in FIG. It is explanatory drawing which shows the example of a low resolution image, Comprising: (a) shows an example of a low resolution image, (b) has shown the other example of the low resolution image. It is explanatory drawing which shows detailed operation | movement of a horizontal edge detection part and a vertical edge detection part, (a) shows the example of a face image, (b) shows the light / dark value (light quantity) of the pixel row Ya of an image vertical direction, (C) shows the gray value (light quantity) of the pixel row Xa in the horizontal direction of the image. It is explanatory drawing about the variation | change_quantity of a light / dark value. It is explanatory drawing of the minimum point detected by a horizontal edge detection part, and edge data, (a) shows a minimum point, (b) has shown edge data. It is explanatory drawing of the minimum point detected by the vertical edge detection part, and edge data, (a) shows a minimum point, (b) has shown edge data. It is explanatory drawing which shows the common point extracted by the common point extraction part. It is explanatory drawing which shows the nose candidate detected by the nose candidate detection part. It is explanatory drawing which shows the eye candidate detected by the eye candidate detection part. It is explanatory drawing which shows the eye-nose candidate selected by the eye-nose candidate selection part. It is explanatory drawing when a nose coordinate estimation part forms frequency distribution as statistical processing, (a) shows frequency distribution, (b) has shown the example of an image. It is a graph which shows the detection rate of a nose.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Nose detection apparatus 10 ... Camera (photographing means)
20 ... Image acquisition unit 30 ... Nose detection unit 31 ... Pre-processing unit (edge detection means)
31a: Low-resolution image generation unit (low-resolution image generation means)
31b ... Horizontal edge detection unit (horizontal edge detection means)
31c... Vertical edge detector (vertical edge detector)
32 ... Nose candidate detection unit (nasal candidate detection means)
32a ... Common point extraction unit 32b ... Nose candidate selection unit 33 ... Eye candidate detection unit (eye candidate detection means)
34 ... Eye-nose candidate selection unit (eye-nose candidate selection means)
35 ... Nose coordinate estimation unit (nasal coordinate estimation means)
40 ... Near-infrared illumination

Claims (10)

Photographing means for photographing the driver's face;
Edge detection means for detecting edges in the vertical and horizontal directions of the face image taken by the photographing means;
Nose candidate detection means for detecting a nose candidate from a common point of the vertical and horizontal edges of the image detected by the edge detection means;
Eye candidate detection means for detecting eye candidates from edges in the horizontal direction of the image detected by the edge detection means;
A combination of nose and eye candidates detected by the nose candidate detection unit and the eye candidate detection unit, and an ophthalmic nose candidate selection unit that selects what is established as a candidate for an eye nose from these combinations;
Nose coordinate estimation means for estimating the coordinates of the nose by statistical processing from the candidates for the eye nose selected by the eye nose candidate selection means;
A nose detection device comprising: A low-resolution image generating unit that generates a low-resolution image with a reduced resolution of the face image captured by the imaging unit;
The edge detection means includes a vertical edge detection means for detecting an edge in the image vertical direction for the low resolution image generated by the low resolution image generation means, and a low resolution image generated by the low resolution image generation means. The nose detection device according to claim 1, further comprising: a horizontal edge detection unit configured to detect an edge in the horizontal direction of the image.   The low-resolution image generation means generates a low-resolution image only for a predetermined area of a face image set in advance when generating a low-resolution image in which the resolution of the captured face image is reduced. The nose detection device according to claim 2.   The low-resolution image generation means generates an average gray value for a plurality of pixels of the face image corresponding to each pixel of the low-resolution image when generating a low-resolution image in which the resolution of the captured face image is reduced. 3. The nose detection device according to claim 2, wherein the average value is obtained as a gray value of the pixels of the low resolution image.   The horizontal edge detecting means detects a horizontal edge of the low-resolution image by searching for a minimum point at which the difference between the gray values of adjacent pixels in each vertical line of the low-resolution image turns from decreasing to increasing. The nose detection apparatus according to any one of claims 2 to 4, wherein the nose detection apparatus is characterized.   The vertical edge detection means detects a vertical edge of the low resolution image by searching for a minimum point where the difference between the gray values of adjacent pixels in each horizontal line of the low resolution image turns from decreasing to increasing. The nose detection apparatus according to any one of claims 2 to 5, wherein the nose detection apparatus is characterized.   The said nose candidate detection means combines a common point with the edge of an image vertical direction and a horizontal direction, and detects what can be materialized as a nostril by this combination as a nose candidate. The nose detection apparatus of any one of Claims.   8. The eye candidate detection unit according to claim 1, wherein the eye candidate detection unit combines edges in the vertical direction of the image and detects an eye candidate that can be formed as an eye by this combination. 9. Nose detection device.   The nose coordinate estimation means forms a frequency distribution based on the nose candidates as candidates for the eye nose selected by the eye nose candidate selection means as the statistical processing, and most of the formed nodal distribution The nose detection apparatus according to claim 1, wherein a nose candidate having a high candidate frequency is identified as a nose, and coordinates are estimated.   The nose coordinate estimation means forms a frequency distribution of eye nose candidates obtained from face images of a certain period in the past based on the nose coordinates, and the nose with the highest frequency of the eye nose candidate among the formed frequency distributions The candidate position is set as an average coordinate position, and in the current face image, the nose candidate closest to the average coordinate position is specified as the nose, and the coordinates are estimated. The nose detection device according to item 1.