JP2021193624A - Recognition device, recognition method and recognition program - Google Patents

Abstract

To provide a recognition device or the like capable of accurately recognizing the direction of an object coated with a coating member.SOLUTION: A recognition device 1 includes acquisition means for acquiring an image obtained by photographing an object coated with a coating member in a state where the object is irradiated by irradiation means, detection means for detecting a center line in a front vertical direction of the surface of the object on the basis of the luminance value of a pixel included in the image, and recognition means for recognizing the orientation of the object on the basis of the center line.SELECTED DRAWING: Figure 1

Description

The present application relates to a technical field such as a recognition device that recognizes a direction in which an object is facing based on an image of the object.

Conventionally, there is a technique of recognizing the direction of a person's face from an image of the person's face. Generally, the orientation of the face is recognized using the characteristics of each part of the face, so if a part of the face is covered with a covering material such as a mask, the recognition accuracy is lowered.

Therefore, the technique described in Patent Document 1 calculates the left-right angle of the face from the relative position of the nose in the center of the mask with respect to the face width of the face region when the mask is present.

Japanese Patent No. 5359266

However, in the technique described in Patent Document 1, although the central part of the mask is the position of the nose, the central part of the mask may not always be the actual position of the nose, and the left-right angle of the face may not be calculated accurately. be.

In view of these circumstances, it is an object of the present invention to provide a recognition device or the like capable of accurately recognizing the orientation of an object covered with a covering member.

The invention according to claim 1 is based on an acquisition means for acquiring an image obtained by irradiating an object covered with a covering member with light by an irradiation means and a luminance value of a pixel included in the image. A detection means for detecting the center line in the front vertical direction on the surface of the target and a recognition means for recognizing the direction of the target based on the detection result of the detection means are provided.

The invention according to claim 6 is a recognition method using a recognition device, which includes an acquisition step of acquiring an image taken by irradiating an object covered with a covering member with light by an irradiation means, and the image. It includes a detection step of detecting a center line in the front vertical direction on the surface of the target based on the brightness value of the pixel, and a recognition step of recognizing the orientation of the target based on the detection result of the detection step.

The invention according to claim 7 is based on an acquisition means for acquiring an image taken by a computer in a state where an object covered with a covering member is irradiated with light by an irradiation means, and a luminance value of a pixel included in the image. , The detection means for detecting the center line in the front vertical direction on the surface of the target, and the recognition means for recognizing the orientation of the target based on the detection result of the detection means.

It is a block diagram of the recognition device 1. This is an example of a color-coded image. This is an example of a color-coded image. This is an example of a block diagram of the face orientation recognition device D. This is an example of a functional block diagram of the face orientation recognition device D. (A), (B), and (C) are diagrams for explaining an example of a method for detecting features of a face or a mask. This is an example of a color-coded image taken with various face orientations. (A) and (B) are diagrams showing an example of a mask having a special shape. It is a flowchart which shows an example of the face orientation determination processing by a face orientation recognition apparatus D. It is a flowchart which shows an example of the mask wearing determination processing by a face orientation recognition apparatus D. It is a flowchart which shows an example of the yaw angle determination processing by a face orientation recognition apparatus D. It is a flowchart which shows an example of the pitch angle determination processing by a face orientation recognition apparatus D. (A) and (B) are diagrams showing an example of the features of the left and right edges of the mask and the features of the face center line (near the upper end of the mask). (A) and (B) are diagrams showing an example of the characteristics of the left and right edges of the mask and the characteristics of the face center line (the portion applied to the mask). It is a figure which shows an example of the image which took the state that the camera C and the irradiation part L were arranged in different directions with respect to an object.

A mode for carrying out the present invention will be described with reference to FIG.

As shown in FIG. 1, the recognition device 1 includes acquisition means 111A, detection means 111B, recognition means 111C, storage control means 111D, and storage means 111E. The storage means may be provided outside the recognition device 1.

The acquisition means 111A acquires an image taken in a state where the target covered with the covering member is irradiated with light by the irradiation means. The object covered by the covering member is, for example, a face wearing a mask. The irradiation means preferably irradiates the target with light from a certain direction.

The detection means 111B detects the center line in the front vertical direction on the surface of the target based on the luminance value of the pixels included in the image acquired by the acquisition means 111A. The front vertical centerline on the surface of the subject is the line between the eyes, the center of the nose, and the center of the mouth that passes through the face surface when the subject is the face (general face). be.

Here, the center line will be described with reference to FIG. Figure 2 shows an image of a masked face (an example of a "covering material") taken with light shining from the direction of the camera (to protect the privacy of the model, the eyes are painted black. The same applies to the image taken by the model below), which is a color-coded image 200 based on the brightness value. Specifically, the portion having a luminance value of less than 100 is represented by the color indicated by the reference numeral 201, the portion having a luminance value of 100 or more and less than 160 is represented by the color indicated by the reference numeral 202, and the portion having a luminance value of 160 or more and less than 220 is indicated by the reference numeral 203. It is represented by a color, and a portion having a luminance value of 220 or more is represented by a color indicated by reference numeral 204. When irradiated with light, the brightness value of the mask area becomes larger than that of the skin. As shown in FIG. 2, the mask region has a large luminance value (luminance value 220 or more). The face wearing the mask is bilaterally symmetrical with respect to the center of the face, and the center of the face tends to protrude slightly with respect to other facial parts. Can detect the left end 211 of the mask and the face (mask) centerline 212 (an example of the "centerline"). That is, the detection means 111B detects the edge on the left end side of the mask (right side in the image) of the region having a luminance value of 220 or more as the left end 211 of the mask, and is opposite to the face center side (opposite to the left end side of the mask) in the region having a luminance value of 220 or more. The edge on the side) is detected as the face center line 212. However, if the lateral orientation (yaw angle) exceeds a certain angle (about 50 degrees) with respect to the front of the camera, the edge on the face center side (opposite to the left end side of the mask) in the region with a brightness value of 220 or more becomes the face. Since it does not match the center line 212, in this case, it is preferable to detect the edge on the face center side in the region having a luminance value of 160 or more and less than 220 as the face center line 212B.

The recognition means 111C recognizes the direction of the target based on the detection result of the detection means 111B.

As described above, according to the operation of the recognition device 1 according to the embodiment, the orientation of the target is recognized based on the detection result of the center line in the front vertical direction on the surface of the target based on the luminance value of the pixels included in the image. Therefore, even when the object is covered with the covering member, the direction of the object can be accurately recognized.

Further, the recognition means 111C may recognize the vertical direction (pitch angle) of the target based on the inclination of the face center line 212. Since it is known as a result of the inventor's investigation that there is a correlation between the vertical orientation of the face and the inclination of the face center line 212, each vertical orientation (pitch angle) of the face (pitch angle) (for example, every 10 degrees) ), And the inclination of the face center line 212 is compared with the threshold to determine the vertical angle of the face. This makes it possible to recognize the vertical orientation of the target.

Further, the detection means 111B further detects the upper boundary 213 which is the upper boundary of the covering member in the target when the recognition means 111C recognizes the vertical direction of the target, and the storage control means 111D further detects the upper boundary 213, and the storage control means 111D detects the detection means 111B. Stores the inclination of the upper boundary 213 detected by the storage means 111E in association with the vertical orientation of the target recognized by the recognition means 111B, and the recognition means 111C stores the image of the target whose vertical orientation is not recognized. Is newly acquired, the vertical orientation associated with the tilt of the upper boundary 213 of the covering member detected for the image is substantially the same as the vertical orientation of the object to be reflected in the image. May be recognized as. As a result of the inventor's investigation, it is known that when the inclinations of the upper boundary 213 of the covering member are substantially the same, the orientation of the target in the vertical direction is also substantially the same. Become. As a result, once the vertical orientation of the target is recognized, the upper boundary 213 is detected for the subsequent images and compared with the upper boundary 213 stored in the storage means in the vertical direction. Can recognize the direction of. That is, the processing load for recognizing the vertical orientation is reduced.

Furthermore, the recognition means 111C may recognize the lateral orientation (yaw angle) of the target based on the position of the intersection of the upper boundary line 213 and the face center line 212 in the width of the target.

Here, an example in the case of recognizing the lateral orientation of the target using FIG. 3 will be described. The recognition means 111C recognizes the lateral orientation of the object based on the three perpendiculars 221-223. The perpendicular line 221 is a perpendicular line passing through the point at the end of the left ear. The perpendicular line 222 is a perpendicular line passing through the intersection of the face center line 212 and the upper boundary 213. The vertical line 223 is a vertical line that moves horizontally from the intersection of the face center line 212 and the upper boundary 213 toward the back of the face and passes through the point where the right end of the face is reached. The recognition means 111C has any two of the width WA of the vertical line 221 and the vertical line 222 (front side face width WA), the width WB of the vertical line 222 and the vertical line 223 (back side face width WB), and the width WC of the vertical line 221 and the vertical line 223. Recognize the lateral orientation of the object based on its width. As a result of the inventor's investigation, it is known that there is a correlation between the lateral orientation of the face and the ratio of the front side face width WA and the back side face width WB. ) (For example, every 10 degrees), a threshold for the ratio of the front face width WA and the back face width WB is set, and the ratio of the front face width WA and the back face width WB is compared with the threshold. The lateral angle of the face can be determined. This makes it possible to recognize the lateral orientation of the object.

Furthermore, the detecting means 111B further detects the left-right boundary which is the boundary of at least one of the left side and the right side of the covering member, and the recognition means 111C detects the left-right boundary when the detecting means 111B detects the left-right boundary. The lateral orientation of the object is recognized based on the left-right boundary and the center line. The left-right boundary means, for example, at least one boundary of the left boundary indicated by the edge of the left end 211 of the mask (see FIG. 2) and the right boundary indicated by the right end edge of the mask. That is, the left-right boundary can be said to be a general term for the left boundary and the right boundary. The recognition means 111C recognizes that the target is facing the right side based on the detection of the edge (left boundary) of the left end 211 of the mask, and recognizes that the target is facing the left side of the mask at the right end edge (left boundary). (Right boundary) is recognized based on the detection.

Next, specific examples corresponding to the above-described embodiments will be described.

Examples will be described with reference to FIGS. 4- and 14. The examples described below are examples when the present invention is applied to the face orientation recognition device D (hereinafter, may be referred to as “recognition device D”).

The recognition device D of this embodiment is connected to an irradiation unit L or a camera C which is a light source such as an LED. The recognition device D recognizes the direction of the face F based on an image taken in a state where the face F covered with the mask M is irradiated with light by the irradiation unit L.

Specifically, the recognition device D detects a face (or a face feature) by signal processing the captured image, and determines whether or not the mask is worn. When it is determined that the mask is worn, the features of the mask (mask shadow area, mask edge, mask edge gradient, mask string edge gradient, etc.) and facial features (face centerline, face width, face contour, etc.) are detected. Then, from these extracted features, the orientation of the face is recognized.

More specifically, first, it is determined whether or not the face is reflected in the image, and the position (mouth area) where the mask wearing determination is performed is determined. For example, in a face frame detected by a conventionally known face detection process, an approximate position of the mouth is determined. It should be noted that the position where the mask wearing determination is performed may be determined by detecting the characteristics of each part of the face. For example, the position of the eyes may be detected, the area below the eyes may be determined as the approximate position of the mouth, and the mask wearing determination may be performed. Next, it is determined whether or not the mask is worn. Generally, when the face F with the mask M is irradiated with light to take an image, a high luminance value is obtained in the mask-wearing area. judge. It should be noted that it may be determined whether or not the mask is worn from the external shape information of the mask and the like. When it is determined that the mask is worn, the features of the mask and the features of the face are detected, and the orientation of the face is recognized from these extracted features. In addition, from the feature detection results when the facial features and mask features are detected from the image, the intensity of the irradiation light can be adjusted and the camera parameters can be adjusted to adjust the mask, facial shadows, brightness gradient, etc. Detect more prominently. In this embodiment, a case where the camera C and the irradiation unit L are facing the target from substantially the same direction will be described. The case where the camera C and the irradiation unit L are facing the target from different directions will be described with reference to a modification described later.

[1. Configuration of recognition device D]
Next, the configuration of the recognition device D according to the present embodiment will be described with reference to FIG. As shown in FIG. 4, the recognition device D is roughly classified and includes a control unit 311, a storage unit 312, a communication unit 313, a display unit 314, and an operation unit 315. The communication unit 313 is connected to the camera C and the irradiation unit L.

The storage unit 312 is configured by, for example, a hard disk drive or the like, and stores various programs including an OS (Operating System) and a face orientation recognition program for recognizing the orientation of the face. In addition, the storage unit 312 stores images taken by the camera and various data used in the face orientation recognition program.

The communication unit 313 controls the communication state with the irradiation unit L and the camera C.

The display unit 314 is composed of, for example, a liquid crystal display or the like, and is designed to display an image or the like taken by the camera C.

The operation unit 315 is composed of, for example, a keyboard, a mouse, or the like, and receives an operation instruction from an operator and outputs the instruction content as an instruction signal to the control unit 311.

The control unit 311 has a CPU (Central Processing Unit) and a ROM (Read Only Memory).
, RAM (Random Access Memory), etc. Then, the CPU realizes various functions by reading and executing various programs including the face orientation recognition program stored in the ROM and the storage unit 312.

[2. Function of recognition device D]
Next, the function of the recognition device D (control unit 311) according to the present embodiment will be described with reference to FIG. FIG. 5 is a diagram showing a functional block diagram of the recognition device D. The recognition device D includes an image analysis unit 351, an irradiation light adjustment unit 357, and a camera adjustment unit 358. The functions of the image analysis unit 351 and the irradiation light adjustment unit 357 and the camera adjustment unit 358 are realized by the control unit 311 executing the face orientation recognition program.

The image analysis unit 351 includes a mouth area detection unit 352, a mask wearing determination unit 353, a mask feature detection unit 354, a face feature detection unit 355, and a face orientation direction recognition unit 356.

The mouth region detection unit 352, for example, has face detection processing, eye detection processing, and nose detection processing for an image taken by the camera C with the face F covered with the mask M irradiated with light by the irradiation unit L. , Performs mouth detection processing. When the face region can be determined by the face detection process, it can be determined that the mouth region is in the lower region of the face detection frame. If the eye area can be determined by the eye detection process, it can be determined that the mouth area is below the eyes. Even if the nose region can be determined by the nose detection process, it can be determined that the mouth region is in the lower part of the nose. If the mouth area can be detected by the mouth detection process, it is determined that the area is the mouth area.

The mask wearing determination unit 353 determines whether or not the mask M is worn for the mouth region detected by the mouth region detection unit 352. Specifically, the determination is made based on the brightness value. For example, as shown in FIG. 6A, the average luminance value of the right eye region 401 and the average luminance value of the mouth region 402 are calculated. Generally, when the light is applied to the face, the brightness value of the mask is larger than the brightness value of the skin. For example, the average luminance value of the right eye region 401 and the average luminance value of the mouth region 402 are compared, and if they are substantially the same value, it is determined that the mask is not worn, and if the average luminance value of the mouth region 402 is larger (threshold value is set). If it exceeds the size), it can be determined that the mask is worn.

The mask feature detection unit 354 detects features in the mask region based on the luminance value of the mask region. The face feature detection unit 355 detects features in the face region based on the brightness value of the face region. Specifically, the mask feature detection unit 354 and the face feature detection unit 355 (collectively referred to as "feature detection unit") set the luminance value in the image obtained by photographing the face to 3 as shown in FIG. Features are detected by color-coded images divided into four by one threshold value. Specifically, it is divided into (1) a mask area, (2) a skin (bright) area, (3) a skin (dark) area, and (4) a background area.

Here, an example of creating a color-coded image will be described. First, the feature detection unit sets the first threshold value indicating the boundary between (1) the mask region and (2) the skin (bright) region as the maximum value Ja of the skin region (for example, the left eye region 401) (the pixel value is a spike). Considering that noise and the like may be included, here, for example, the fifth largest value is treated as the maximum value), and the determination is made based on the average luminance value Jb of the mouth region 402 when the mask is worn. For example, when Ja is "219" and Jb is "246", the value obtained by adding a margin to Ja is set as the first threshold value. For example, "220", which is obtained by adding "1" to Ja, is set as the first threshold value ("1" is added to obtain a sharp value, but other values may be used. Hereinafter, the second threshold value and the third threshold value may be used. The same applies to). A region composed of pixels having a luminance value larger than the first threshold value can be detected as a mask region.

Next, the feature detection unit sets a second threshold value indicating the boundary between the (2) skin (bright) region and (3) skin (dark) region as the skin region (for example, the lower left eye half region shown in FIG. 6B). It is determined based on the average value Jc of 403) (the lower half region of the left eye is set to 403 because the brightness value of the eyebrows and the black eye portion is small, but the average value of other portions may be used). For example, when Jc is "157", the value obtained by subtracting the margin from Jc is set as the second threshold value. For example, “150”, which is obtained by subtracting “7” from Jc, is set as the second threshold value.

Next, the feature detection unit sets a third threshold value indicating the boundary between the (3) skin (dark) region and (4) background region in the skin region (for example, the left eye left half region 404 shown in FIG. 6C). Minimum value Jd (In consideration that the pixel value may include spike noise and the like, for example, the fifth smallest value is treated as the minimum value. Here, the left eye left half area 404 is used, but other It may be the minimum value of the skin part of). For example, when Jd is "78", the value obtained by subtracting the margin from Jd is set as the third threshold value. For example, “70”, which is obtained by subtracting “8” from Jd, is set as the third threshold value.

The feature detection unit determines each pixel constituting the image into (1) a mask area, (2) a skin (bright) area, and (2) a skin (bright) area, based on the luminance value of the pixel and the first threshold value, the second threshold value, and the third threshold value. A color-coded image is created by dividing into a skin (dark) area and (4) a background area and expressing them in four colors. Then, based on the color-coded image, the feature of the mask region and the feature of the face region are detected.

FIG. 7 is an example of an image taken by changing the vertical orientation (pitch angle) of the face and the horizontal orientation (yaw angle) of the face by 10 degrees, and is the same as the example of FIG. , A portion having a luminance value of less than 100, a portion having a luminance value of 100 or more and less than 160, a portion having a luminance value of 160 or more and less than 220, and a portion having a luminance value of 220 or more are represented by different colors (similar to the case of FIG. 2). .. When irradiated with light, the brightness value of the mask area becomes larger than that of the skin. The mask area has a large luminance value (luminance value 220 or more). When the face is oriented in the left-right direction (yaw angle: 20 to 50 degrees), the face orientation recognition unit 356 can clearly detect the left-right boundary, which is the boundary between the left-right edge of the mask and the skin, and the face is left-right. Direction It is possible to determine which direction the user is facing. On the other hand, when facing the front of the camera (yaw angle: 0 degrees), the boundary between the left and right edges of the mask and the skin (left and right boundaries) cannot be detected, so it can be determined that the camera is facing the front. .. In order to estimate the face orientation angle in the left-right direction, if the center of the face, the edge of the face (contour, ears, etc.) and the left and right edges of the mask can be detected, the approximate face orientation angle can be calculated. Hereinafter, a method in which the face orientation recognition unit 356 recognizes the vertical orientation (pitch angle) of the face and the lateral orientation (yaw angle) of the face will be described in detail.

(Pitch angle: 0 degrees)
When the face is facing the front of the camera (yaw angle: 0 degrees, pitch angle: 0 degrees), the brightness value in the mouth area is large (compared to the skin area), but the mask is prominent on the left and right edges of the mask. Edges etc. are not detected. When the face is facing left 10 degrees (yaw angle: 10 degrees, pitch angle: 0 degrees), the brightness value in the mouth area becomes large, and the mask is tilted to the left with respect to the camera. A prominent mask edge can be detected at the left end. This is because the light is emitted from the front direction of the camera. When the face is 20 degrees to the left (yaw angle: 20 degrees), the brightness value of the area of the mask facing the camera becomes remarkable because the face is tilted more than when the face is 10 degrees. The left edge of the mask can also be noticeably detected. In addition, the face with the mask is symmetrical with respect to the center of the face, and the center of the face tends to protrude slightly with respect to other facial parts. Therefore, the center line of the face can be clearly detected. Similarly, when the face is oriented at 30 degrees to the left, 40 degrees to the left, and 50 degrees to the left, the left edge of the mask and the center line of the face can be remarkably detected. Further, when there is no object in the vicinity of the face, the brightness value of the background does not increase even if the light is irradiated, so the face width WA in front of the camera and the face width WB in the back of the camera are detected. It is possible. When the face direction is tilted to the left by 10 degrees, the edge of the left end of the mask, the center line of the face, the face width WA in front of the camera, the face width WB in the back of the camera, etc. are detected. By doing so, it is possible to determine how many times the face direction is to the left. When the face direction is to the right, the left and right sides are only symmetrical with the face direction to the left, so a detailed explanation is omitted.

(Pitch angle: 10 degrees above)
When the face orientation is 10 degrees above (yaw angle: 0 degrees, pitch angle: 10 degrees above), the brightness value of the mouth area is larger than that of the skin area, but the mask is prominent at the left and right edges of the mask. Edges etc. are not detected. The edge on the upper side of the mask has a steeper slope compared to the front of the camera (yaw angle: 0 degrees, pitch angle: 0 degrees), but the edge slope on the upper side of the mask is due to deformation etc. each time the mask is worn. Since it changes, it cannot be determined that the face is facing upward, but if the reference edge gradient of the upper edge of the mask when the mask is worn is known, it is possible to determine the angle of facing the face in the vertical direction. When the face direction is 10 degrees to the left (yaw angle: 10 degrees), the brightness value of the mouth area becomes large as in the case of the pitch angle: 0 degrees, and the face is tilted to the left with respect to the camera. This makes it possible to detect a prominent edge at the left edge of the mask. Furthermore, when the face is tilted to the left, the edge at the left end of the mask, the center line of the face, the face width WA in front of the camera, and the back side of the camera are the same as when the pitch angle is 0 degrees. It is possible to detect the face width WB and the like to determine how many times the face is tilted to the left.

The upward angle (pitch angle) of the face can be determined by the inclination of the center line of the face when the face is facing left 30 degrees, left 40 degrees, left 50 degrees, or the like. Comparing the inclination of the face center line when the face upward angle (pitch angle) is 0 degrees and 10 degrees, and when the face faces left 30 degrees, left 40 degrees, and left 50 degrees, the face upward angle (pitch angle) It can be determined that the face is clearly facing upward when the upward angle (pitch angle) of the face is 10 degrees as compared with the case where the pitch angle) is 0 degrees. If the slope of the upper edge of the mask when the upward angle (pitch angle) of the face is 10 degrees is recorded as a reference, it is compared with the slope of the upper edge of the mask in the image taken later unless the mask is deformed. If the angles are the same (preferably having a constant width and substantially the same angle), it can be determined that the upward angle (pitch angle) of the face is 10 degrees. When the face direction is 10 degrees to the right, the left and right sides are only symmetrical with the face direction to the left by 10 degrees, so detailed explanation is omitted.

(Pitch angle: 20 degrees above)
When the face orientation is up 20 degrees (yaw angle: 0 degrees, pitch angle: up 20 degrees), the brightness value of the mouth area is larger than that of the skin area, but the mask is prominent at the left and right edges of the mask. Edges etc. are not detected. The edge on the upper side of the mask has a steeper slope compared to the front of the camera (yaw angle: 0 degrees, pitch angle: 0 degrees), but the edge slope on the upper side of the mask is due to deformation etc. each time the mask is worn. Since it changes, it cannot be determined that the mask is facing upward, but if the reference edge gradient of the upper edge of the mask when the mask is worn is known, the face orientation angle in the vertical direction can also be determined. When the face is facing left 10 degrees (yaw angle: 10 degrees, pitch angle: top 20 degrees), the brightness value of the mouth area becomes large as in the case of pitch angle: 0 degrees, and the brightness value of the mouth area becomes large. By tilting the face to the left, a prominent edge can be detected at the left edge of the mask. Furthermore, when the face is tilted to the left, the edge at the left end of the mask, the center line of the face, the face width WA in front of the camera, and the back side with respect to the camera are the same as when the pitch angle is 0 degrees. It is possible to detect the face width WB and the like to determine how many times the face is tilted to the left.

The upward angle (pitch angle) of the face can be determined by the inclination of the center line of the face when the face is facing left 30 degrees, left 40 degrees, and left 50 degrees, as in the case of facing upward 10 degrees. Comparing the case of 10 degrees above the face and the case of 20 degrees above the face, the inclination of the center line of the face at 30 degrees to the left, 40 degrees to the left, and 50 degrees to the left is more when it is 20 degrees above the face. Since it is tilted, it can be determined to be 20 degrees above. Also, if the slope of the upper edge of the mask when the upward angle (pitch angle) of the face is 20 degrees is recorded as a reference, it is compared with the slope of the upper edge of the mask in the image taken later unless the mask is deformed. If the angles are the same (preferably having a constant width and substantially the same angle), it can be determined that the upward angle (pitch angle) of the face is 20 degrees. When the face direction is up 20 degrees to the right, the left and right sides are only symmetrical with the face direction up 20 degrees to the left, so a detailed explanation is omitted.

(Pitch angle: 10 degrees below)
When the face orientation is 10 degrees below (yaw angle: 0 degrees, pitch angle: 10 degrees below), the brightness value of the mouth area is larger than that of the skin area, but the mask is prominent at the left and right edges of the mask. Edges etc. are not detected. The edge on the upper side of the mask has a gentler slope compared to the front of the camera (yaw angle: 0 degrees, pitch angle: 0 degrees), but the edge slope on the upper side of the mask is due to deformation etc. each time the mask is worn. Since it changes, it cannot be determined to be downward by this alone, but if the reference edge gradient of the upper edge of the mask when wearing the mask is known, it is possible to determine the vertical face angle (same as when 10 degrees upward). When the face is facing left 10 degrees (yaw angle: 10 degrees, pitch angle: bottom 10 degrees), the brightness value of the mouth area becomes large as in the case of pitch angle: 0 degrees, and the camera is not exposed to the camera. By tilting the face to the left, a prominent edge can be detected at the left edge of the mask. When the face is tilted to the left, the edge of the left edge of the mask, the center line of the face, the face width WA in front of the camera, and the face width on the back side of the camera are the same as when the pitch angle is 0 degrees. It is possible to detect WB and the like to determine how many times the face is tilted to the left.

The downward angle of the face can be determined by the inclination of the center line of the face when the face is facing left 30 degrees, left 40 degrees, and left 50 degrees, as in the case of facing upward 10 degrees. Comparing when the pitch angle is 0 degrees and when the pitch angle is 10 degrees below, the inclination of the face center line at 30 degrees to the left, 40 degrees to the left, and 50 degrees to the left is 10 degrees below the pitch angle. It can be determined that the face is tilted more downward, and it can be determined that the face is tilted 10 degrees downward. If the slope of the upper edge of the mask when the downward angle (pitch angle) of the face is 10 degrees is recorded as a reference, it is compared with the slope of the upper edge of the mask in the image taken later unless the mask is deformed. If the angles are the same (preferably having a constant width and substantially the same angle), it can be determined that the downward angle (pitch angle) of the face is 10 degrees. When the face direction is 10 degrees down to the right, the left and right sides are only symmetrical with the face direction down 10 degrees to the left, so a detailed explanation is omitted.

(Pitch angle: 20 degrees below)
When the face is facing down 20 degrees (yaw angle: 0 degrees, pitch angle: down 20 degrees), the brightness value of the mouth area is larger than that of the skin area, but the mask is prominent at the left and right edges of the mask. Edges etc. are not detected. The edge on the upper side of the mask has a gentler slope compared to the front of the camera (yaw angle: 0 degrees, pitch angle: 0 degrees), but the edge slope on the upper side of the mask is due to deformation etc. each time the mask is worn. Since it changes, it cannot be determined that the mask is facing downward, but if the reference edge gradient of the upper edge of the mask at the time of wearing is known, the face facing angle in the vertical direction can also be determined (same as when the angle is 10 degrees upward). When the face is facing left 10 degrees (yaw angle: 10 degrees, pitch angle: lower 20 degrees), the brightness value of the mouth area becomes large as in the case of pitch angle: 0 degrees, and the camera is not exposed to the camera. By tilting the face to the left, a prominent edge can be detected at the left edge of the mask. When the face is further tilted to the left, the edge at the left end of the mask, the center line of the face, the face width WA in front of the camera, and the face on the back side of the camera are the same as when the pitch angle is 0 degrees. It is possible to detect the width WB and the like to determine how many times the face is tilted to the left.

The downward angle (pitch angle) of the face can be determined by the inclination of the center line of the face when the face is facing left 30 degrees, left 40 degrees, and left 50 degrees, as in the case of facing downward 10 degrees. Comparing the case of 10 degrees below the face and the case of 20 degrees below the face, it is judged that the inclination of the center line of the face at 30 degrees to the left, 40 degrees to the left, and 50 degrees to the left indicates that the face is inclined more downward. It can be determined to be 20 degrees below. If the slope of the upper edge of the mask when the downward angle (pitch angle) of the face is 20 degrees is recorded as a reference, it is compared with the slope of the upper edge of the mask in the image taken later unless the mask is deformed. If the angles are the same (preferably having a constant width and substantially the same angle), it can be determined that the downward angle (pitch angle) of the face is 20 degrees. When the face direction is 20 degrees down to the right, the left and right sides are only symmetrical with the face direction down 20 degrees to the left, so a detailed explanation is omitted.

When the face is turned to the left at 50 degrees, the brightness value of the skin near the left end of the mask may be high, but the string part of the mask to be put on the ear also has a high brightness value due to light irradiation, and the edge is remarkable. Can be detected, so it is also possible to detect the left end of the mask based on that information. From this string edge information, it is also possible to determine the vertical orientation of the face.

In addition, since the face widths WA and WB may differ in the vertical direction of the face (depending on how the vertical line 222 is taken in FIG. 2), the vertical direction of the face is determined, and the face is determined based on the standard for each vertical direction. It is also possible to determine the orientation based on the width.

Further, when irradiating light from the same direction as the camera C, when the face is downward at 20 degrees and the forehead is not shielded by hair, the brightness of the forehead region becomes large. Since the brightness of the forehead region becomes larger than a predetermined threshold value, it may be determined that the degree is 20 degrees downward.

Furthermore, once the mask upper edge gradient, which is a reference in the vertical direction of the face, is determined, the pitch angle in the vertical direction can be determined based on the value. For example, when the upper edge gradient of the mask is determined once at 20 degrees downward and then the face orientation becomes 20 degrees upward, the face orientation is 20 degrees upward based on the reference edge gradient at 20 degrees downward. It is also possible to calculate the edge gradient of the above and compare the value with the edge gradient detected from the image, and if the values are close to each other, it can be determined that the face orientation is 20 degrees upward. The reference edge gradient can be calculated if there is no deformation or attachment / detachment when wearing the mask. It changes when the distance between the face and the camera changes, but if the distance is substantially constant, once a standard is determined, it can be determined based on that standard. Further, even when the distance between the face and the camera changes, the face orientation direction can be determined by changing the reference edge gradient based on the depth information of the face and the size information of the face.

Further, since there are various types of masks, it is possible to determine which type of mask is worn and estimate the face orientation direction based on the shape information peculiar to the mask being worn. According to the research of the inventor, there are various types of masks currently on the market. Regarding the above-mentioned edge of the left end (right end) of the mask, although each mask has its own characteristics, when the face is turned to the left or right. In addition, it has been found that the shape peculiar to the mask at the left and right ends of the mask can be detected for almost all masks by irradiating with light. In addition, the center line of the face (mask) when the face is turned to the left and right also has a different shape depending on the type of mask, but almost all masks have a shape in which the center of the mask protrudes (the face is). Since it has a shape protruding to the center of the face (to fit the shape), it is possible to detect the center of the mask, which is the center of the face, by irradiating light from the camera side. Further, the edge gradient on the upper side of the mask described above varies depending on the type of mask, but once the reference is determined by the above-mentioned process, the vertical angle of the face can be determined based on the reference. can. As a peculiar shape, for example, in the mask shown in FIG. 8A, the shape of the left and right ends of the mask is not the vertical edge line, but the shape of the left and right ends of the mask can be detected. Further, although there is a hemispherical cup-shaped mask as shown in FIG. 8B, it is possible to detect the center of the mask and the slope of the upper end edge of the mask. Further, even with a black mask, when infrared photography is performed, a white image can be obtained although there is a slight difference in the luminance value. Therefore, the mask can be detected and the features at the left and right ends and the upper ends of the mask can be detected in the same manner.

Further, although the mask may be deformed, almost no change occurs in the characteristics of the edges at the left and right ends of the mask described above. Regarding the center line of the mask when the face is turned to the left and right, since the mask changes its shape flexibly, there is a possibility that the center line of the mask cannot be detected due to deformation. In such a case, the center line of the face can be detected not in the mask area but in the upper half of the face when the angle is from 30 degrees to the left to 50 degrees to the left, and the vertical direction of the face can be determined. It is also possible to detect the inclination of the string portion of the mask worn on the ear and determine the vertical direction of the face. The gradient of the upper edge of the mask may also change due to mask deformation, but once the reference gradient is determined, if the mask deformation is taken into consideration and the reference is calculated each time, the upper side of the mask can be calculated. It is also possible to deal with deformation of the slope of the edge.

When the face orientation recognition unit 356 recognizes the orientation of the face, the edge gradient or the like may be extracted based on the brightness value, but the camera parameters (exposure time, etc.) and the irradiation light intensity are changed depending on the shooting conditions. In some cases, one (either one or both) can significantly extract features such as edge gradient. In such a case, the irradiation light adjusting unit 357 irradiates from the irradiation unit L based on the brightness values detected by the mouth area detection unit 352, the mask wearing determination unit 353, the mask feature detection unit 354, and the face feature detection unit 355. The intensity of the light is adjusted, and the camera adjusting unit 358 adjusts the parameters of the camera C.

[3. Operation example of recognition device D]
Next, an example of the face orientation determination process of the recognition device D will be described with reference to the flowchart of FIGS. 9-12.

First, the control unit 311 of the recognition device D acquires an image taken by the camera C while the irradiation unit L irradiates the face F covered with the mask M with light (step S11).

Next, the control unit 311 detects the mouth region of the face reflected in the image acquired in the process of step S11 (step S12).

Next, the control unit 311 performs a mask wearing determination process on the mouth region detected in the process in step S12 (step S13).

Here, the mask wearing determination process will be described with reference to FIG.

First, the control unit 311 calculates the average luminance value Ia of the skin region (for example, the right eye region 401 in FIG. 6) (step S21).

Next, the control unit 311 calculates the average luminance value Ib of the mouth region (for example, the mouth region 402 in FIG. 6) (step S22).

Next, the control unit 311 determines whether or not "Ib-Ia> Ic" is satisfied (step S23). Here, Ic is a threshold value (for example, Ic = 40). Ic can be set according to the observed luminance value and the like. When it is determined that "Ib-Ia> Ic" is satisfied (step S23: YES), the control unit 311 determines that the mask is worn (step S24), and ends the mask wearing determination process. On the other hand, when it is determined that "Ib-Ia> Ic" is not established (step S23: NO), the control unit 311 determines that the mask is not worn (step S25), and ends the mask wearing determination process.

Returning to FIG. 9, the control unit 311 determines whether or not it is determined that the mask is worn in the mask wearing determination process (step S14). If it is determined that the mask is not worn (not worn), the control unit 311 ends the face orientation determination process (step S14: NO). On the other hand, if the control unit 311 determines that the mask is worn (step S14: YES), the control unit 311 then performs a yaw angle determination process (step S15).

Here, the yaw angle determination process will be described with reference to FIG.

First, the control unit 311 determines whether or not the left and right ends of the mask can be detected from the image acquired in the process of step S11 or step S18 described later (step S31). For masks with special shapes at the left and right edges as shown in FIG. 8A, the mask portion having a large luminance value with respect to the luminance value of the skin is first determined and recorded. It can be processed in the same manner in the process of.

When the control unit 311 determines that the left and right edges of the mask cannot be detected (step S31: NO), the control unit 311 determines the yaw angle to be "0 degree" (step S32), and ends the yaw angle determination process. On the other hand, if it is determined that the left and right edges of the mask can be detected (step S31: YES), then it is determined whether or not the face center line (see the face center line 212 in FIG. 2) can be detected (step S33).

When the control unit 311 determines that the face center line cannot be detected (step S33: NO), the control unit 311 determines the yaw angle to be "10 degrees" (step S34), and ends the yaw angle determination process. On the other hand, if it is determined that the face center line can be detected (step S33: YES), then the front side face width WA and the back side face width WB are calculated (see FIG. 3) (step S35).

Next, the control unit 311 determines the yaw angle based on the front face width WA and the back face width WB calculated in the process of step S35 (step S36). In this embodiment, the yaw angle is determined by comparing WB / WA with predetermined threshold values Y1, Y2, and Y3 (Y1> Y2> Y3). The threshold values Y1, Y2, and Y3 may be changed based on the observed values of the distance between the camera C and the face F.

Specifically, the control unit 311 determines the yaw angle as follows.
(1) Y1 <WB / WA → Yaw angle: 20 degrees (2) Y2 <WB / WA <Y1 → Yaw angle: 30 degrees (3) Y3 <WB / WA <Y2 → Yaw angle: 40 degrees (4) WB / WA <Y3 → Yaw angle: 50 degrees

Next, the control unit 311 performs a face center line correction process (step S37), and ends the yaw angle determination process. In the face center line correction process, when the yaw angle with respect to the irradiation unit L becomes large, the brightness value of the mask center portion decreases, and the mask center line detected by the above-mentioned brightness value condition and the face center line do not match. (For example, when the yaw angle is 50 degrees in FIG. 7. If the yaw angle is 40 degrees or less, the mask center line and the face center line are likely to match). Will be done. In this embodiment, when the yaw angle is less than 50 degrees, the line corresponding to the edge formed by the color-coded pixels having a luminance value of 220 or more is set as the face center line. On the other hand, when the yaw angle is 50 degrees or more, the line corresponding to the edge formed by the color-coded pixels having a luminance value of 160 or more and less than 220 is set as the face center line.

Returning to FIG. 9, the control unit 311 performs the pitch angle determination process (step S16).

Here, the pitch angle determination process will be described with reference to FIG.

First, the control unit 311 determines whether or not the inclination of the upper end of the mask has been recorded for all pitch angles (lower 20 degrees, lower 10 degrees, 0 degrees, upper 10 degrees, upper 20 degrees) (step S51). .. At this time, if the control unit 311 determines that the inclination of the upper end of the mask has not been recorded for all pitch angles (step S51: NO), the characteristics of the left and right ends of the mask and the characteristics of the face center line (near the upper end of the mask). The pitch angle is determined based on (step S53). Specifically, as shown in FIGS. 13 (A) and 13 (B), the characteristics of the left and right edges of the mask and the characteristics of the face center line differ depending on the pitch angle, and thus serve as a reference for each pitch angle in advance. The features of the left and right edges of the mask and the face center line are recorded in the storage unit 312, and the features of the left and right edges of the mask and the face center line in the image acquired this time are compared and the most similar features of the left and right edges of the mask and the face center line are obtained. The associated pitch angle is determined to be the pitch angle of the face reflected in the image. In this embodiment, the inclination of the upper string of the mask is recorded as a feature of the left and right ends of the mask, and the inclination near the upper end of the mask is recorded as a feature of the center line of the face.

Hereinafter, a specific description will be given.
-Inclination of the mask upper string when the pitch is 0 degrees (horizontal) recorded in the storage unit 312: m0
-Inclination of the face center line when the pitch is 0 degrees (horizontal) recorded in the storage unit 312: v0
-Inclination of the mask upper string at 10 degrees on the pitch recorded in the storage unit 312: m1
-Inclination of the face center line at 10 degrees on the pitch recorded in the storage unit 312: v1
-Inclination of the mask upper string at 20 degrees on the pitch recorded in the storage unit 312: m2
-Inclination of the face center line at 20 degrees on the pitch recorded in the storage unit 312: v2
-Inclination of the mask upper string at 10 degrees below the pitch recorded in the storage unit 312: m3
-Inclination of the face center line at 10 degrees below the pitch recorded in the storage unit 312: v3
-Inclination of the mask upper string at 20 degrees below the pitch recorded in the storage unit 312: m4
-Inclination of the face center line at 20 degrees below the pitch recorded in the storage unit 312: v4
-Inclination of the mask upper string of the image acquired this time: mA
-Inclination of the face center line of the image acquired this time: vA
When the inclination of the upper string of the mask (an example of "characteristics of the left and right ends of the mask") and the inclination of the face center line are defined as described above, the control unit 331 determines the pitch angle as follows.

(1) [(v3 + v0) / 2 <vA <(v0 + v1) / 2] or / and
[(M3 + m0) / 2 <mA <(m0 + m1) / 2] → Pitch angle: 0 degrees

(2) [(v0 + v1) / 2 <vA <(v1 + v2) / 2] or / and
[(M0 + m1) / 2 <mA <(m1 + m2) / 2] → Pitch angle: Top 10 degrees

(3) [(v1 + v2) / 2 <vA] or / and
[(M1 + m2) / 2 <mA] → Pitch angle: 20 degrees above

(4) [(v3 + v4) / 2 <vA <(v0 + v3) / 2] or / and
[(M3 + m4) / 2 <mA <(m0 + m3) / 2] → Pitch angle: 10 degrees below

(5) [vA <(v3 + v4) / 2] or / and
[mA <(m3 + m4) / 2] → Pitch angle: 20 degrees down

Next, the control unit 311 detects the upper end of the mask (upper boundary 213 in FIG. 2), associates the pitch angle determined in the process of step S53 with the inclination of the upper end of the mask, and records it in the storage unit 312 (step S54). , Ends the pitch angle determination process.

On the other hand, when the control unit 311 determines in the process of step S51 that the inclination of the upper end of the mask has been recorded for all pitch angles (step S51: YES), the recorded inclination of the upper end of the mask is acquired this time. The inclination of the upper end of the mask of the image is compared, the pitch angle is determined (step S52), and the pitch angle determination process is completed. Here, a case where the pitch angle is determined by comparing the inclination of the recorded upper end of the mask with the inclination of the upper end of the mask of the image will be specifically described.

-Inclination of the upper end of the mask when the pitch is 0 degrees (horizontal) recorded in the storage unit 312: g0
-Inclination of the upper end of the mask at 10 degrees above the pitch recorded in the storage unit 312: g1
-Inclination of the upper end of the mask at 20 degrees on the pitch recorded in the storage unit 312: g2
-Inclination of the upper end of the mask at 10 degrees below the pitch recorded in the storage unit 312: g3
-Inclination of the upper end of the mask at 20 degrees below the pitch recorded in the storage unit 312: g4
-Inclination of the upper end of the mask of the image acquired this time: gA
When the inclination of the upper end of the mask is defined as described above, the control unit 331 determines the pitch angle as follows.

(1) [(g3 + g0) / 2 <gA <(g0 + g1) / 2] → Pitch angle: 0 degrees

(2) [(g0 + g1) / 2 <gA <(g1 + g2) / 2] → Pitch angle: Top 10 degrees

(3) [(g1 + G2) / 2 <gA] → Pitch angle: 20 degrees above

(4) [(g3 + g4) / 2 <gA <(g0 + g3) / 2] → Pitch angle: 10 degrees below

(5) [gA <(g3 + g4) / 2] → Pitch angle: 20 degrees below

Returning to FIG. 9, the control unit 311 determines whether or not to continue the face orientation determination process (step S17). At this time, when the control unit 311 determines that the face orientation determination process is not continued (step S17: NO), the control unit 311 ends the face orientation determination process. On the other hand, when the control unit 311 determines that the face orientation determination process is to be continued (step S17: YES), the control unit 311 acquires the next image (step S18) and proceeds to the process of step S15.

As described above, the control unit 311 (an example of the “acquisition means”, the “detection means”, and the “recognition means”) of the recognition device D according to the present embodiment is covered with a mask (an example of the “covering member”). An image taken with the face (an example of the "object") irradiated with light by the irradiation unit L (an example of the "irradiation means") is acquired, and based on the brightness value of the pixels included in the acquired image, the image is taken. The face center line 212 (an example of "the center line in the front vertical direction on the surface of the target") is detected, and the orientation of the target is recognized based on the detection result.

Therefore, according to the recognition device D according to the present embodiment, the orientation of the face is recognized based on the detection result of the face center line 212 based on the brightness value of the pixels included in the image, so that the face is covered with the mask. Even if it is present, the orientation of the face can be accurately recognized. In addition, the orientation of the face can be recognized without performing high processing load processing such as pattern matching. Further, since it is not necessary to store the pattern image for pattern matching, the storage unit 312 can be used as a storage device having a small capacity.

Further, the control unit 311 of the recognition device D according to the present embodiment recognizes the vertical direction (pitch angle) of the face based on the inclination of the face center line 212.

Further, the control unit 311 (an example of "detection means", "memory control means", and "recognition means") of the recognition device D according to the present embodiment is the upper side of the mask when recognizing the vertical orientation of the face. The upper boundary 213, which is the boundary of the above, is further detected, and the inclination of the upper boundary 213 is stored in the storage unit 312 (an example of the "storage means") in association with the vertical orientation of the recognized object, and the vertical orientation. When a new image of a face that does not recognize is acquired, the vertical orientation associated with the inclination of the upper boundary 213 of the mask detected for the image is substantially the same as that of the image. Recognize as the vertical orientation of the reflected face. As a result, once the vertical orientation of the face is recognized, the upper boundary 213 is detected for the subsequent images and compared with the upper boundary 213 stored in the storage unit 312, thereby performing the vertical direction. The direction of the direction can be recognized. That is, the processing load for recognizing the vertical orientation is reduced.

Furthermore, the control unit 311 (an example of the "recognition means") of the recognition device D according to the present embodiment is based on the position of the intersection of the upper boundary line 213 and the face center line 212 in the width of the face (that is,). (Based on the front side face width WA and the back side face width WB), the lateral orientation (yaw angle) of the face is recognized.

The recognition device D according to this embodiment includes an in-vehicle camera (driver safety confirmation by face orientation detection), a surveillance camera (face orientation detection when wearing a mask, human interest and behavior detection, etc.), and a medical camera (mask wearing). It can be utilized by incorporating it into face detection of time, behavior detection of doctors and patients, etc.).

[4. Modification example]
Next, a modification of the above embodiment will be described. The modifications described below can be combined as appropriate.

[4.1. Modification 1]
In the above embodiment, as shown in FIGS. 13A and 13B, the inclination of the reference mask upper string (an example of "characteristics of the left and right ends of the mask") and the inclination of the face center line (near the upper end of the mask) ( An example of "features of the face center line") is recorded in the storage unit 312 in association with the pitch angle, and the pitch angle is calculated by comparing the inclination of the mask upper string and the inclination of the face center line in the newly acquired image. Instead of this, as shown in FIGS. 14A and 14B, the characteristics (shape) of the left and right ends of the mask and the characteristics (shape) of the center line of the face (the part on the mask) are determined. ) Is recorded in the storage unit 312 as it is in association with the pitch angle, and the features (shape) of the left and right edges of the mask and the features (shape) of the face center line in the newly acquired image are compared and the highest degree of similarity is obtained. The pitch angle associated with the object may be acquired and determined as the pitch angle of the face F appearing in the image.

[4.2. Modification 2]
In the above embodiment, it is assumed that the camera C and the irradiation unit L are facing the target from substantially the same direction, but the present invention is also applied to the case where the camera C and the irradiation unit L are facing the target from different directions. Can be done. That is, even when the camera C and the irradiation unit L are located in different directions with respect to the object, the position of the irradiation unit L is fixed, and how light is applied to the face when wearing the mask to capture an image. If it is known in advance, it is possible to estimate the face orientation direction by determining the mouth region, determining the wearing of the mask, and detecting the mask feature and the face feature, as in the above embodiment.

FIG. 15 is an example of an image taken in a state where the camera C and the irradiation unit L are arranged in different directions when viewed from the target and the light is irradiated from the lateral direction of the face. When the light is irradiated from the side of the face, the center line 601 of the face can be detected when the face faces the vicinity of the front of the camera. When facing the direction of the light, the contour of the face, the feature 602 of the mask facing the direction of the light, and the front area of the face can be detected. These can be detected and similarly determined every 10 degrees of yaw angle. However, when irradiating light from the lateral direction of the face, if the face is oriented in the opposite direction to the irradiation of the light, the mask features and the features on the front surface of the face due to the light irradiation cannot be detected, so that the range is wider. When it is desired to detect the face orientation angle, it is desirable to install the camera C and the irradiation unit L in the same direction when viewed from the face, as in the above embodiment. Depending on the application and the environment, light may be irradiated from the lateral direction of the face to detect the feature 602 of the mask and the like to determine the direction of the face.

In this way, even when the camera C and the irradiation unit L are arranged in different directions when viewed from the target, the mask feature and the front surface feature of the face can be clearly detected. For example, when the irradiation unit L is arranged away from the camera C to irradiate the light from the lateral direction of the face, or when the dominant light (for example, sunlight) is irradiated from the lateral direction of the camera C. Even if it is done, the face orientation can be determined by the same processing.

[4.3. Modification 3]
In the determination of the mouth region of the face and the determination of the face orientation angle, the results of determination in the past captured images may be used for the above determination. For example, once the mouth region is estimated, in the image taken at the next time, a process of searching the mouth region around the past estimation region can be considered. Further, in the determination of the face orientation, for example, if it is determined to be 10 degrees above the face and 20 degrees to the left, it is highly possible that the image taken at the next time is oriented toward the periphery of the face estimated in the past. It is conceivable to perform a process of preferentially detecting the features of 20 degrees to the left of the face, 10 degrees to the left of the face, 10 degrees to the left of the face, and 20 degrees to the left of the face. That is, assuming that the orientation of the face does not change significantly in the images taken at a close time, when newly determining the face orientation, the determination processing may be performed from the direction close to the most recently determined face orientation. can. By processing the data in time series and effectively performing the determination process as in this modification, it is possible to reduce the processing load for the determination of the mouth region and the determination of the face orientation angle.

[4.4. Modification 4]
As a result of the implementation by the inventor of the present application, it was found that in many masks, the brightness value of the mask region is larger than that of the skin in the infrared captured image. However, depending on the material of the mask, for example, when a material that absorbs infrared rays is used, the mask region may be imaged darker (lower brightness value) than the skin in the infrared image. In such a case, it may be determined that the brightness value of the mouth region is smaller than that of the skin region, and the mask may be detected. In this case, after the mask is detected, the process of detecting the mask edge and the like can be performed in the same manner as in the above-described embodiment.

1 Recognition device 111A Acquisition means 111B Detection means 111C Recognition means 111D Storage control means 111E Storage means D Face recognition device D
311 Control unit 312 Storage unit 313 Communication unit 314 Display unit 315 Operation unit

Claims (1)

An acquisition means for acquiring an image taken in a state where an object covered with a covering member is irradiated with light by an irradiation means, and an acquisition means.
A detection means for detecting a center line in the front vertical direction on the surface of the target based on the luminance value of the pixels included in the image.
A recognition means that recognizes the orientation of the target based on the detection result of the detection means, and
A recognition device equipped with.

Images