JP2021009503A - Personal data acquisition system, personal data acquisition method, face sensing parameter adjustment method for image processing device and computer program - Google Patents

Abstract

To provide a personal data acquisition system that can improve accuracy of face sensing for a specific individual.SOLUTION: A personal data acquisition system includes: an imaging unit C that is installed in front of an imaging target and can acquire a distance image; a captured image data storage unit 1 that stores the distance image; and an image processing calculation unit (PC) that processes an image input from the captured image data storage unit 1 and executes an instruction. The image processing calculation unit has a feature point matching unit 7a, a normal vector calculation unit 7b, a reference face orientation angle image definition unit 7c, and a relative face orientation angle calculation unit 7d. The instruction points to multiple points of viewing points.SELECTED DRAWING: Figure 6

Description

The present invention relates to a personal data acquisition system, a personal data acquisition method, a method for adjusting face sensing parameters for an image processing device, and a computer program.

Patent Document 1 below discloses a robot device used as a service providing device that can switch to an appropriate service according to the situation of a target (person) to which the service is provided.

The robot device is equipped with a first camera, a second camera, and an information processing device including a CPU, and the CPU includes a face detection unit, an attribute determination unit, a person detection unit, a person position calculation unit, and a movement unit. It is equipped with functions such as a vector detector.

According to the robot device, when the service is provided to a group of people who have a relationship such as communicating with each other, the first service is provided to provide information based on close communication. To determine.
On the other hand, when the service is provided to a group of people whose relationship such as communication with each other is unknown, the second method of providing information unilaterally without exchanging information. Decide to provide the service. With these, it is possible to provide appropriate services according to the situation of the service provision target.

Further, Non-Patent Document 1 below discloses a face orientation estimation method using a monocular distance camera, and a verification experiment for verifying the accuracy of the face orientation estimation method is carried out in addition to the estimation method. There is. In this verification experiment, an acceleration sensor (3-axis acceleration sensor) is attached to the subject's head to obtain the true face orientation value, and the accuracy of the face orientation estimation method is verified by comparing with this true value. There is.

[Problems to be solved by the invention]
In the robot device disclosed in Patent Document 1, the face detection unit is configured to detect a person's face using the first camera, and a known technique is used for the face detection. Can be done.
However, with conventional face detection technology, if a part of the facial organs such as eyes, nose, and mouth is missing or greatly deformed due to injury, a large mole, swelling, or body decoration such as tattoo will appear on the face. Assuming that there is a difference in the age and gender of a specific individual (general person, age, gender, etc.), such as when it is given or when the arrangement of the facial organs deviates significantly from the average arrangement due to a genetic disease. However, there is a problem that the accuracy of face detection for (referring to an individual having characteristics different from the common facial features) is lowered.

Further, in the verification experiment of the face orientation estimation method disclosed in Non-Patent Document 1, a 3-axis acceleration sensor is attached to the head of the subject to obtain a true value for verification. However, for example, when trying to apply the above face orientation estimation method to an actual driver monitoring device or the like, it is necessary to have the sensor attached to the user's head, which is difficult to realize in the actual field. There was a problem such as there.

Further, if the image taken by the multi-angle shooting system as shown in FIG. 17 can be used for the verification of the face orientation estimation method, the accuracy can be considerably improved. However, when trying to apply it to an actual driver monitoring device, such a large-scale multi-angle shooting system must be prepared, for example, at all dealer stores located all over the country. At the actual site, there was a problem that it was difficult to realize.

Japanese Unexamined Patent Publication No. 2014-14899

Information Processing Society of Japan Research Report Vol.2014-MBL-73 No.18 / Vol.2014- ITS-59 No.18 2014/11/21 "Estimation of Face Orientation Using Monocular Distance Camera" Graduate School of Natural Science and Technology, Kumamoto University

Means for solving problems and their effects

The present invention has been made in view of the above problems, does not impose a burden on the user, does not require the preparation of a large-scale device, and even if it is a simple system, the face for a specific individual as described above. It is an object of the present invention to provide a personal data acquisition system, a personal data acquisition method, a method of adjusting face sensing parameters for an image processing device, and a computer program capable of improving the accuracy of sensing.

In order to achieve the above objectives, the personal data acquisition system (1) pertaining to this disclosure is
An imaging unit that is installed in front of the imaging target and can acquire distance images,
An image data storage unit that stores the distance image and
An image processing calculation unit that processes an image input from the image data storage unit,
Instruction department and
It is a personal data acquisition system configured to include
The image processing calculation unit has a feature point matching unit, a normal vector calculation unit, a reference face orientation angle image definition unit, and a relative face orientation angle calculation unit.
The instruction unit is characterized in that it points to a plurality of visual recognition points.

The instruction unit is configured by, for example, attaching a mark M having a number 1 to 9 on a wall or the like.
According to the personal data acquisition system (1), by introducing the instruction unit, the user's head is not equipped with a 3-axis acceleration sensor or the like, in other words, the personal data is not burdened on the user. Can be obtained.
In addition, the equipment required to actually configure the system is only a camera and a personal computer, and there is no need to prepare a large-scale device as shown in FIG. 17, which is an extremely small and simple device that keeps costs down. It is possible to acquire high-precision face-to-face angle true value data for an individual.

In addition, the personal data acquisition system (2) according to the present disclosure is the personal data acquisition system (1) described above.
Further, it is characterized in that it includes a display unit and an operation unit.

The display unit (display) and operation unit (keyboard, mouse, etc.) are also components of a personal computer, and even in the personal data acquisition system (2), a personal data acquisition system is realized with an extremely small and simple device. can do.

Further, in order to achieve the above object, in the personal data acquisition system (3) according to the present disclosure, in the personal data acquisition system (1) or (2), the imaging unit uses the distance image and the distance image. It is characterized in that it can acquire a two-dimensional image such as RGB or infrared light having the same viewpoint as the above.

According to the personal data acquisition system (3), the imaging unit is configured to be able to acquire the distance image and a two-dimensional image of the same viewpoint, and by this configuration, learning / setting of face sensing parameters can be performed. It is possible to obtain data for performing verification with high accuracy.

In addition, the personal data acquisition system (4) according to the present disclosure is the personal data acquisition system (1) to (3) described above.
The image processing calculation unit is further characterized in that it includes a face sensing parameter setting unit and a face sensing parameter verification unit.

According to the personal data acquisition system (4), it is possible to obtain data for highly accurate learning / setting / verification of face sensing parameters as in the case of the personal data acquisition system (3). , It is possible to verify the set face sensing parameters with high accuracy.

In addition, the personal data acquisition system (5) according to the present disclosure is the personal data acquisition system (1) to (4) described above.
Furthermore, it is characterized by having a background curtain.

According to the personal data acquisition system (5), by adopting the optimum color and pattern background curtain for acquiring personal data, the data is more suitable for learning, setting, and verifying face sensing parameters. Can be easily obtained.

Further, the personal data acquisition method (1) according to the present disclosure is a personal data acquisition method using any of the above personal data acquisition systems (1) to (5).
An imaging unit capable of acquiring the distance image is placed in front of the user.
It is characterized in that an instruction based on the instruction unit is given to the user, and image data whose face orientation is changed in a predetermined order and at a predetermined time interval based on the instruction unit is acquired.

According to the above personal data acquisition method (1), it is possible to realize a more reproducible and stable face-facing angle by using an extremely small-scale and simple-structured personal data acquisition system that suppresses costs. , It is possible to obtain more suitable data for learning, setting, and verification of face sensing parameters.

Further, the personal data acquisition method (2) according to the present disclosure is a personal data acquisition method using any of the above personal data acquisition systems (1) to (5).
A step of defining 3D image data in which the depth of facial feature points such as eyes and mouth is almost constant (within a predetermined threshold) as an image with a reference face orientation angle (pitch angle 0 degrees, yaw angle 0 degrees), and
Steps to select common facial feature points in image I and image J in the image series,
Steps for matching facial feature points in image I and image J,
The step of calculating the normal vector of the face-oriented plane consisting of the face feature points of image I,
Steps to calculate the normal vector of the face-oriented plane consisting of the face feature points of image J,
The step of calculating the relative angle of the face orientation of the image I with respect to the image J (calculating the relative normal vector of the face orientation plane), and
It is characterized by containing.

According to the personal data acquisition method (2), the common facial feature points between the two images can be accurately matched, and the height is high based on the reference facial angle image set with high accuracy. It is also possible to calculate the relative face angle with accuracy.

Further, the method (1) for adjusting the face sensing parameter for the image processing device according to the present disclosure is described.
This is a method for adjusting face sensing parameters for an image processing device, which is performed using personal data acquired by using any of the personal data acquisition systems (1) to (5).
Steps to input 2D image data group and
Steps to learn and set face sensing parameters based on the input 2D image data group,
Steps to capture the set face sensing parameters and
Steps to perform face sensing processing trials (face detection, face orientation estimation, etc.) and
Steps to capture and output face sensing result groups for 2D image data group,
A step to capture and output a true value group such as a face detection rectangle and a face orientation angle corresponding to a two-dimensional image data group, and
A step of verifying the face sensing parameter using the face sensing result group and the true value group, and
Compare whether the absolute value of the difference between the face detection rectangle result and the true value and the absolute value of the difference between the face orientation angle result and the true value are smaller than the predetermined threshold value.
If it is judged to be small, the process ends, and if it is judged not to be small, the step to return to the first step and
It is characterized by containing.

According to the method (1) for adjusting the face sensing parameter for the image processing device, the face sensing parameter can be adjusted to the desired high accuracy.

In addition, the computer program (1) according to the present disclosure is
A computer program for causing at least one or more computers to perform adjustment processing of face sensing parameters for an image processing device.
To at least one of the above computers
Steps to capture 2D image data and
Steps to learn and set face sensing parameters based on the captured 2D image data group,
Steps to capture the set face sensing parameters and
Steps to import and output 2D image data groups for learning and setting face sensing parameters,
Steps to perform face sensing processing trials, face detection, face orientation estimation, etc.
Steps to capture and output face sensing result groups for 2D image data group,
A step to capture and output a true value group such as a face detection rectangle and a face orientation angle corresponding to a two-dimensional image data group, and
A step of verifying the face sensing parameter using the face sensing result group and the true value group, and
It is compared whether the absolute value of the difference between the face detection rectangle result and the true value is smaller than the predetermined threshold value, and whether the absolute value of the difference between the face orientation angle result and the true value is smaller than the predetermined threshold value.
If it is determined that it is small, the process is completed, and if it is determined that it is not small, the step of returning to the first step is performed.
It is characterized by executing.

According to the computer program (1), it is possible to improve the adjustment of face sensing parameters to a desired high accuracy.

(A) to (c) are schematic flow charts showing the personal data acquisition in the personal data acquisition system according to the embodiment of the present disclosure, and the subsequent flow. It is a top view which shows the schematic structure of the personal data acquisition system which concerns on embodiment. It is a side view which shows the schematic structure of the personal data acquisition system which concerns on embodiment. It is a schematic diagram which shows the example of the instruction at the time of personal data acquisition which concerns on embodiment. It is a schematic diagram which shows the situation which the image series after the personal data acquisition by the instruction which concerns on embodiment is constructed. It is a block diagram which shows the schematic functional structure of the personal data acquisition system which concerns on embodiment. It is a flowchart which shows the personal data acquisition method in the personal data acquisition system which concerns on embodiment. It is a schematic diagram which looked at the face from the front which shows the example of a face feature point. (A) and (b) are schematic plan views showing the case where the arrangement of facial feature points is viewed from above. (A) and (b) are schematic plan views showing the case where the arrangement of other facial feature points is viewed from above. (A) and (b) are distance images showing the feature points used for calculating the face orientation angle. (A) and (b) are distance images showing the feature points used for the calculation when the face orientation angle becomes deep. It is a flowchart which shows the learning / adjustment process which improves the accuracy by using the personal data in the personal data acquisition system which concerns on embodiment. It is a schematic diagram which shows an example of the in-vehicle system including the driver monitoring apparatus which concerns on embodiment. It is a block diagram which shows an example of the hardware composition of the in-vehicle system including the driver monitoring apparatus which concerns on embodiment. It is a block diagram which shows the functional structure example of the image processing part of the driver monitoring apparatus which concerns on embodiment. It is a photograph which shows the partial structure of a multi-angle photography system.

Hereinafter, embodiments of a personal data acquisition system, a personal data acquisition method, a method for adjusting face sensing parameters for an image processing device, and an embodiment of a computer program according to the present invention will be described with reference to the drawings.

[Application example of personal data acquisition system]
1 (a) to 1 (c) are schematic flow charts showing the flow of personal data acquisition and subsequent work processes in the personal data acquisition system according to the embodiment. (A) shows a step of acquiring personal data, and (b) uses the acquired personal data so that the performance of (face sensing (face detection, face orientation estimation, etc.) can be fully exhibited. The step of learning / adjusting the parameters and creating a memory for image processing in which the result of the learned / adjusted parameters is reflected is shown in (c). (C) shows the created memory, for example, a driver Morita. It is shown to be mounted on a ring device.

FIG. 2 is a plan view showing a schematic configuration of the personal data acquisition system according to the embodiment, and shows a state in which the user U is sitting on a chair. A black blackout curtain B as a background curtain is arranged behind the user U, and as a camera C capable of acquiring a distance image 60 to 80 cm in front of the user U, for example, ToF (Time of Flight). A camera is arranged, and a personal computer P is connected to the ToF camera C.
It is desirable that the size of the prepared room for photography has a width of 1.2 to 2 m and a length of about 2 to 2.5 m.
The dealer's staff D will be in charge of shooting, for example, and will execute the shooting.

FIG. 3 is a side view showing a schematic configuration of the personal data acquisition system according to the embodiment.
The ToF camera C is firmly fixed to the tripod S so as to realize stable shooting.

FIG. 4 shows an example of the instruction at the time of shooting, and a mark M on which the numbers 1 to 9 are written is attached to the wall or the like, and “5-2-5-8-5-2-1-2-3” is shown. Look at the numbers in the order of -2-5-4-5-6-5-8-7-7-8-9-8 and change your face (not your eyes). ] And so on. Marks 1, 4, and 7 are positions where the user's face direction is about 90 degrees to the left, marks 2, 5, and 8 are front positions (0 degrees), and marks 3, 6, and 9 are positions where the user's face direction is about right angles. It is pasted at a position that is 90 degrees. In addition, the marks 1, 2 and 3 are positions where the user's face orientation is about 30 degrees upward, the marks 4, 5 and 6 are front positions (0 degrees), and the marks 7, 8 and 9 are the user's face orientation. Marks M are arranged and pasted at positions where is about 30 degrees downward.

FIG. 5 shows image sequences 0-7 obtained by the instructions shown in FIG. At position A, images with reference face angles of image series 0, 1, 2, and 3 are obtained, at position B, images with reference face angles of image series 4 and 5 are obtained, and at position C, images with reference face angles are obtained. It is configured so that images of reference face angle of 6 and 7 can be obtained.

FIG. 6 is a block diagram showing a schematic functional configuration of the personal data acquisition system. A captured image data storage unit 1 is connected to the ToF camera C, and the captured image data storage unit 1 includes a two-dimensional image data storage unit 1a and a three-dimensional distance image data storage unit 1b, and the captured image data storage unit 1 Is connected to the display unit 2 and the operation unit 3 via a bus, and is further connected to a CPU (Central Processing Unit) 4, a ROM (Read Only Memory) 5, and a RAM (Random Access Memory) 6. The RAM 6 includes an image storage unit 6a and a common feature point storage unit 6b. The CPU 4, ROM 5, and RAM 6 are connected to form an image processing calculation unit 7. The image processing calculation unit 7 includes a feature point matching unit 7a, a normal vector calculation unit 7b, a reference face orientation angle image definition unit 7c, a relative face orientation angle calculation unit 7d, a face sensing parameter setting unit 7e, and a face sensing parameter verification unit 7f. It has.

FIG. 7 is a flowchart showing a personal data acquisition method in the personal data acquisition system.
First, in step S1, the three-dimensional image data group consisting of the distance image data group is captured. These three-dimensional image data groups are photographed by the ToF camera C and are stored in the three-dimensional distance image data storage unit 1b.

Next, in step S2, the three-dimensional image data in which the depths of facial feature points such as eyes and mouth are almost constant (within a predetermined threshold) are converted into an image having a reference face orientation angle (pitch angle 0 degrees, yaw angle 0 degrees). Is defined as.

FIG. 8 shows an example of facial feature points used for calculating the face orientation angle, the feature point e1 shows the outer corner point of the right eye, and the feature point e2 shows the inner corner point of the right eye. The feature point e3 indicates the inner corner point of the left eye, and the feature point e4 indicates the outer corner point of the right eye. Further, the midpoint of the feature point e1 and the feature point e2 and the midpoint of the feature point e3 and the feature point e4 may be set as the feature points, respectively.

Further, the feature point m1 indicates the right corner point of the mouth, and the feature point m2 indicates the left corner point of the mouth. Further, the midpoint between the feature point m1 and the feature point m2 may be set as the feature point.

9 (a) and 9 (b) show the case where the facial feature points are arranged from above, and the feature points e1 to e4 and the feature points m1 and m2 are arranged in a plane. The face orientation can be obtained by calculating the normal vector n with respect to this plane.

If the state of FIG. 9 (a) is defined as the front direction (pitch angle 0 degrees, yaw angle 0 degrees), the normal vector n can be read as 10 degrees to the right in the state of FIG. 9 (b). It is possible to obtain the face orientation angle.

10 (a) and 10 (b) show a case where the lateral angle is deeper than the state of FIGS. 9 (a) and 9 (b), and the feature point e1 and the like are in a difficult-to-detect state. The feature point e5 (ears) or the like is adopted instead of the feature point e1 or the like, indicating a state in which the face orientation angle can be continuously and accurately obtained.

11 (a) and 11 (b) are distance images showing an example in which the lateral angle of the face is shallow and the feature points of both eyes and the mouth are used to calculate the facial angle, and FIGS. 12 (a) and 12 (b) are shown. Is a distance image showing an example in which the lateral angle of the face becomes deeper and the feature points of the eyes, mouth, and ears are used to calculate the facial angle.

Following step S2 in which the image of the reference face orientation angle is defined, in step S3, K of the image series K shown in FIG. 5 is set to 0, and in step S4, J of one image J of the image series K is set to 0. In step S5, I of one image I of the image series K is set to 1.

Next, in step S6, the selection of the common facial feature points in the image I and the image J in the image series K is incorporated. The selection of the common facial feature points may be made manually.

Next, in step S7, matching of facial feature points in image I and image J is performed.
This matching is performed using, for example, a technique such as template matching.

Next, in step S8, the normal vector of the face-facing plane consisting of the face feature points of the image I is calculated using the least squares method.
The details of step S8 are as described above with reference to FIGS. 8 to 12.

Next, in step S9, the normal vector of the face-facing plane consisting of the face feature points of the image J is calculated by the least squares method as in step S8.

In step S10, the relative angle of the face to the image J of the image I (the angle formed by the relative normal vector of the face plane) is calculated as described with reference to FIGS. 9 and 10.

Next, in step S11, J = I is set, in other words, the image J is advanced by one screen.

In step S12, the image I is advanced by one screen.

Next, in step S13, it is determined whether or not the image I is smaller than the number of images numI of the image sequence K, in other words, whether or not the image I has reached the number of images of the image sequence K, and it is determined that the image I is small. Then, the process returns to step S6, and if it is determined that the image is not small, the process proceeds to step S14. In step S14, one K of the image sequence K is added.

Next, in step S15, it is determined whether or not the image sequence K is smaller than the number of image sequences numK, in other words, whether or not the image sequence K has reached the number of sequences, and if it is determined that the number has not been reached, Return to step S4. On the other hand, if it is determined that the image sequence K has reached the number of sequences, that is, the flow is terminated.

FIG. 8 shows an example of facial feature points used for calculating the face orientation angle, and is a schematic view of the face viewed from the front.
The feature points e1 to e4 indicate the feature points of the eye portion, and the feature points m1 and m2 indicate the feature points of the mouth portion.

9 (a) and 9 (b) are schematic plan views showing the arrangement of facial feature points when viewed from above, and show a state in which feature points e1 to e4 and feature points m1 and m2 are arranged. 10 (a) and 10 (b) are schematic plan views showing an arrangement of feature points e3, e4, e5, and m2 including other facial feature points of eyes, ears, and mouth when viewed from above.

11 (a) and 11 (b) are distance images showing an example when the feature points of both eyes and the mouth are used for calculating the face orientation angle, and FIGS. 12 (a) and 12 (b) show the face orientation is more lateral. , This is a distance image showing an example when the feature points of the eyes, mouth, and ears are used for calculating the face orientation angle.

FIG. 13 is a flowchart showing a learning / adjusting process for improving accuracy by using personal data in a personal data acquisition system.

First, in step S21, a two-dimensional image data group acquisition process for learning and setting face sensing parameters is performed.

Next, in step S22, the face sensing parameters are learned and set. For this learning / setting, for example, a two-dimensional image data group (a data group in which moving image data is a plurality of still images) acquired by a dealer is used.

Next, in step S23, the face sensing parameter acquisition process learned and set in step S22 is performed.

In step S24, trials of face sensing processing such as face detection and face orientation estimation are performed.

Next, in step S25, the face sensing result group is captured for the two-dimensional image data group such as the face detection result and the face orientation angle estimation result.

Next, in step S26, a process of capturing a true value group corresponding to a two-dimensional image data group such as a face detection rectangle and a face orientation angle is performed.

In step S27, the face sensing result group for the two-dimensional image data group such as the face detection result and the face orientation angle estimation result in step S25, and the two-dimensional image such as the face detection rectangle and the face orientation angle in step S26. Face sensing parameters are verified based on the true value group corresponding to the data group.

In step S28, it is compared whether the absolute value of the difference between the face detection rectangle result and the true value is smaller than the predetermined threshold value, and whether the absolute value of the difference between the face orientation angle result and the true value is smaller than the predetermined threshold value. Etc. are compared, and if it is judged to be small, the process ends. On the other hand, if it is determined that the size is not small, the process returns to step S21, and adjustment / relearning of the face sensing parameters is performed.

Hereinafter, an image processing device, a monitoring device, and a control system to which the personal data acquisition system according to the present invention is actually applied will be described with reference to the drawings.

The applied image processing device can be widely applied to, for example, a device or system for monitoring an object such as a person using a camera. For example, a person who operates, monitors, or performs predetermined work on various equipment such as machines and devices in a factory, in addition to devices and systems that monitor drivers (operators) of various moving objects such as vehicles. It can also be applied to devices and systems that monitor such things.

[Application example]
FIG. 14 is a schematic view showing an example of an in-vehicle system including the driver monitoring device according to the embodiment. In this application example, an example in which the image processing apparatus is applied to the driver monitoring apparatus 30 will be described.

The in-vehicle system A includes a driver monitoring device 30 that monitors the state of the driver DP of the vehicle EC (for example, facial behavior), and one or more ECUs (Electronic Control Units) that control the running, steering, or braking of the vehicle EC. ) 40, and one or more sensors 41 that detect the state of each part of the vehicle, the state around the vehicle, and the like are included, and these are connected via the communication bus 43. The in-vehicle system A is configured as, for example, an in-vehicle network system that communicates according to a CAN (Controller Area Network) protocol. As the communication standard of the in-vehicle system A, a communication standard other than CAN may be adopted. The driver monitoring device 30 is an example of the "monitoring device", and the in-vehicle system A is an example of the "control system".

The driver monitoring device 30 transmits information based on image processing by the camera 31 for capturing the face of the driver DP, the image processing unit 32 that processes the image input from the camera 31, and the image processing unit 32, and the communication bus 43. It is configured to include a communication unit 36 that performs a process of outputting to a predetermined ECU 40 via the above. The image processing unit 32 is an example of the "image processing device" of the present invention. The camera 31 is an example of the "imaging unit" of the present invention.

The driver monitoring device 30 detects the face of the driver DP from the image captured by the camera 31, and detects the behavior of the face such as the direction of the face of the detected driver DP, the direction of the line of sight, or the open / closed state of the eyes. The driver monitoring device 30 may determine the state of the driver DP, for example, a state of forward gaze, inattentiveness, dozing, backward facing, prone, etc., based on the detection results of these facial behaviors. Further, the driver monitoring device 30 outputs a signal based on the state determination of the driver DP to the ECU 40, and the ECU 40 performs attention and warning processing to the driver DP or operation control of the vehicle EC (for example, deceleration) based on the signal. Control, guidance control to the road shoulder, etc.) may be executed.

One of the purposes of the driver monitoring device 30 is to improve the accuracy of face sensing for a specific individual.

In the conventional driver monitoring device, the driver DP of the vehicle EC has a part of the facial organs such as eyes, nose, and mouth missing or greatly deformed due to, for example, an injury, or a large hokuro or squid on the face, or Accuracy of detecting the face from the image captured by the camera when the facial organs are displaced from the average position due to body decoration such as tattoo or a disease such as a hereditary disease. There is a problem that the accuracy of estimating the face orientation is lowered.

Further, if face sensing processing such as face detection and face orientation estimation is not properly performed, state determination such as inattentiveness or dozing of the driver DP cannot be appropriately performed, and the ECU 40 is based on the state determination. There is a problem that various controls to be executed may not be properly performed.

In order to solve the problem, the driver monitoring device 30 according to the embodiment is a general person (ordinary person) who is common regardless of a specific individual, in other words, an age difference, and a difference such as gender (individual difference). In order to improve the accuracy of face sensing processing for a specific individual who has features different from those of a person), the following configuration was adopted.

The image processing unit 32 has learned facial features such as face sensing processing (face detection, face orientation estimation, etc.) from the image as learned facial features, such as a specific individual's face feature and a normal face feature. (In other words, the amount of facial features used when the person is a person other than a specific individual) is stored.

The image processing unit 32 performs face detection processing for detecting a face region while extracting a feature amount for detecting a face from the input image of the camera 31. Then, the image processing unit 32 determines whether or not the face in the face region is the face of the specific individual by using the detected feature amount of the face region and the face feature amount of the specific individual. Performs specific individual judgment processing.

In the specific individual determination process, an index showing the relationship between the feature amount extracted from the face region and the face feature amount of the specific individual, for example, a correlation coefficient is calculated, and based on the calculated correlation coefficient, It may be determined whether or not the face in the face region is the face of the specific individual.
For example, when the correlation coefficient is larger than a predetermined threshold value, it is determined that the face in the face region is the face of the specific individual, and when the correlation coefficient is equal to or less than the predetermined threshold value, the face in the face region is It may be determined that it is not the face of the specific individual. In the specific individual determination process, an index other than the correlation coefficient may be used.

Further, in the specific individual determination process, it may be determined whether or not the face in the face region is the face of the specific individual based on the result of determination for one frame of the input image from the camera 31. Based on the result of determination for a plurality of frames of the input image from the camera 31, it may be determined whether or not the face in the face region is the face of the specific individual.

As described above, in the driver monitoring device 30, the learned facial feature amount of the specific individual is stored in advance in the image processing unit 32, and the face feature amount of the specific individual is used to obtain the face of the specific individual. It is possible to accurately determine whether or not it is.

Further, when the face of the specific individual is determined by the specific individual determination process, the image processing unit 32 executes the face image process for the specific individual by using the face feature amount of the specific individual. It is possible to accurately perform the face image processing of the specific individual.
On the other hand, when it is determined that the face is not the face of the specific individual, in other words, a normal face (in other words, a face other than the specific individual), the image processing unit 32 uses the normal face feature amount. Since the normal face image processing is executed, the normal face image processing can be performed with high accuracy. Therefore, regardless of whether the driver DP is a specific individual or an ordinary person other than the specific individual, it is possible to accurately perform sensing of each face.

[Hardware configuration example]
FIG. 15 is a block diagram showing an example of the hardware configuration of the in-vehicle system A including the driver monitoring device 30 according to the embodiment.

The in-vehicle system A includes a driver monitoring device 30, one or more ECUs 40, and one or more sensors 41 for monitoring the state of the driver DP of the vehicle EC, and these are connected via a communication bus 43. Further, one or more actuators 42 are connected to the ECU 40.

The driver monitoring device 30 includes a camera 31, an image processing unit 32 that processes an image input from the camera 31, and a communication unit 36 for exchanging data and signals with an external ECU 40 and the like. There is.

The camera 31 is a device that captures an image including the face of the driver DP seated in the driver's seat. For example, a lens unit, an image sensor unit, a light irradiation unit, an interface unit, a camera control unit that controls each of these units, and the like. Can be configured to include. The image sensor unit may include an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), a filter, a microlens, or the like. The image pickup device unit may be an element capable of forming an image pickup image by receiving light in a visible region, or an element capable of forming an image pickup image by receiving light in a near infrared region. The light irradiation unit is configured to include a light emitting element such as an LED (Light Emitting Diode), and may include a near infrared LED or the like so that the driver's face can be imaged day or night. The camera 31 captures an image at a predetermined frame rate (for example, several tens of frames per second), and the data of the captured image is input to the image processing unit 32. The camera 31 may be an external type as well as an integrated type.

The image processing unit 32 is configured as an image processing device including one or more CPUs (Central Processing Units) 33, a ROM (Read Only Memory) 34, and a RAM (Random Access Memory) 35. The ROM 34 includes a program storage unit 341 and a facial feature amount storage unit 342, and the RAM 35 includes an image memory 351 that stores an input image from the camera 31. The driver monitoring device 30 may be provided with another storage unit, and the storage unit may be used as the program storage unit 341, the facial feature amount storage unit 342, and the image memory 351. The other storage unit may be a semiconductor memory or a storage medium that can be read by a disk drive or the like.

The CPU 33 is an example of a hardware processor, and by reading, interpreting, and executing data such as a program stored in the program storage unit 341 of the ROM 34 and the face feature amount stored in the face feature amount storage unit 342. , Processing of the image input from the camera 31, for example, face image processing such as face detection processing is performed. Further, the CPU 33 performs a process of outputting the result (for example, processing data, determination signal, control signal, etc.) obtained by the face image processing to the ECU 40 or the like via the communication unit 36.

The face feature amount storage unit 342 contains the face feature amount 342a of a specific individual shown in FIG. 16 as a learned face feature amount that has been learned to perform face sensing processing (face detection, face orientation estimation, etc.) from the image. And the normal facial feature amount 342b are stored.
Various features can be used as the learned facial features. For example, a feature amount (Haar-like feature amount) focusing on the difference in brightness (difference in average brightness between two rectangular areas of various sizes) in a local area of the face may be used. Alternatively, a feature amount (LBP (Local Binary Pattern) feature amount) focusing on the combination of the brightness distributions in the local area of the face may be used, or the distribution of the brightness in the local area of the face in the gradient direction may be used. A feature quantity focusing on a combination (HOG (Histogram of Oriented Gradients) feature quantity) or the like may be used.

Various machine learning methods can be used as a method for extracting features effective for face detection. Machine learning is a process of finding a pattern inherent in data (learning data) by a computer. For example, AdaBoost may be used as an example of a statistical learning method. AdaBoost selects a large number of discriminators (weak discriminators) with low discriminating ability, selects a weak discriminator with a small error rate from these many weak discriminators, adjusts parameters such as weights, and has a hierarchical structure. It is a learning algorithm that can construct a strong discriminator by setting. The discriminator may be referred to as a discriminator, a classifier, or a learner.

For example, one feature amount effective for face detection is discriminated by one weak discriminator, a large number of weak discriminators and their combinations are selected by AdaBoost, and these are used for strong discrimination having a hierarchical structure. You may build a vessel. Note that one weak discriminator may output information such as 1 for a face and 0 for a non-face. Further, as the learning method, a learning method called Real AdaBoost, which can output a real number from 0 to 1 instead of 0 or 1, may be used. Further, as these learning methods, a neural network having an input layer, an intermediate layer, and an output layer may be used.

A large number of face images captured under various conditions and a large number of non-face images (non-face images) are given as training data to a learning device equipped with such a learning algorithm, learning is repeated, weighting, etc. By adjusting the parameters and optimizing it, it is possible to construct a strong discriminator having a hierarchical structure capable of detecting a face with high accuracy. Then, one or more feature amounts used in the weak discriminators of each layer constituting the strong discriminator can be used as the learned facial feature amounts.

The face feature amount 342a of a specific individual is, for example, individually imaged at a predetermined place in advance under various conditions (conditions such as various face orientations, line-of-sight directions, or eye open / closed states). Then, these a large number of captured images are input to the learning device as teacher data, and are parameters that indicate the facial features of a specific individual adjusted by the learning process. The facial feature amount 342a of the specific individual may be, for example, a combination pattern of the difference in brightness of the local region of the face obtained by the learning process. The facial feature amount 342a of a specific individual stored in the facial feature amount storage unit 342 may be only the facial feature amount of one specific individual, or when a plurality of specific individuals drive a vehicle EC, a plurality of specific individuals may drive the vehicle EC. The facial features of a specific individual may be stored.

The normal facial feature amount 342b is the learning apparatus described above using images of a normal human face image captured under various conditions (conditions such as various face orientations, line-of-sight directions, or eye open / closed states) as teacher data. It is a parameter indicating the characteristics of a normal human face, which is input to and adjusted by the learning process. The normal facial feature amount 342b may be, for example, a combination pattern of light and dark differences in a local region of the face obtained by a learning process.

The learned facial feature amount stored in the facial feature amount storage unit 342 is fetched from a server on the cloud or the like via a communication network such as the Internet or a mobile phone network and stored in the facial feature amount storage unit 342. It may be configured as such.

The ECU 40 is composed of a computer device including one or more processors, a memory, a communication module, and the like. Then, the processor mounted on the ECU 40 reads, interprets, and executes the program stored in the memory, so that predetermined control for the actuator 42 and the like is executed.

The ECU 40 includes, for example, at least one of a traveling system ECU, a driving support system ECU, a body system ECU, and an information system ECU.

The traveling system ECU includes, for example, a drive system ECU, a chassis system ECU, and the like. The drive system ECU includes a control unit related to a "running" function such as engine control, motor control, fuel cell control, EV (Electric Vehicle) control, or transmission control. The chassis-based ECU includes a control unit related to a "stop, turn" function such as brake control or steering control.

The driving support system ECU has, for example, an automatic braking support function, a lane keeping support function (also referred to as LKA / Lane Keep Assist), a constant speed driving / inter-vehicle distance support function (also referred to as ACC / Adaptive Cruise Control), and a forward collision warning function. , Lane departure warning function, blind spot monitoring function, traffic sign recognition function, etc., functions that automatically improve safety or realize comfortable driving by linking with driving ECUs (driving support function or automatic driving function) It may be configured to include at least one control unit for.

The driving support system ECU includes, for example, Level 1 (driver assistance), Level 2 (partially automatic driving), and Level 3 (conditional automatic driving) at the automatic driving level presented by the American Society of Automotive Engineers of Japan (SAE). ) May be equipped with at least one of the functions. Further, the functions of level 4 (highly automatic driving) or level 5 (fully automatic driving) of the automatic driving level may be equipped, and only the functions of level 1 and 2 or only level 2 and 3 are equipped. May be good. Further, the in-vehicle system A may be configured as an automatic driving system.

The body system ECU may be configured to include at least one control unit related to the function of the vehicle body such as a door lock, a smart key, a power window, an air conditioner, a light, an instrument panel, or a winker.

The information system ECU may be configured to include, for example, an infotainment device, a telematics device, or an ITS (Intelligent Transport Systems) related device. The infotainment device may include, for example, an HMI (Human Machine Interface) device that functions as a user interface, a car navigation device, an audio device, and the like. The telematics device may include a communication unit or the like for communicating with the outside. The ITS-related device may include an ETC (Electronic Toll Collection System), a communication unit for performing road-to-vehicle communication with a roadside unit such as an ITS spot, or vehicle-to-vehicle communication.

The sensor 41 may include various in-vehicle sensors that acquire sensing data necessary for controlling the operation of the actuator 42 by the ECU 40. For example, in addition to vehicle speed sensors, shift position sensors, accelerator opening sensors, brake pedal sensors, steering sensors, etc., peripheral monitoring of external imaging cameras, millimeter-wave radar (Radar), riders (LIDER), ultrasonic sensors, etc. A sensor or the like may be included.

The actuator 42 is a device that executes an operation related to traveling, steering, braking, etc. of the vehicle EC based on a control signal from the ECU 40, and includes, for example, an engine, a motor, a transmission, a hydraulic cylinder, an electric cylinder, and the like.

[Functional configuration example]
FIG. 16 is a block diagram showing a functional configuration example of the image processing unit 32 of the driver monitoring device 30 according to the embodiment.
The image processing unit 32 includes an image input unit 21, a face detection unit 22, a specific individual determination unit 25, a first face image processing unit 26, a second face image processing unit 26a, an output unit PO, and a face feature amount storage unit 342. It is configured to include.

The image input unit 21 performs a process of capturing an image including the face of the driver DP captured by the camera 31.

The face detection unit 22 is configured to include a face detection unit 23 of a specific individual and a normal face detection unit 24, and performs a process of detecting a face region while extracting a feature amount for detecting a face from an input image. Do.
The face detection unit 23 of the specific individual uses the face feature amount 342a of the specific individual read from the face feature amount storage unit 342 to perform a process of detecting the face region from the input image.
The normal face detection unit 24 uses the normal face feature amount 342b read from the face feature amount storage unit 342 to perform a process of detecting a face region from an input image.

The method of detecting the face area from the image is not particularly limited, but a method of detecting the face area at high speed and with high accuracy is adopted. The face detection unit 22 extracts features for detecting a face in each search area while scanning a predetermined search area (search window) on the input image, for example. The face detection unit 22 may extract, for example, the difference in brightness (brightness difference) of a local region of the face, the edge strength, or the relationship between these local regions as a feature amount. Then, the face detection unit 22 uses the feature amount extracted from the search area, the normal face feature amount 342b read from the face feature amount storage unit 342, or the face feature amount 342a of a specific individual, and has a hierarchical structure ( A detector (hierarchical structure that captures the details of the face from the hierarchy that roughly captures the face) determines whether the face is face or non-face, and performs processing to detect the face area from the image.

The specific individual determination unit 25 uses the feature amount of the face area detected by the face detection unit 22 and the face feature amount 342a of the specific individual read from the face feature amount storage unit 342 to detect the face in the face area. Performs a process of determining whether or not is the face of a specific individual.

The specific individual determination unit 25 calculates an index showing the relationship between the feature amount extracted from the face area and the face feature amount 342a of the specific individual, for example, a correlation coefficient, and based on the calculated correlation coefficient, the face area. It may be determined whether or not the face of is the face of a specific individual. For example, the correlation of feature quantities such as Haar-like features of one or more local regions in the face region may be obtained. Then, when the correlation coefficient is larger than a predetermined threshold value, it is determined that the face in the detected face area is the face of a specific individual, and when the correlation coefficient is equal to or less than the predetermined threshold value, the face in the detected face area is a specific individual. It may be determined that it is not the face of.

Further, the specific individual determination unit 25 may determine whether or not the face in the detected face region is the face of a specific individual based on the result of determination for one frame of the input image from the camera 31. Based on the result of determination for a plurality of frames of the input image from the camera 31, it may be determined whether or not the face in the detected face region is the face of a specific individual.

When the specific individual determination unit 25 determines that the face is a specific individual's face, the first face image processing unit 26 performs face image processing for the specific individual using the face feature amount 342a of the specific individual. The first face image processing unit 26 includes a face orientation estimation unit 27 of a specific individual, an eye opening / closing detection unit 28 of the specific individual, and a line-of-sight direction estimation unit 29 of the specific individual, but is still different. It may include a configuration for estimating or detecting facial behavior.

The face orientation estimation unit 27 of the specific individual performs a process of estimating the face orientation of the specific individual. The face orientation estimation unit 27 of the specific individual uses, for example, the face feature amount 342a of the specific individual to position the facial organs such as eyes, nose, mouth, and eyebrows from the face area detected by the face detection unit 23 of the specific individual. And shape are detected, and the orientation of the face is estimated based on the position and shape of the detected facial organs.

The method for detecting the facial organs from the facial region in the image is not particularly limited, but it is preferable to adopt a method capable of detecting the facial organs at high speed and with high accuracy. For example, a method of creating a three-dimensional face shape model, fitting it to a face region on a two-dimensional image, and detecting the position and shape of each organ of the face can be adopted. As a technique for fitting a three-dimensional face shape model to a human face in an image, for example, the technique described in Japanese Patent Application Laid-Open No. 2007-249280 can be applied, but the technique is not limited thereto.

Further, the face orientation estimation unit 27 of the specific individual can use the estimation data of the face orientation of the specific individual, for example, the pitch angle of vertical rotation (around the X axis) included in the parameters of the three-dimensional face shape model. The yaw angle for left-right rotation (around the Y-axis) and the roll angle for the entire rotation (around the Z-axis) may be output.

The eye opening / closing detection unit 28 of the specific individual performs a process of detecting the opening / closing state of the eyes of the specific individual. The eye opening / closing detection unit 28 of the specific individual, for example, is based on the position and shape of the facial organs obtained by the face orientation estimation unit 27 of the specific individual, particularly the position and shape of the feature points (eyelids, pupils) of the eyes. Detects the open / closed state, for example, whether the eyes are open or closed. For the open / closed state of the eye, for example, the feature amount of the image of the eye (the position of the eyelid, the shape of the pupil (black eye), the area size of the white eye part and the black eye part, etc.) in various open / closed states of the eye is previously learned. It may be detected by learning using and evaluating the degree of similarity with these learned feature amount data.

The line-of-sight direction estimation unit 29 of the specific individual performs a process of estimating the line-of-sight direction of the specific individual. The line-of-sight direction estimation unit 29 of a specific individual is based on, for example, the orientation of the face of the driver DP and the position and shape of the facial organs of the driver DP, particularly the position and shape of the feature points of the eyes (outer corners of eyes, inner corners of eyes, pupils). Estimate the direction of the line of sight. The direction of the line of sight is the direction that the driver DP is looking at, and is determined by, for example, a combination of the direction of the face and the direction of the eyes.

In addition, the direction of the line of sight is, for example, the feature amount of the image of the eye in various combinations of face orientation and eye orientation (relative position of outer corner, inner corner of eye, pupil, relative position of white eye portion and black eye portion, shading, etc. (Texture, etc.) may be detected by learning in advance using a learning device and evaluating the degree of similarity with the learned feature amount data. In addition, the line-of-sight direction estimation unit 29 of the specific individual estimates the size and center position of the eyeball from the size and orientation of the face, the position of the eyes, etc., using the fitting result of the three-dimensional face shape model, and the pupil. The position of the eyeball may be detected, and the vector connecting the center of the eyeball and the center of the pupil may be detected as the line-of-sight direction.

When the specific individual determination unit 25 determines that the face is not the face of a specific individual, the second face image processing unit 26a performs normal face image processing using the normal face feature amount 342b. The second face image processing unit 26a includes a normal face orientation estimation unit 27a, a normal eye opening / closing detection unit 28a, and a normal line-of-sight direction estimation unit 29a. The processing performed by the normal face orientation estimation unit 27a, the normal eye opening / closing detection unit 28a, and the normal line-of-sight direction estimation unit 29a uses the normal face feature amount 342b, etc., but the face orientation of a specific individual. Since it is basically the same as the estimation unit 27, the eye opening / closing detection unit 28 of the specific individual, and the line-of-sight direction estimation unit 29 of the specific individual, the description thereof will be omitted here.

The output unit PO performs a process of outputting information based on the image processing by the image processing unit 32 to the ECU 40 or the like. The information based on the image processing may be, for example, information on the behavior of the face such as the direction of the face of the driver DP, the direction of the line of sight, or the open / closed state of the eyes, or the driver DP determined based on the detection result of the behavior of the face. It may be information about the state of (for example, forward gaze, inattentiveness, dozing, backward facing, prone, etc.). Further, the information based on the image processing may be a predetermined control signal (control signal for performing caution or warning processing, control signal for performing operation control of the vehicle EC, etc.) based on the state determination of the driver DP.

Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in all respects. Needless to say, various improvements and changes can be made without departing from the scope of the present invention.
In the above embodiment, the case where the image processing device according to the present invention is applied to the driver monitoring device 30 has been described, but the application example is not limited to this. For example, in a device or system for monitoring a person who operates, monitors, or performs a predetermined work on various facilities such as machines and devices in a factory, the above-mentioned specific individual is included in the monitoring target person. In some cases, the image processing apparatus according to the present invention can be applied.

1 Captured image data storage unit 1a 2D image data storage unit 1b 3D distance image data storage unit 2 Display unit 3 Operation unit 4 CPU
5 ROM
6 RAM
6a Image storage unit 6b Common feature point storage unit 7 Image processing calculation unit 7a Feature point matching unit 7b Normal vector calculation unit 7c Reference face orientation angle image definition unit 7d Relative face orientation angle calculation unit 7e Face sensing parameter setting unit 7f Face sensing Parameter verification unit

21 Image input unit 22 Face detection unit 23 Face detection unit of a specific individual 24 Normal face detection unit 25 Specific individual judgment unit 26 First face image processing unit 26a Second face image processing unit 27 Face orientation estimation unit 28 of a specific individual Individual eye opening / closing detection unit 29 Specific individual line-of-sight direction estimation unit 27a Normal face orientation estimation unit 28a Normal eye opening / closing detection unit 29a Normal line-of-sight direction estimation unit 30 Driver monitoring device 31 Camera 32 Image processing unit 33 CPU
34 ROM
35 RAM
36 Communication Department

40 ECU
41 Sensor 42 Actuator 43 Communication bus

B Blackout C Cameras capable of acquiring distance images (ToF cameras, etc.)
D Dealer staff M Mark as instruction department P Laptop S Tripod U User d Distance between user and camera

e1-e4 Feature points (eyes)
e5 Feature points (ears)
m1, m2 feature points (mouth)
n normal vector

A In-vehicle system EC vehicle DP driver PO output unit

Claims (9)

An imaging unit that is installed in front of the imaging target and can acquire distance images,
An image data storage unit that stores the distance image and
An image processing calculation unit that processes an image input from the image data storage unit,
Instruction department and
It is a personal data acquisition system configured to include
The image processing calculation unit has a feature point matching unit, a normal vector calculation unit, a reference face orientation angle image definition unit, and a relative face orientation angle calculation unit.
A personal data acquisition system characterized in that the instruction unit points to a plurality of visual recognition points. In addition, the display and
Operation unit and
The personal data acquisition system according to claim 1, wherein the personal data acquisition system is provided. The first or second aspect of the invention, wherein the imaging unit is configured to be capable of acquiring the distance image and a two-dimensional image such as RGB or infrared light having the same viewpoint as the distance image. Personal data acquisition system. The personal data acquisition system according to any one of claims 1 to 3, wherein the image processing calculation unit is further configured to include a face sensing parameter setting unit and a face sensing parameter verification unit. The personal data acquisition system according to any one of claims 1 to 4, further comprising a background curtain. A personal data acquisition method using the personal data acquisition system according to any one of claims 1 to 5.
An imaging unit capable of acquiring the distance image is placed in front of the user.
A personal data acquisition method characterized in that an instruction based on the instruction unit is given to a user, and image data whose face orientation is changed in a predetermined order and at a predetermined time interval based on the instruction unit is acquired. A personal data acquisition method using the personal data acquisition system according to any one of claims 1 to 5.
A step of defining 3D image data in which the depth of facial feature points such as eyes and mouth is almost constant (within a predetermined threshold) as an image with a reference face orientation angle (pitch angle 0 degrees, yaw angle 0 degrees), and
Steps to select common facial feature points in image I and image J in the image series,
Steps for matching facial feature points in image I and image J,
The step of calculating the normal vector of the face-oriented plane consisting of the face feature points of image I,
Steps to calculate the normal vector of the face-oriented plane consisting of the face feature points of image J,
The step of calculating the relative angle of the face orientation of the image I with respect to the image J (calculating the relative normal vector of the face orientation plane), and
A method of acquiring personal data, which is characterized by including. A method for adjusting face sensing parameters for an image processing device, which is performed using personal data acquired by using the personal data acquisition system according to any one of claims 1 to 5.
Steps to input 2D image data group and
Steps to learn and set face sensing parameters based on the input 2D image data group,
Steps to capture the set face sensing parameters and
Steps to perform face sensing processing trials (face detection, face orientation estimation, etc.) and
Steps to capture and output face sensing result groups for 2D image data group,
A step to capture and output a true value group such as a face detection rectangle and a face orientation angle corresponding to a two-dimensional image data group, and
A step of verifying the face sensing parameter using the face sensing result group and the true value group, and
Compare whether the absolute value of the difference between the face detection rectangle result and the true value and the absolute value of the difference between the face orientation angle result and the true value are smaller than the predetermined threshold value.
If it is judged to be small, the process ends, and if it is judged not to be small, the step to return to the first step and
A method for adjusting face sensing parameters for an image processing device, which comprises. A computer program for causing at least one or more computers to perform adjustment processing of face sensing parameters for an image processing device.
To at least one of the above computers
Steps to capture 2D image data and
Steps to learn and set face sensing parameters based on the captured 2D image data group,
Steps to capture the set face sensing parameters and
Steps to import and output 2D image data groups for learning and setting face sensing parameters,
Steps to perform face sensing processing trials, face detection, face orientation estimation, etc.
Steps to capture and output face sensing result groups for 2D image data group,
A step to capture and output a true value group such as a face detection rectangle and a face orientation angle corresponding to a two-dimensional image data group, and
A step of verifying the face sensing parameter using the face sensing result group and the true value group, and
It is compared whether the absolute value of the difference between the face detection rectangle result and the true value is smaller than the predetermined threshold value, and whether the absolute value of the difference between the face orientation angle result and the true value is smaller than the predetermined threshold value.
If it is determined that it is small, the process is completed, and if it is determined that it is not small, the step of returning to the first step is performed.
A computer program characterized by running a computer program.

Images