JP2008186247A - Face direction detector and face direction detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face direction detector allowing achievement of sufficient detection accuracy by reducing influence upon the detection accuracy by an individual difference when detecting a face direction from two-dimensional characteristic point coordinates even if the face direction detection device is a small device such as a portable device. <P>SOLUTION: A face direction decision device 100 has: an image input part 101 for entering a face of a user as an image; a characteristic point extraction part 102 for extracting a characteristic point of the face of the user from the image; a face direction decision part 103 for deciding the face direction of the user based on a prescribed threshold from the characteristic point; an individual difference adjustment part 104 for acquiring individual difference information for adjusting the individual difference about each the face direction of each the user; and an individual difference information storage part 105 updating the threshold by use of the individual difference information and registering it in each the user. By updating the threshold, the face direction detection wherein the individual difference of the user is considered can be performed. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a face orientation detection device and a face orientation detection method, and more particularly, to a non-contact interface that operates a pointer by detecting feature points of a user's face from an input image and determining the face orientation. The present invention relates to a face orientation detection device and a face orientation detection method.

  In recent years, non-contact interface technology has become common, and for example, many methods for operating the apparatus by detecting the direction of the user's face and the direction of the line of sight from an image input from a camera or the like have been proposed. . The method of detecting the face orientation can be broadly divided into a method of matching the input image using a pattern of the face image in each direction as a template, and a method of detecting facial feature points of face parts (eyes, mouth, nose, etc.). There is a method to detect the face direction geometrically from the above.

  The former method is unsuitable for use in pointer operations that require real-time performance because it requires a great deal of data processing and calculation processing in order to improve accuracy. It cannot be applied to devices with poor performance.

  The latter method is relatively easy to increase accuracy. In particular, a method has been proposed in which a highly accurate face orientation can be obtained by using a factorization method that handles a three-dimensional shape of an object with relatively simple calculations (see, for example, Patent Document 1). However, the method using the factorization method cannot be obtained with sufficient accuracy and performance in consideration of the resolution and image quality of the input image and the processing performance of the terminal to be applied to the current portable terminal such as a cellular phone. Therefore, in order to realize a non-contact interface using face orientation detection in such a device, only the coordinate calculation of some two-dimensional face feature points is used, or two-dimensional feature point coordinates and There has been proposed a method of determining the face orientation in combination with face area extraction (see, for example, Patent Documents 2 to 4).

Japanese Patent No. 3822483 JP-A-4-281578 JP-A-10-274516 JP-A-11-281661 Japanese Patent No. 2570966

  However, when the face orientation is detected from the two-dimensional feature point coordinates, there is a problem that it is difficult to obtain sufficient detection accuracy because the influence on the detection accuracy due to individual differences is large. Therefore, for example, Patent Document 5 proposes providing means for measuring and correcting individual differences of users, that is, physical characteristics in advance.

  However, in Patent Document 5, there is only a description suggesting that the individual difference acquisition means is mechanical acquisition, and no realization means has been presented. Moreover, acquiring the difference in physical characteristics by mechanical means is complicated for the user and is not easy to use. Furthermore, small devices such as portable devices have a problem that it is difficult to have mechanical measuring means from the viewpoint of manufacturing cost of the apparatus and from the viewpoint of volume and weight.

  The present invention has been made in view of the above-described problems of the background art, and an object of the present invention is to detect a face orientation from two-dimensional feature point coordinates even in a small device such as a portable device. Another object of the present invention is to provide a new and improved face orientation detection apparatus and face orientation detection method capable of reducing the influence of individual differences on detection accuracy and obtaining sufficient detection accuracy.

  In order to solve the above problems, according to a first aspect of the present invention, there is provided a face orientation determination device for determining a face orientation of a user of an information processing device and using it in an application of the information processing device. The face orientation determination device (100) of the present invention includes an image input unit (101) that inputs a user's face as an image, and a feature point extraction unit (102) that extracts a feature point of the user's face from the image. And a face direction determination unit (103) that determines the face direction of the user from the feature points based on a predetermined threshold and transmits information to the application, and adjusts individual differences for each face direction for each user. An individual difference adjustment unit (104) for acquiring individual difference information of the user, and an individual difference information storage unit (105) for updating the threshold value using the individual difference information and registering it for each user. It is characterized (claim 1).

  Furthermore, a display unit (107) for displaying various information, and a pointer control unit (106) for controlling a pointer displayed on the display unit according to the face orientation determined by the face orientation determination unit. It can comprise, and can comprise (claim 2).

  According to such a configuration, the user is made to perform a face-to-face operation in the front and in each direction, that is, a detection operation of a non-contact interface that detects the face direction and operates a pointer or the like, and features at that time For each detection operation, a detection threshold value that is optimal for the user is calculated and a control unit that updates the threshold value is provided, so that face orientation detection that takes into account individual differences among users can be performed. Therefore, it can be expected that the user can easily improve the accuracy of face orientation detection without increasing the manufacturing cost of the apparatus. In addition to the pointer, it can also be used to scroll the screen of the display device according to the face orientation determined by the face orientation determination unit, and to move a character (for example, the game machine) displayed on the display unit. Can do.

  In the above description, the reference numerals in parentheses attached to the constituent elements are merely examples of corresponding constituent elements in the following embodiments and drawings for the convenience of explanation, and the present invention is not limited thereto. It is not limited to. The same applies to the following.

  Various applications are possible in the face orientation determination device of the present invention, and some examples are as follows.

  The individual difference adjusting unit (104) issues an instruction to the user to perform a single operation for each face, and the individual difference information is obtained from each face-oriented image input from the image input unit. (Claim 4).

  The individual difference adjusting unit (104) instructs the display unit (107) that displays various types of information in the form of a graphic or an image when issuing an instruction to the user to perform each face-facing operation. You may make it take out (Claim 5).

  The individual difference adjustment unit (104) issues a warning (warning sound, display of a warning screen, etc.) to the user when the movement of each face of the user is inappropriate (too much). (Claim 6).

  In order to solve the above problems, according to a second aspect of the present invention, there is provided a face orientation determination method for determining the face orientation of a user. The face orientation determination method of the present invention includes an image input step of inputting a user's face as an image, a feature point extraction step of extracting a feature point of the user's face from the image, and a predetermined threshold based on the predetermined threshold value. A face orientation determining step for determining the face orientation of the user from the feature points, an individual difference adjusting step for obtaining individual difference information for adjusting individual differences for each face orientation for each user, and the threshold value for the individual difference And an individual difference information storing step of updating for each user and registering for each user (claim 7).

  According to such a method, the user is allowed to perform a face detection operation in front and in each direction, that is, a detection operation of a non-contact interface that detects the face direction and operates a pointer or the like, and features at that time For each detection operation, a detection threshold value that is optimal for the user is calculated and a control unit that updates the threshold value is provided, so that face orientation detection that takes into account individual differences among users can be performed. Therefore, it can be expected that the user can easily improve the accuracy of face orientation detection without increasing the manufacturing cost of the apparatus.

  Various applications are possible in the face orientation determination method of the present invention, and some examples thereof are as follows.

  The individual difference adjustment step includes a step of instructing the user to perform a single operation for each face, and the individual difference information from each face-oriented image input from the image input unit. And a step of acquiring (claim 8).

  In the step of giving an instruction for causing the user to perform each face-facing operation as a whole, an instruction may be given as a graphic or an image on a display unit for displaying various information.

  In the individual difference adjustment step, if the movement of each face of the user is inappropriate (too much), a warning (warning sound, warning screen display, etc.) is issued to the user. (Claim 10).

  According to another aspect of the present invention, a program for causing a computer to function as the face orientation detecting device according to the first aspect of the present invention, and a computer-readable recording medium on which the program is recorded Is provided. Here, the program may be described in any programming language. As the recording medium, for example, a CD-ROM, a DVD-ROM, a flexible disk, a flash memory, a built-in memory of a portable terminal device, etc., a recording medium that is currently used as a recording medium capable of recording a program, or a future Any recording medium used can be employed.

  As described above, according to the present invention, even in a small device such as a portable device, when detecting a face orientation from two-dimensional feature point coordinates, the influence on the detection accuracy due to individual differences is reduced, and sufficient It is possible to obtain a high detection accuracy. Other excellent effects of the present invention will be described in the following description of the best mode for carrying out the invention.

  Exemplary embodiments of a face direction detection apparatus and a face direction detection method according to the present invention will be described below in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is an explanatory diagram showing the configuration of an embodiment of the present invention. In the present embodiment, an example will be described in which the face orientation is determined in five directions of front, up, down, left, and right, and the pointer is moved in that direction (does not move in the front).

  As shown in FIG. 1, the face direction detection apparatus 100 according to the present embodiment includes an image input unit 101, a feature point extraction unit 102, a face direction determination unit 103, an individual difference adjustment unit 104, and individual difference information. A storage unit 105, a pointer control unit 106, and a display unit 107 are included. Note that description and illustration of processing units and connection relationships for realizing various functions that are not directly related to the description of the present embodiment will be omitted. Hereinafter, each component will be described in detail.

  The image input unit 101 is a processing unit that acquires an image including a user's face in real time as a plurality of continuous images, and is, for example, a camera device. The acquired image is sent to the feature point extraction unit 102.

  The feature point extraction unit 102 specifies the position of a face area and a face component (for example, eyes, mouth, nose, eyebrows, etc.) for each image from the image input from the image input unit 101, and the position of each feature point. Is a processing unit that extracts as two-dimensional coordinates. The extracted feature point coordinates are sent to the face orientation determination unit 103 and the individual difference adjustment unit 104.

  The face orientation determination unit 103 determines the face orientation as one of front, top, bottom, left, and right based on the obtained feature point coordinates and a predetermined face orientation judgment formula, and the result is face orientation information. As a processing unit that outputs as Face orientation information is sent to the pointer control unit 106.

  In order to extract the physical characteristics of the user, the individual difference adjustment unit 104 causes the user to measure the individual difference information interactively using the display unit 107 and stores the extracted individual difference information in the personal difference information. This is a processing unit registered in the unit 105.

  The individual difference information storage unit 105 is a processing unit that corrects the individual difference to the face orientation determination formula of the face orientation determination unit 103 after the user once registers the individual difference information.

  The pointer control unit 106 is a control unit that moves the pointer based on the result of the face orientation determination unit 103.

  The display unit 107 displays instruction information for obtaining individual difference information to the user in accordance with the display of the movement result of the pointer obtained from the pointer control unit 106 and the individual difference adjustment unit 104. In addition, information necessary for other functions of the device is displayed.

  The present embodiment is configured as described above. The operation of this embodiment will be described below.

(Operation of this embodiment)
First, the operation of the pointer operation will be described.

  When the user holds his / her face in front of the camera, an image including the user's face is sent from the image input unit 101 to the feature point extraction unit 102 in real time. The feature point extraction unit 102 extracts a face area and a face part from the input image, and outputs them as feature point coordinates of each face part. For example, the technique disclosed in Japanese Patent Application Laid-Open No. 2003-281539 “Face Parts Search Device and Face Parts Search Method” can be used for this face part search method.

  FIG. 2 is an explanatory diagram illustrating an example of extracted feature points. In the example shown in FIG. 2, the left eye (LE) 201, right eye (RE) 202, nose (UM) 203, right mouth end (RM) 204, lower mouth (DM) 205, left mouth end (LM) 206, left eyebrow ( (LEB) 207 and right eyebrow (REB) 208 are shown as examples of feature points. However, it is not necessary to extract all of these feature points, and arbitrary features can be extracted. Moreover, you may set the other location of a face as a feature point. In this embodiment, “two eyebrows, two eyes, and four mouths” (reference numerals 201 to 208) indicated by “x” in FIG. 2 are extracted as feature points.

  The feature point coordinates extracted by the feature point extraction unit 102 are sent to the face direction determination unit 103, and the direction of the face direction is determined by a predetermined determination formula.

  An example of a specific method for determining the face orientation is shown below. Description will be made in the order of determination in the vertical direction, determination in the horizontal direction, and determination in the front direction.

[Judging the vertical direction]
First, the vertical direction determination method will be described with reference to FIGS. 3 and 4.
First, an angle (∠LE) 304 and an angle (∠RE) 305 in the triangle on both eyes-mouth shown in FIG. 3 are calculated (step S401). These can be easily calculated from the three feature point coordinates of the left eye (LE) 301, the right eye (RE) 302, and the mouth (UM) 303.

  Next, the average of the calculated angle (∠LE) 304 and angle (∠RE) 305 (hereinafter referred to as the “both eye-mouth angle”) is compared with a predetermined upward threshold value and a downward threshold value (step S402, step S402). S403). Although there are individual differences, due to the characteristics of the human face, it is used that the angle decreases when facing upward and increases when facing downward, and if it is equal to or greater than the upward threshold, it is upward (step S404), downward If it is less than or equal to the threshold value, it is determined to face downward (step S405).

  Here, generally, the frontal state is registered in advance, and both eyes-upper angle (average of ∠LE and ∠RE) in the frontal state and both eyes-upper angle (検 出 LE, ∠RE of the detected face image). By calculating an average difference and comparing a difference threshold value with a predetermined front face to determine the vertical direction, a decrease in detection accuracy due to a difference in user physical characteristics can be reduced to some extent.

[Judgment in the horizontal direction]
Next, a determination method in the left-right direction will be described with reference to FIGS.
First, as shown in FIG. 5A, the X-coordinate distance: distance (REB.x-CE.x) 511 between the center of both eyes (CE) 509 and the left inner eyebrow and the center of the eye is calculated (step S601). .

Next, as shown in FIGS. 5B and 5C, from the distance (CE.x-LEB.x) and the distance (REB.x-CE.x), An eye center division ratio is calculated (step S602). These distances and ratios can be easily calculated from the respective feature point coordinates.
Right eyebrow-eye center ratio = distance (REB.x-CE.x) ÷ distance (CE.x-LEB.x)%
Left eyebrow-eye center ratio = distance (CE.x-LEB.x) ÷ distance (REB.x-CE.x)%

  From this calculation result, the leftward determination is made by comparing the left eyebrow-eye center ratio with a leftward determination threshold value (for example, 160%) (step S603), and if it exceeds the threshold value, it is considered leftward (step S605).

  Similarly, in the right direction determination, the right eyebrow-eye center ratio is compared with the right direction determination threshold value (for example, 160%) (step S604), and if it exceeds the threshold value, it is considered to be right direction (step S606).

[Determination of the front direction]
Next, a method for determining the front direction will be described.
The determination of the front direction is regarded as the front if it is not a determination in each of the up / down / left / right directions described above as a comprehensive determination.

  The face orientation information determined in this way is sent to the pointer control unit 106, and the pointer control unit 106 moves the pointer and cursor up, down, left and right according to the face orientation information. In the case of pointer movement, for example, the input image is moved in the direction indicated by the face orientation information. In the case of a cursor, when the same face orientation information continues for the number of images, the cursor is moved in the direction indicated by the face orientation information in units of icons or menus on the display unit.

  The above is the description of the pointer operation.

  Next, the individual difference adjustment operation will be described with reference to FIGS. 7 to 12, FIG. 13A, and FIG. 13B. Although the adjustment method in the vertical direction is described here, the operation is the same in the horizontal direction.

  While displaying the image input from the image input unit 101 on the display unit 107, the individual difference adjustment unit 104 instructs the display unit 107 to shoot the front state (step S1001). Specifically, as in the example shown in FIG. 7A, the screen display “Please turn to the front” is displayed. At this time, similarly to the up / down determination of the face orientation determination, the binocular-mouth angle in the front state (average value of ∠LE, ∠RE): ∠LE-RE_Neutral is calculated and held (step S1002).

  Next, the individual difference adjustment unit 104 instructs the display unit 107 to face up to the user (step S1003). Specifically, as in the example shown in FIG. 7B, the screen displays “Please turn up”. At this time, the user faces upward from the front state, but the input image is continuously processed here, and the user faces too much upward, so that the feature point extraction unit 102 cannot detect the feature point coordinates of the facial part. (Step S1004), the both eyes-mouth angle (average value of ∠LE and ∠RE) at the time when the user stops moving the face for a certain time (step S1006): and LE-RE_Up are held (step S1005) .

  Furthermore, the individual difference adjustment unit 104 instructs the user to face down (step S1007). Specifically, as in the example shown in FIG. 7C, the message “Please turn down” is displayed on the screen. As a result, the user faces down, but the input image is continuously processed in the same manner as in the up direction, and whether the user can face down too much to detect the feature point coordinates (step S1008). At the time when no change is made (step S1010), both eyes-mouth angle (average value of ∠LE and ∠RE): ∠LE-RE_Down is held (step S1009).

  When viewed from the user, these series of operations are merely turning the face up and down from the front toward the camera (image input unit 101) in accordance with an instruction of the device (face orientation determination device 100).

Next, the operation for correcting the individual difference of the threshold will be described.
The threshold in the vertical direction in the face orientation determination unit 103 is stored in advance as a threshold obtained by evaluating many images including the device and the face in the design stage. This threshold value is generally an intermediate value between the front direction and the upward direction (or downward direction) (reference numeral 801 in FIG. 8).

  On the other hand, the individual difference adjustment unit 104 has a front-eye binocular-upper angle obtained by the above procedure: ∠LB-RE_Neutral (reference numeral 803 in FIG. 8), and a binocular-upper-eye angle in the upward state: ∠LE-. The intermediate value is calculated from RE-Up (reference numeral 802 in FIG. 8) and registered in the individual difference information storage unit 105 as a correction threshold value for upward determination (reference numeral 804 in FIG. 8).

The upward threshold correction threshold value is expressed as follows.
Upward determination correction threshold = (∠LE-RB_Neutral−∠LE-RE_Up) / 2

  The face orientation determination unit 103 subsequently performs upward determination using the registered threshold value. That is, the determination is made using a threshold value that is optimal for the registered user.

  Similarly for the downward direction, both eyes-upper angle in the front state: ∠LE-RE_Neutral (reference numeral 902 in FIG. 9), and both eyes-upper angle in the downward state: ∠LE-RE_Down (reference numeral 903 in FIG. 9) between them. A value is calculated and registered in the individual difference information storage unit 105 as a threshold value for downward determination (reference numeral 904 in FIG. 9). An unexplained reference numeral 901 is an initial threshold similar to the reference numeral 801 in FIG. The face orientation determination unit 103 subsequently performs downward determination using this threshold value.

  Here, the correction of the face downward threshold in FIG. 9 will be further described. In particular, as shown in FIG. 9, the reason why the binocular-mouth angle increases when the face of the user moves downward from the front will be described in detail with reference to FIGS.

  When the user is facing the front, when looking at the human face from the side, the position of the center of the eyes (black eyes) and the point on the lips It protrudes forward (front direction when viewed from a human). Accordingly, when facing downward from the front, as shown in FIG. 10, the triangle on both eyes-mouth changes in a longitudinal direction. That is, in FIG. 10, angle A <angle B.

  In addition, since the control of the pointer assumed in the present embodiment is an operation while looking at the screen of the apparatus, a human usually looks blacker when looking down than when looking at the front. Move so that it looks up. Since the point above the mouth does not move in the upward direction, as shown in FIG. 11, the triangle on both eyes-mouth changes to extend in the vertical direction. That is, in FIG. 11, angle C <angle D.

  In addition, when it is assumed to be used in a mobile terminal device such as a mobile phone, when the device is held in the hand and the in-camera of the device is pointed at the user to view the screen, as shown in FIG. In most cases, the relative positional relationship between the in-camera and the face is below the front of the face. As a result, the front state is more inclined due to the relationship between the center of the eye and the mouth when viewed from the side shown in FIG. That is, as shown in FIGS. 10 and 12, the angle a <angle A, and the angle a <angle A <angle B.

  For the above reasons, when the user's face direction moves downward from the front, the result is that the binocular-mouth angle increases. However, since only the point on the mouth has individual differences and may not provide sufficient accuracy, other feature points (for example, feature points of the nose) may be used in combination.

  Although the individual difference adjustment in the vertical direction has been described above, the same applies to the individual difference adjustment in the horizontal direction. Optimum threshold for the user by holding the center of each eye-right eyebrow horizontal distance (and eye center-left eyebrow horizontal distance) with the user facing right and left and registering the intermediate value as a threshold The face orientation is determined with. Moreover, although this embodiment demonstrated the case where an individual difference adjustment of an up-down direction and a left-right direction was performed separately, you may carry out collectively up-down and left-right.

  The face orientation detection apparatus 100 according to the present embodiment has been described above. Such a face orientation detection apparatus 100 can cause a computer to function as the face orientation detection apparatus 100 by incorporating a computer program for realizing the above functions into the computer. Such a computer program can be distributed in the market in a form recorded on a predetermined recording medium (for example, a CD-ROM) or downloaded via an electronic network. The computer here naturally includes small devices such as mobile phones.

(Effect of this embodiment)
As described above, according to the present embodiment, the user is allowed to perform the entire face-to-face operation in the front and each direction, that is, the detection operation of the non-contact interface that detects the face direction and operates the pointer. Detecting the state of the feature point at that time, calculating the optimal detection threshold for the user for each detection operation, and providing a control unit that updates the threshold, the face orientation taking into account individual differences of the user Since the detection can be performed, it can be expected that the user can easily improve the accuracy of the face orientation detection without increasing the manufacturing cost of the apparatus.

  The preferred embodiments of the face direction detection apparatus and the face direction detection method according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious for those skilled in the art that various modifications or modifications can be conceived within the scope of the technical idea described in the claims, and these are naturally within the technical scope of the present invention. It is understood that it belongs.

  For example, in the above-described embodiment, the example in which the pointer operation is performed by the face direction determination based on the threshold value has been described. is there. In that case, the threshold value is not corrected, but a function (for example, a conversion formula or a conversion table) for converting the face angle into coordinates is corrected by the individual difference information.

  In the above embodiment, as an example of the determination method of face orientation, an example using the binocular-upper angle in the vertical direction and the left eyebrow (right eyebrow) -center ratio of both eyes in the left-right direction has been described. The present invention can be similarly applied by using other facial feature points such as a center-mouth end angle and a determination formula.

  In the above embodiment, the display of characters is used to instruct the user to acquire the feature points for individual difference adjustment. For example, as shown in FIG. 14, an arrow symbol, a figure imitating an arrow, or an image is used. May be. FIG. 14A is an instruction to face the front with a black triangle “▲”, and FIG. 14B is an instruction to face the top with a black triangle “▲”. (C) is an instruction to face down with a figure of a hand.

  Further, when the user's face-facing motion is inappropriate (too much), a warning may be issued to the user as shown in FIG. 15, for example. FIG. 15A issues a warning to the user with a warning sound, and FIG. 15B issues a warning to the user by displaying a warning screen (symbol).

  In the above embodiment, as the procedure for acquiring the individual difference information of the user, only the procedure for instructing the face direction and having the user face each direction has been described, but the present invention is not limited to this. For example, when the feature point detection limit is exceeded in the process of changing the direction of the face, the user is made to confirm the operational feeling after correcting the threshold value once due to individual differences. It is also possible to perform difference adjustment or to allow the user to change the correction threshold roughly. 16 and 17 are explanatory diagrams illustrating an example of the confirmation operation after such adjustment.

  FIG. 16 is an explanatory diagram showing a case where a point moves when a threshold value is exceeded, and a symbol or figure indicating a direction is displayed / flashing. In the upward 1302 screen example, when the user faces upward and exceeds the threshold value, the pointer 1302a moves upward and the direction symbol 1302b is displayed or blinks. The same applies to the front direction 1301, the left direction 1303, the right direction 1304, and the downward direction 1305.

  FIG. 17 is an explanatory diagram showing a state in which symbols, graphics, and the like indicating directions exceeding the threshold are displayed and blinked. In the screen example of upward 1402, a scale (two axes, upper and lower, left and right) 1406 indicating an angle and a threshold, a marker 1408 indicating how much the current direction is, and a frame 1407 indicating the threshold are displayed in real time. ing. The user can learn the face-to-face interval while watching the movement of the marker. The threshold value can also be changed while viewing the frame 1407 indicating the threshold value. The same applies to the front direction 1401, the left direction 1403, the right direction 1404, and the downward direction 1405.

  In addition, feature points can be superimposed on the display of the input image from the camera during face-to-face operation, and the shape and color of the feature points superimposed on the input image can be changed when the feature point limit is exceeded. By combining with various mechanisms for improving the convenience and recognizability of users when acquiring individual difference information, it is possible to adjust individual differences more reliably and easily. FIG. 18 is an explanatory diagram showing an example of the confirmation operation after such adjustment. FIG. 18A shows a state in which it is not facing too much, and feature points are on the face. FIG. 18B shows a state in which it is upward and not too much, and the feature points are on the face. FIG. 18 (c) shows a case where it is upward and excessively directed, and the feature points are indicated by “x” marks at the positions in the front state.

  Further, in the above embodiment, as an example of the procedure for acquiring the individual difference information of the user, the example in which the user is directed to the top, bottom, left, and right only once has been described. The average value or the median value of a plurality of acquired values can be adopted as the correction threshold value.

  In the above-described embodiment, the pointer control unit 106 that controls the pointer displayed on the display unit 107 according to the face orientation determined by the face orientation determination unit 103 has been described as an example. In addition, the present invention can be applied to scrolling the screen of the display device 107 according to the face orientation determined by the face orientation determining unit 103, moving a character (for example, a game machine) displayed on the display device, and the like. Can be used.

  INDUSTRIAL APPLICABILITY The present invention can be used for a face orientation detection device and a face orientation detection method, and in particular, non-contact that detects a feature point of a user's face from an input image and operates the pointer by determining the face orientation The present invention can be used for a face orientation detection device and a face orientation detection method used for an interface.

It is explanatory drawing which shows the structure of one Embodiment of this invention. It is explanatory drawing which shows a feature point. It is explanatory drawing which shows the feature point calculation of a vertical direction determination. It is a flowchart which shows up-down direction determination. It is explanatory drawing which shows the feature point calculation of left-right direction determination. It is a flowchart which shows left-right direction determination. It is explanatory drawing which shows the screen display (up-down direction) of individual difference adjustment. It is explanatory drawing which shows adjustment of an upward threshold. It is explanatory drawing which shows adjustment of a downward threshold. It is explanatory drawing (1) of a downward threshold value. It is explanatory drawing (2) of a downward threshold value. It is explanatory drawing (3) of a downward threshold. It is a flowchart which shows correction of a threshold value. It is a flowchart which shows correction of a threshold value. It is explanatory drawing which shows the example of face direction instruction | indication. It is explanatory drawing which shows the example of an excessive direction warning. It is explanatory drawing which shows the confirmation operation example (1) after adjustment. It is explanatory drawing which shows the confirmation operation example (2) after adjustment. It is explanatory drawing which shows the example of confirmation operation after adjustment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Face direction detection apparatus 101 Image input part 102 Feature point extraction part 103 Face direction determination part 104 Individual difference adjustment part 105 Individual difference information storage part 106 Pointer control part 107 Display part

Claims (9)

In a face orientation determination device for determining a face orientation of a user of an information processing device and using it in an application of the information processing device,
An image input unit for inputting a user's face as an image;
A feature point extraction unit for extracting feature points of the user's face from the image;
A face direction determination unit that determines a user's face direction from the feature points based on a predetermined threshold and transmits information to the application;
An individual difference adjustment unit for acquiring individual difference information for adjusting individual differences for each face orientation for each user;
An individual difference information storage unit that updates the threshold using the individual difference information and registers for each user;
A face orientation determining device comprising: further,
A display unit for displaying various information;
A pointer control unit that controls a pointer displayed on the display unit in accordance with the face direction determined by the face direction determination unit;
The face orientation determination device according to claim 1, comprising: The individual difference adjustment unit
An instruction is given to the user to perform a single movement for each face,
The face orientation determination device according to claim 1, wherein the individual difference information is acquired from an image of each face orientation input from the image input unit. The individual difference adjustment unit
4. The instruction according to claim 3, wherein an instruction is given to the display unit for displaying various types of information as a graphic or an image when giving an instruction for the user to perform a single face-facing operation. 5. Face orientation determination device. The individual difference adjustment unit
The face orientation determination device according to claim 3 or 4, wherein a warning is issued to the user when an operation of each face orientation of the user is inappropriate. In the face orientation determination method for determining the face orientation of the user,
An image input process for inputting a user's face as an image;
A feature point extracting step of extracting feature points of the user's face from the image;
A face orientation determination step of determining a user's face orientation from the feature points based on a predetermined threshold;
Individual difference adjustment process for obtaining individual difference information for adjusting individual differences for each face orientation for each user;
An individual difference information storing step of updating the threshold using the individual difference information and registering it for each user;
A face orientation determination method characterized by comprising: The individual difference adjustment process includes:
Instructing the user to perform a series of actions for each face;
Obtaining the individual difference information from each face-oriented image input from the image input unit;
The face orientation determination method according to claim 6, further comprising: In the step of issuing an instruction to cause the user to perform a single operation for each face,
The face orientation determination method according to claim 7, wherein an instruction is given as a graphic or an image to a display unit that displays various types of information. In the individual difference adjustment step,
9. The face orientation determination method according to claim 7 or 8, wherein a warning is issued to the user when an operation of each face orientation of the user is inappropriate.

Images