JP2006119780A - Apparatus control device utilizing face information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus control device capable of controlling the operation of an apparatus which is reflected by the presence/absence of personal action will to the apparatus. <P>SOLUTION: The apparatus control device is provided with an image information input means 101, a face existence determining means 102 for determining whether a human face is included in an image obtained from inputted image information and a face feature extraction means 104 for extracting the appearance feature information of the face from the image when the face existence determining means 102 determines the existence of a human face in the image and constituted so as to judge user's action will for the use of the apparatus 106 at least on the basis of the feature information extracted by the face feature extraction means 104 and control the apparatus 106. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a device control apparatus and method for controlling the operation of a device in a non-contact manner.

  Conventionally, as this type of device control device and method, there is a device control device and method that controls the operation of a device based on the detection result of a human detection sensor that detects a person. Further, as a general human detection sensor, there are sensors using infrared rays, ultrasonic waves, or image processing.

  Conventional human detection sensors using infrared rays, ultrasonic waves, and the like are used in, for example, an automatic cash handling machine installed in a bank, an automatic door, an air conditioner that senses the position of a person, and the like. Human detection sensors that use infrared rays or ultrasonic waves cannot accurately distinguish between objects, so it is possible to distinguish between people who are willing to use the device and those who are not willing to use the device and are near by accident It is difficult to do.

  In addition, in the conventional device control apparatus and method using infrared rays and ultrasonic waves, other devices such as animals and moving machines such as automatic vacuum cleaners also react, so people such as pet shops and zoos It is difficult to distinguish whether a person is moving when there is something other than moving.

  In the case of a human detection sensor using conventional image processing, for example, a plurality of photographs are taken with a time difference from the same fixed camera, and a person is detected by capturing moving objects by obtaining the difference between the photographs. There is a way to do it.

  In the device control apparatus and method using this image processing, the movement, color, size, and position of the detected object can be determined, but it cannot be determined whether the detected object is a person. Even if it is a person, it cannot be distinguished whether it is a person who intends to use the device or a person who is near by chance and does not intend to use it.

The following Patent Documents 1 to 3 disclose a method for determining whether or not a detected object is a person by extracting a human face feature. However, even in the methods disclosed in these patent documents, only methods for extracting the human face area and face orientation are presented, and there is also an indication until confirmation of the action intention of the detected human device is confirmed. It is not disclosed.
JP 05-225344 A JP 2000-105819 A JP 2001-250121 A

  As described above, in a device control apparatus using a conventional human detection sensor, even if an animal or machine other than a human is detected, or even if a human is detected correctly, the person has an intention to act on the controlled device. In some cases, when a person or the like is simply detected, even if a person who does not intend to use the device is detected, some operation control may be performed on the device.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a device control apparatus or method capable of controlling the operation of a device that reflects the presence or absence of human action intentions on the device.

  To achieve the above object, the device control apparatus according to the present invention includes an image information input unit and a face presence that determines whether a human face is included in an image obtained from the input image information. A determination unit; and a facial feature extraction unit that extracts appearance feature information of the face from the image when the facial presence determination unit determines that there is a human face in the image. , Based on at least the feature information extracted by the facial feature extraction means, the user's intention to use the device is determined, and the device is controlled.

  An apparatus control method according to the present invention for achieving the above object includes an input step of inputting image information, and determining whether a human face is included in an image obtained from the input image information. A face feature extracting step of extracting facial feature information of the face from the image when the face presence determining step determines that there is a human face in the image by the face presence determining step; An action intention determination step for determining an action intention of the user to use the device based on the feature information extracted in the face feature extraction step, and the action intention determined in the action intention determination step Based on the above, control is performed on the device.

  According to the device control apparatus or method of the present invention, the presence / absence of the intention to use is determined by determining the presence / absence of the person's intention to use the device to be controlled from the presence information of the person and the facial feature information of the person. The operation of the device can be controlled based on the above. Specifically, since there is no reaction when there are moving objects other than humans, such as animals and moving machinery, it is possible to reliably distinguish only people even in places where it is difficult to react other than humans, such as pet shops and zoos. In addition, even if the face of a person who does not intend to use the target device is accidentally detected by the image information input means, there is no intention to use the device with a person who intends to use the device. Since a person can be distinguished, appropriate operation control can be performed on the control target device. Thereby, it is possible to prevent wasteful use of controlling the operation of the device until a person unrelated to the operation of the device is detected.

  In the device control apparatus or method according to the present invention, it is also preferable to calculate the size of the face area in the image as the feature information and determine the action intention according to the calculated size of the face area.

  By calculating the size of the face area as feature information, if the size of the face area is large, the person in the face is near the control target device, and if the size of the face area is small, the person in the face is Since it can be seen that the device to be controlled is far away, the area in which the user's device is controlled can be limited, and a person who intends to use the device can be distinguished from a person who does not intend to use the device.

  Furthermore, in the apparatus control apparatus or method according to the present invention, it is also preferable to calculate a feature amount related to the orientation of the face in the image as the feature information, and to determine the action intention according to the calculated feature amount.

  By calculating the orientation of the face in the image as feature information, the user of the device to be controlled is generally oriented in a specific direction with the device as a reference. Can be distinguished from those who do not intend to use it.

  Furthermore, in the apparatus control apparatus or method according to the present invention, it is also preferable to calculate the presence position of the face area in the image as the feature information and determine the action intention according to the calculated presence position.

  By calculating the presence position of the face area in the image as the feature information, the operation of the control target device can be controlled according to the position of the face area, so that the area for controlling the user device can be limited. Therefore, it is possible to distinguish between a person who intends to use the control target device and a person who does not intend to use the apparatus.

  Furthermore, in the device control apparatus or method according to the present invention, as the feature information, the number of face areas present in the image is calculated, and the action intention is determined according to the calculated number of face areas. preferable.

  By calculating the number of face areas present in the image as feature information and controlling the operation of the controlled device according to the number of face areas, when there are multiple people who intend to use the equipment, A person who intends to use the control target device can be distinguished from a person who does not intend to use the device.

  Furthermore, in the apparatus control apparatus or method according to the present invention, a period during which it is determined that there is a human face in the image is measured, and the action intention is determined according to the length of the measured period. It is also preferable to determine.

  By measuring the period during which it is determined that there is a person's face in the image, a person who does not intend to use the control target device has a period during which the presence of the face is continuously determined. Short, conversely, a person who intends to use the face will be longer than a certain period of time to continuously determine the presence of the face, so by setting an appropriate effective detection time, By using together with the determined facial feature information, a person who intends to use the device and a person who does not intend to use the device can be more accurately distinguished.

  Furthermore, in the device control apparatus or method according to the present invention, a period during which the constant feature information is continuously extracted from the image is measured, and the action intention is determined according to the length of the measured period. It is also preferable to determine.

  A period during which a person who does not intend to use the control target device continuously maintains the feature information of the face by measuring the period during which the feature information is continuously extracted from the image. However, the person who intends to use the information will be longer than a certain period of time when the facial feature information is continuously maintained, so by setting an appropriate effective detection time, By using together with the facial feature information determined to be present, it is possible to more accurately distinguish between a person who intends to use the device and a person who does not intend to use the device.

  The computer program according to the present invention comprises program steps for executing the face presence determination step, the face feature extraction step, and the action intention determination step of the device control method according to the present invention having the above characteristics on a computer. It is characterized by.

  A computer-readable recording medium according to the present invention records the computer program according to the present invention.

  Thus, by installing the computer program according to the present invention on a general computer via the recording medium or the network, the face presence determination step of the device control method according to the present invention having the above characteristics, the face The feature extraction step and the action intention determination step can be executed on the computer. In other words, the computer can function as a face presence determination unit, a face feature extraction unit, and an action intention determination unit of the device control apparatus according to the present invention. As a result, the operational effects of the device control apparatus or method according to the present invention described above are specifically exhibited by computer processing.

  Next, an apparatus control apparatus and method according to the present invention (hereinafter referred to as “the present apparatus” and “the present method” as appropriate) will be described with reference to the drawings.

<First Embodiment>
FIG. 1 shows a schematic functional block configuration of the device 100 of the present invention. As shown in FIG. 1, the device 100 of the present invention controls operations of an image information input unit 101, a face presence determination unit 102, a face area detection unit 103, a face feature extraction unit 104, and a control target device 106. The control means 105 is provided.

  The input unit 101 is an imaging unit such as a small camera or an image sensor, for example, and is used to acquire image information for determining the action intention of the user who uses or operates the control target device 106. The image information acquired by the input unit 101 is converted into luminance image information or color information for each pixel in real time into an electrical image signal, for example, an RGB format image signal, and output to the face presence determination unit 102.

  The face presence determination unit 102 determines whether or not a human face is included in the input image generated by the image signal input from the input unit 101. In this embodiment, the eyes, mouth, and nose that characterize a human face are detected by pattern matching, and if the detected eyes, mouth, and nose are in an appropriate relative positional relationship, the human face is present in the input image. Judge that. As a method for determining a human face by eye / mouth / nose pattern matching, for example, there is a method disclosed in Patent Document 1 described above.

  The face area detection unit 103 detects a face area from the input image. In the present embodiment, the face area is detected by detecting the skin color area from the input image including the face area. As a method for detecting a face region by detecting a skin color region, for example, there is a method disclosed in Patent Document 2 described above. When the face presence determination unit 102 determines whether a human face is included in the input image based on the face area detected by the face region detection unit 103, the face region detection unit 103 includes the face presence determination unit 102. Functions as a pre-processing means. The face area detection unit 103 also functions as a preprocessing unit for the face feature extraction unit 104 described later.

  The face feature extraction unit 104 determines the size of the face area based on the image information of the face area detected by the face area detection unit 103 when the face presence determination unit 102 determines that the human face is included in the image. The orientation of the face in the image, the position of the face area in the image, the number of face areas in the image, and the like are extracted as appearance feature information of the face in the image. In the first embodiment, the size of the face area detected by the face area detection unit 103 is used as feature information. Specifically, the face feature extraction unit 104 calculates the area of the face region as an index indicating the size of the face region by counting the number of pixels of the skin color region detected by the face region detection unit 103. The size of the face area is not limited to the area of the face area as long as the eigenvalue can be quantified, for example, using the face height, the face width, or the distance between the left and right eyes. It doesn't matter.

  Based on the determination result of the face presence determination unit 102 and the feature information extracted by the face feature extraction unit, the control unit 105 determines the action intention of the face user having the feature information to use the device 106, Action intention determination unit 107 that outputs a predetermined control signal, and drive control unit 108 that converts the control signal output from the action intention determination unit 107 into a predetermined drive control signal capable of driving the device 106 and outputs the drive signal to the device 106. And controls the operation of the device 106.

  In practice, the device 100 of the present invention is such that the face presence determination means 102, the face area detection means 103, the face feature extraction means 104, and the action intention determination means 107 among the above means are processed by the computer. A storage unit (corresponding to a computer-readable recording medium) storing a computer program composed of program steps to be executed above, and a CPU (central processing unit) that executes the computer program and performs the processing of each means described above ) Etc.

  Specifically, as shown in FIG. 2, the inventive device 100 includes a CPU 201, a main storage device 202, an external storage device 203, a communication interface 204, an internal bus 205, a small camera 101 as input means, and a drive control means 108. It is configured with. Here, in each block block diagram of FIG. 1 and FIG. 2, the same code | symbol is attached | subjected and demonstrated to a common component.

  The CPU 201 functions as the face presence determination unit 102, the face area detection unit 103, the face feature extraction unit 104, and the action intention determination unit 107 shown in FIG. 1, and a program in which each processing procedure by each unit is described is as follows. For example, what is stored in the external storage device 203 and loaded into the main storage device 202 is used for program processing by the CPU 201. Further, the CPU 201 exchanges data with the small camera 101, the main storage device 202, the external storage device 203, and the drive control means 108 that are mutually connected to the CPU 201 via the internal bus 205, and the flowchart of FIG. 3. Each process indicated by is executed.

  The main storage device 202 is usually composed of a semiconductor memory device such as a DRAM (dynamic random access memory).

  The external storage device 203 is composed of, for example, a non-volatile magnetic recording medium such as a hard disk drive (HDD) or a flexible disk, and includes a face presence determination unit 102, a face area detection unit 103, a face feature extraction unit 104, and The computer program for performing each process of the action intention determination means 107 is memorize | stored.

  The CPU 201 can also use the storage device of another network device 207 as an external storage device through a network 206 that is connected to the internal bus 205 and the communication interface 204 by wire or wirelessly.

  The drive control unit 108 is connected to the control target device 106 and the internal bus 205, and the control signal of the action intention determination unit 107 transmitted from the CPU 201 via the internal bus 205 is converted into a drive control signal that can drive the control target device 106. Convert and output.

  Next, the processing procedure of the method of the present invention executed by the device 100 of the present invention will be described with reference to the flowchart of FIG.

  In step S301, an image signal of image information captured by the small camera 101 is input (input process), and the process proceeds to step S302. In step S302, the face area detecting unit 103 detects a face area from the input image generated from the input image signal. Subsequently, in step S303, the face presence determination unit 102 determines whether or not a human face is included in the input image obtained from the image signal (face presence determination step). If the presence of a human face cannot be detected, the process proceeds to step S304, where the behavior intention determination unit 107 performs control according to the state of the control target device 106 at that time without performing the behavior intention determination. A signal is output to the drive control means 108, and the drive control means 108 controls the drive of the control target device 106 and returns to the start.

  If it is determined in step S303 that the face presence determination unit 102 has one or more faces, the process proceeds to step S305. In step S305, the face feature extraction unit 104 extracts feature information from the face area detected by the face region detection unit 103 (face feature extraction step). In the first embodiment, the size of the face area is extracted as the feature information. In step S306, if the detected value of the detected feature information is within a predetermined determination area, the action intention determination unit 107 determines that the user has an intention to act on the control target device 106, and generates a control signal corresponding thereto. It outputs to the drive control means 108, it transfers to step S307, and when the said detected value is not in a predetermined determination area | region, it determines with the user's intention of action with respect to the control object apparatus 106 not existing, and starts. Return (behavior determination process). In step S307, the drive control means 108 performs drive control on the control target device 106 based on the received control signal, and returns to the start. In the first embodiment, in the action intention determination step, if the size of the face area is equal to or greater than a predetermined determination threshold, the user is near the control target apparatus 106 and has an action intention for the apparatus 106. judge.

Second Embodiment
Next, a second embodiment of the device of the present invention will be described. FIG. 4 shows a schematic functional block configuration of the inventive device 400 according to the second embodiment. As shown in FIG. 4, the device 400 of the present invention controls the operations of the image information input means 101, the face presence determination means 102, the face area detection means 103, the face feature extraction means 104, and the control target device 106. In addition to the control unit 105, the face presence determination unit 102 includes a time measuring unit 401 that measures a period during which it is continuously determined that a human face is present in the input image. The difference from the first embodiment is that the inventive device 400 according to the second embodiment is provided with the time measuring means 401, and the action intention judging means 107 of the control means 105 takes into account the period measured by the time measuring means 401. Thus, the user's intention to act on the control target device 106 is determined. In the block configuration diagrams of FIGS. 1 and 4, common components are denoted by the same reference numerals for description.

  Next, the processing procedure of the method of the present invention executed by the device 400 of the present invention will be described with reference to the flowchart of FIG.

  In step S501, an image signal of image information captured by the small camera 101 is input (input process), and the process proceeds to step S502. In step S502, the face area detection unit 103 detects a face area from an input image generated from the input image signal. Subsequently, in step S503, the face presence determination unit 102 determines whether or not a human face is included in the input image obtained from the image signal (face presence determination step). If the presence of a human face cannot be detected, the process proceeds to step S504, where the behavior intention determination unit 107 performs control according to the state of the control target device 106 at that time without performing the behavior intention determination. A signal is output to the drive control means 108, and the drive control means 108 controls the drive of the control target device 106 and returns to the start.

  If it is determined in step S503 that the face presence determination unit 102 has one or more faces, the process proceeds to step S505. In step S505, a fixed determination time effective as a period during which the presence of a human face is continuously determined is waited, and the process proceeds to step S506. In step S506, an image signal of new image information captured by the small camera 101 is input (input process), and in step S507, the face area detection unit 103 again in the input image generated from the input image signal. To detect a face region. In step S508, the face presence determination unit 102 determines again whether or not a human face is included in the input image obtained from the image signal (face presence determination step). In step S508, if the presence of a human face cannot be detected, the process proceeds to step S504. If the face presence determination unit 102 determines that one or more human faces exist, the process proceeds to step S509. In step S509, the face feature extraction means 104 extracts feature information from the face area detected by the face area detection means 103 (face feature extraction step). In the second embodiment, as in the first embodiment, the size of the face area is extracted as the feature information. In step S510, if the action intention determination unit 107 detects that the detected value of the feature information is within a predetermined determination area, the action intention determination unit 107 determines that the user has an action intention for the control target device 106, and generates a control signal corresponding to the action intention. It outputs to the drive control means 108, it transfers to step S511, and when the said detected value is not in a predetermined determination area | region, it determines with the user's intention to behave with respect to the control object apparatus 106, and starts. Return (behavior determination process). In step S511, the drive control unit 108 performs drive control on the control target device 106 based on the received control signal, and returns to the start. In the second embodiment, as in the first embodiment, if the size of the face area is equal to or larger than a predetermined determination threshold in the action intention determination step, the user is near the control target device 106. It is determined that there is an intention to act on the device 106.

  As described above, in the second embodiment, a valid fixed determination time can be set as a period during which the presence of a human face is continuously determined. Therefore, an effective determination time can be set appropriately. Thus, even if the face of a person who does not intend to use the control target device 106 is accidentally captured by the small camera 101 and the presence of the face is determined by the face presence determination unit 102, the presence of the face is temporarily set. If it is less than the effective determination time, as a result, the temporarily detected face person can be treated as a person who does not intend to use the control target device 106 and the state of the control target device 106 can be prevented from changing.

<Third Embodiment>
Next, a third embodiment of the device of the present invention will be described. FIG. 6 shows a schematic functional block configuration of the inventive device 600 according to the third embodiment. As shown in FIG. 4, the device 600 of the present invention controls operations of the image information input unit 101, the face presence determination unit 102, the face area detection unit 103, the face feature extraction unit 104, and the control target device 106. In addition to the control means 105, the face feature extraction means 104 is configured to include a time measuring means 601 for measuring a period during which human face feature information existing in the input image is continuously extracted in a certain state. ing. The difference from the first embodiment is that the device 600 of the present invention according to the third embodiment is provided with the time measuring means 601, and the action intention determination means 107 of the control means 105 takes into account the period measured by the time measuring means 601. Thus, the user's intention to act on the control target device 106 is determined. Also, the difference from the second embodiment is that the time counting means 401 according to the second embodiment measures a period during which the face presence determination means 102 continuously determines that there is a human face in the input image. On the other hand, the time measuring unit 601 according to the third embodiment has a period during which the facial feature extracting unit 104 continuously extracts the facial feature information of the person existing in the input image in a certain state. It is a point to measure. For example, in the third embodiment, when the face feature extraction unit 104 extracts the size of a face as feature information, the period during which the face area is equal to or greater than a certain size as a certain state is constant. It is measured as a period of continuous extraction. In the block configuration diagrams of FIGS. 1 and 6, common components are denoted by the same reference numerals for description.

  Next, the processing procedure of the method of the present invention executed by the device 600 of the present invention will be described with reference to the flowchart of FIG.

  In step S701, an image signal of image information captured by the small camera 101 is input (input process), and the process proceeds to step S702. In step S702, the face area detection unit 103 detects a face area from an input image generated from the input image signal. Subsequently, in step S703, the face presence determination unit 102 determines whether or not a human face is included in the input image obtained from the image signal (face presence determination step). If the presence of a human face cannot be detected, the process proceeds to step S704, where the behavior intention determination unit 107 performs control according to the state of the control target device 106 at that time without performing the behavior intention determination. A signal is output to the drive control unit 108, and the drive control unit 108 controls the drive of the control target device 106. In step S705, the counter value N, which will be described later, is reset.

  If it is determined in step S703 that the face presence determination unit 102 has one or more faces, the process proceeds to step S706. In step S706, the face feature extraction unit 104 extracts feature information from the face area detected by the face region detection unit 103 (face feature extraction step). In the third embodiment, as in the first embodiment, the size of the face area is extracted as feature information. In step S707, the behavior intention determination unit 107 proceeds to step S708 if the detected value of the detected feature information is within the predetermined determination region, and if the detected value is not within the predetermined determination region, It is determined that there is no action intention for the user's control target device 106, and the process returns to the start via step S705 (action intention determination step). In step S708, the time measuring means 601 waits for a certain unit delay time to elapse and increments the counter value N by 1. In step S709, it is determined whether or not the counter value N has reached the maximum count value Nmax. When the counter value N is less than the maximum count value Nmax, the process returns to the start and the same process is repeated. Note that whether or not the detected value of the feature information is within the predetermined determination area in step S707 is determined based on whether or not the size of the face area is greater than or equal to a predetermined determination threshold, as in the first embodiment.

  In step S709, when the counter value N matches the maximum count value Nmax, the period during which the facial feature extraction unit 104 continuously extracts facial feature information of a person existing in the input image in a certain state. However, it is determined that a predetermined effective determination period has elapsed, and the process proceeds to step S710. In step S <b> 710, the behavior intention determination unit 107 determines that the detected value of the detected feature information is within a predetermined determination region for a predetermined effective determination period, and the user's intention to act on the control target device 106. It is determined that it exists, and a control signal corresponding to it is output to the drive control means 108. Moreover, the drive control means 108 performs drive control with respect to the control object apparatus 106 based on the received control signal, and returns to the start.

  As described above, in the third embodiment, it is possible to set a constant determination time that is effective as a period in which human face feature information existing in the input image is continuously extracted in a fixed state. By appropriately setting the effective determination time by adjusting the unit delay time and the maximum count value Nmax, the face of a person who does not intend to use the control target device 106 is accidentally captured by the small camera 101, Even if the presence of the face is determined by the face presence determination means 102 and it is determined that the person is willing to use the control target device 106 temporarily, the extraction of the feature information of the face is temporarily set and effective. If the time is equal to or shorter than the determination time, as a result, the temporarily detected face person can be treated as a person who does not intend to use the control target device 106 and the state of the control target device 106 can be prevented from changing.

<Fourth embodiment>
Next, a fourth embodiment of the device of the present invention will be described. The device of the present invention according to the fourth embodiment uses the facial feature extraction means 104 of the device 100, 400 or 600 of the present invention in any of the first to third embodiments as appearance feature information of a human face. Replaced by means for extracting the orientation of the face in the input image, other means, that is, input means 101, face presence determination means 102, face area detection means 103, face feature extraction means 104, control means 105, action intention determination The means 107, the drive control means 108, and the timing means 401 or 601 are configured to include the same as those in the first to third embodiments.

  As a method for extracting the orientation of the face in the input image, the face feature extraction unit of the fourth embodiment obtains each position of the eyes, mouth, and nose from the face area detected by the face area detection unit 103, and compares them. The orientation of the face is extracted from the specific position. As a method for extracting the same face orientation, for example, there is a method disclosed in Patent Document 3 described above.

  Since the feature information extracted by the face feature extraction unit is the same as the first to third embodiments except that the face information is the orientation of the face, the configuration and processing procedure of the device of the present invention according to the fourth embodiment are as follows. Omits duplicate explanations.

<Fifth Embodiment>
Next, a fifth embodiment of the device of the present invention will be described. The device of the present invention according to the fifth embodiment uses the face feature extraction means 104 of the device 100, 400 or 600 of the present invention in any of the first to third embodiments as appearance feature information of a human face. Replaced by means for extracting the position of the face area in the input image, other means, that is, input means 101, face presence determination means 102, face area detection means 103, face feature extraction means 104, control means 105, action The intention determination unit 107, the drive control unit 108, and the time measuring unit 401 or 601 are configured to include the same ones as in the first to third embodiments.

  For example, the face feature extraction unit of the fifth embodiment obtains the center position of the face area detected by the face area detection unit 103 as a method for extracting the position of the face area in the input image, and the input image of the center position is obtained. Extract the relative position inside.

  Since the feature information extracted by the face feature extraction means is the same as that in the first to third embodiments except that the feature information is the position of the face area, the configuration and processing of the device of the present invention according to the fifth embodiment For the procedure, duplicate explanation is omitted.

<Sixth Embodiment>
Next, a sixth embodiment of the device of the present invention will be described. The device of the present invention according to the sixth embodiment uses the facial feature extraction means 104 of the device 100, 400 or 600 of the present invention according to any of the first to third embodiments as appearance feature information of a human face. Substituting with means for extracting the number of face areas present in the input image, other means, that is, input means 101, face presence determination means 102, face area detection means 103, face feature extraction means 104, control means 105, The action intention determination unit 107, the drive control unit 108, and the time measuring unit 401 or 601 are configured to include the same components as those in the first to third embodiments.

  The face feature extraction means of the sixth embodiment extracts the number of face areas detected by the face area detection means 103 existing in the input image and determined as a human face by the face presence determination means 102.

  Since the feature information extracted by the face feature extraction means is the same as the first to third embodiments except that the feature information is the number of face regions, the configuration and processing procedure of the device of the present invention according to the sixth embodiment I will omit the duplicate explanation.

  Hereinafter, another embodiment of the device of the present invention and the method of the present invention will be described.

  <1> In each of the embodiments described above, the face feature extraction means determines the size of the face area, the orientation of the face in the image, and the face area in the image from the image information of the face area detected by the face area detection means 103. Although the case where any one of the existing position and the number of face regions existing in the image is extracted as appearance feature information of the face existing in the image has been described, the features extracted by the face feature extracting unit The information is not limited to any one of the size of the face area, the orientation of the face in the image, the position of the face area in the image, and the number of face areas present in the image. The feature information other than the above may be used, or two or more of the feature information may be used in combination.

  <2> In each of the above embodiments, each program step for executing the processes of the face presence determination unit 102, the face area detection unit 103, the face feature extraction unit 104, and the action intention determination unit 107 on the computer In the above description, the computer program is stored in the external storage device 203 of the computer constituting the device of the present invention and used. However, the computer program supply method is not limited to the method of each of the above embodiments. Absent.

  For example, the computer program is stored in a portable recording medium such as a CD-ROM or DVD, the recording medium is supplied to the apparatus of the present invention, and the computer (or CPU or MPU) constituting the apparatus of the present invention is The program code stored in the recording medium may be read out, loaded into the main storage device 202, and executed. Further, the program code stored in the recording medium may be directly read out and executed using a semiconductor memory or the like that can be accessed at high speed as the recording medium. As a recording medium for supplying the program code, for example, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a magnetic tape, a nonvolatile memory card, or the like can be used.

  In this case, the program code itself read from the recording medium realizes the functions of the above embodiments, and the recording medium storing the program code constitutes the apparatus of the present invention.

  Further, the program code may be downloaded from another computer system to the main storage device 202 or the external storage device 203 of the present invention via a transmission medium such as the network 206 in FIG.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code. A part or all of the actual processing may be performed, and the functions of the above embodiments may be realized by the processing.

  Further, after the program code read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. The CPU or the like provided in the board or function expansion unit may perform part or all of the actual processing, and the functions of the above embodiments may be realized by the processing.

The block diagram which shows the schematic functional structure in 1st Embodiment of the apparatus control apparatus which concerns on this invention. The block diagram which shows the schematic hardware constitutions in 1st Embodiment of the apparatus control apparatus which concerns on this invention. The flowchart which shows the rough process sequence in 1st Embodiment of the apparatus control method which concerns on this invention. The block diagram which shows the schematic functional structure in 2nd Embodiment of the apparatus control apparatus which concerns on this invention. The flowchart which shows the rough process sequence in 2nd Embodiment of the apparatus control method which concerns on this invention. The block diagram which shows the schematic functional structure in 3rd Embodiment of the apparatus control apparatus which concerns on this invention. The flowchart which shows the rough process sequence in 3rd Embodiment of the apparatus control method which concerns on this invention.

Explanation of symbols

100, 400, 600: Device control apparatus according to the present invention 101: Image information input means (small camera)
102: Face presence determination means 103: Face area detection means 104: Face feature detection means 105: Control means 106: Control target device 107: Action intention determination means 108: Drive control means 201: CPU
202: Main storage device 203: External storage device 204: Communication interface 205: Internal bus 206: Network 207: Network device 401, 601: Timekeeping means

Claims (16)

Image information input means;
Face presence determination means for determining whether a human face is included in an image obtained from the input image information;
Face feature extracting means for extracting appearance feature information of the face from the image when the face presence determining means determines that there is a human face in the image;
With
A device control apparatus that determines control of the device by determining a user's intention to use the device based on at least the feature information extracted by the face feature extraction unit. The face feature extraction means calculates a size of a face area in the image as the feature information,
The device control apparatus according to claim 1, wherein the action intention is determined according to the size of the face area calculated by the face feature extraction unit. The face feature extraction means calculates a feature amount related to the orientation of the face in the image as the feature information,
The device control apparatus according to claim 1, wherein the action intention is determined according to the feature amount calculated by the face feature extraction unit. The face feature extraction means calculates the presence position of a face region in the image as the feature information,
The device control apparatus according to claim 1, wherein the action intention is determined according to the presence position calculated by the face feature extraction unit. The face feature extraction means calculates the number of face regions present in the image as the feature information,
The device control apparatus according to claim 1, wherein the action intention is determined according to the number of the face areas calculated by the face feature extraction unit. A timing unit that measures a period during which the face presence determination unit continuously determines that there is a human face in the image;
The device control apparatus according to claim 1, wherein the action intention is determined according to the length of the period measured by the time measuring unit. A timing means for measuring a period during which the face feature extraction means continuously extracts the feature information from the image;
The device control apparatus according to claim 1, wherein the action intention is determined according to the length of the period measured by the time measuring unit. An input process for inputting image information;
A face presence determination step of determining whether a human face is included in an image obtained from the input image information;
A face feature extraction step of extracting appearance feature information of the face from the image when it is determined by the face presence determination step that there is a human face in the image;
An action intention determination step of determining an action intention of the user using the device based on at least the feature information extracted in the face feature extraction step;
A device control method, comprising: controlling the device based on the behavior intention determined in the behavior intention determination step. In the face feature extraction step, as the feature information, a size of a face region in the image is calculated,
9. The apparatus control method according to claim 8, wherein, in the action intention determination step, the action intention is determined according to the size of the face area calculated in the face feature extraction step. In the face feature extraction step, as the feature information, a feature amount related to the orientation of the face in the image is calculated,
The device control method according to claim 8, wherein, in the action intention determination step, the action intention is determined according to the feature amount calculated in the face feature extraction step. In the face feature extraction step, as the feature information, the presence position of the face region in the image is calculated,
The device control method according to claim 8, wherein in the behavior intention determination step, the behavior intention is determined according to the presence position calculated in the face feature extraction step. In the face feature extraction step, as the feature information, the number of face regions present in the image is calculated,
9. The device control method according to claim 8, wherein, in the action intention determination step, the action intention is determined according to the number of the face regions calculated in the face feature extraction step. The face presence determination step includes a time measuring step of measuring a period during which it is continuously determined that there is a human face in the image,
The device control method according to any one of claims 8 to 12, wherein, in the action intention determination step, the action intention is determined according to a length of the period measured in the timing step. The face feature extraction step includes a time measuring step of measuring a period during which the constant feature information is continuously extracted from the image,
The device control method according to any one of claims 8 to 12, wherein, in the action intention determination step, the action intention is determined according to a length of the period measured in the timing step.   The computer program provided with the program step for performing the face presence determination process of the apparatus control method of any one of Claims 8-12, a face feature extraction process, and an action intention determination process on a computer.   A computer-readable recording medium recording the computer program according to claim 15.

Images