JPH10260772A - Object operation device and object operation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To operate and select an object by a non-contact detection method without using a line-of-sight detection method for calculating the line-of-sight direction having strict accuracy by relating, storing, recognizing and controlling the state of the object and the input information of a face image. SOLUTION: This device is provided with an image input part 2, a face image processing part 3 for analyzing the face of a human and obtaining a feature amount, a recognition and judgement part 4 for relating, storing, recognizing and controlling the state of the object and the input information of the face image and an object control part 5. Then, during work, the face of the human is photographed, image analysis is performed and the feature amount is obtained. A dictionary pattern for recognition is prepared by using the feature amount and it is related to the operation contents of the object. At the different point of time, in the case of viewing the object, the face image is similarly obtained and recognized by using the dictionary pattern, and in the case of being close to the face image at the time of operating the object before, the object is automatically selected as an operation object.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an object operation device and a method thereof.

[0002]

2. Description of the Related Art In recent years, the advancement of human interfaces such as computers has become important, and simple and easy interfaces using more advanced input devices have been invented. For example, as an interface using a line-of-sight detection device, without using a device such as a mouse or pen,
For controlling a window, see Japanese Patent Laid-Open No.
027: window selection method). Conventionally, there is also an invention for accurately obtaining gaze information.

[0003]

However, in the non-contact gaze detecting device, it is necessary to perform a calibration for associating a gazing point with a human state (face direction or pupil position). The calibration corrects the gazing point and the pupil position by looking at a specially prepared prescribed index. In order to increase the detection accuracy, a considerable amount of time and repetitive operations are required, so that the burden on the user is large.

Further, in order to obtain a gazing point, near-infrared rays are applied (JP-A-61-172552, JP-A-4-2302).
7) Use stereo equipment to accurately detect objects and pupil positions (Akira Banno, Fumio Kishino, Yukio Kobayashi: "Prototype of eye gaze detection device that allows pupil extraction and head movement", Electronics Transactions of the Communication Society (D) Vol.J76-D-
II, No. 3, pp. 636-646 (1993))
Such a large device was required.

According to the present invention, an object can be operated and selected by a non-contact detection method without using a line-of-sight detection method for calculating a line-of-sight direction with strict accuracy. In the apparatus of the present invention, at first, work is performed on an object by using another device as in the related art. Since the user is watching the object during the work, a human face is photographed during the work, and image analysis is performed to acquire a feature amount. A dictionary pattern for recognition is created using the feature amount, and is associated with the operation content of the object. At another point, if you are looking at the object, if you look at the face image in the same way, perform recognition using the created dictionary pattern, and if it is close to the face image when the object was previously operated, That is, it is possible to automatically select the object as an operation target without explicitly using a device.

Taking a window system of a computer as an example, first, a window is focused using a mouse, and while working in the window, a face image of the working person is acquired and watched. The feature amount of the face in the direction in which it is located is obtained. After the dictionary pattern is created using the feature amount, recognition is performed using the dictionary pattern, and when the face image is close to the face image when looking at a window, focusing is automatically performed on the window. .

Here, when the mouse is operated,
The human performs visual feedback that the mouse cursor follows the eyes of the person 44 as shown in FIG. 13, and the cursor of the mouse 45 controls the position of the mouse by constantly following the line of sight on the display unit 46. I have. That is, the line of sight position can be acquired naturally during the mouse moving operation. If this property is used, there is no need to prepare a special index unlike the conventional method, and it is not necessary to perform calibration in a manner that shows the index.

[0008] Further, when an object such as a window having an area larger than that of a mouse cursor is identified by the line of sight, 1) the window is watched during the operation of the window (such as key input). 2) There is no need for line-of-sight detection accuracy.

Thus, a face image may be acquired during operation of the window, and a dictionary pattern may be created using the face image. Create a dictionary pattern for each window,
When recognizing, by calculating which dictionary is closest,
The window can be selected, and the same effect as that of the visual line detection can be obtained.

[0010] By appropriately creating a dictionary pattern,
It is not necessary to perform calibration using a special index in advance. In addition, it is possible to automatically acquire and use a more easy-to-use environment and style in the course of working without breaking the conventional usage form.

[0011]

In order to achieve the above object, a first aspect of the present invention provides an image input unit for inputting an image, a face image processing unit for analyzing a human face and obtaining a feature amount, and an object. Associating the state of the face with the input information of the face image, memorizing,
It has a recognition determination unit that performs recognition and control, and an object control unit that controls objects. As a result, an operation such as automatically selecting an object based on a face direction or the like becomes possible.

A second invention is a dictionary for storing and collecting feature amounts obtained by analyzing a human face when the recognition judging unit receives operation information of a specified object, and identifying the feature amounts. 2. The object operation device according to claim 1, further comprising a dictionary generation unit that generates a dictionary of patterns. Thus, a dictionary is generated as appropriate for each object, so that calibration is not required.

According to a third aspect of the present invention, the recognition judging section has a recognizing section for identifying a category having a similar degree of similarity by using a dictionary generated based on a feature obtained by analyzing a human face. And a judgment control unit for generating operation information associated with. Thus, the object can be operated only with the feature amount obtained from the face.

According to a fourth aspect of the present invention, the recognition judging section has a dictionary generating section for a feature quantity obtained by analyzing a human face and a recognizing section for identifying the dictionary. Accordingly, a judgment control unit that controls switching of the above two functions is provided. Thereby, recognition control can be performed according to the operation state of the object.

A fifth aspect of the present invention is characterized in that the recognition determining section utilizes a subspace method in the dictionary generating section and the recognizing section. Accordingly, since the subspace method is used to create a dictionary pattern, a dictionary can be easily generated, and stable identification can be performed.

According to a sixth aspect of the present invention, there is provided image input means for inputting an image, face image processing means for analyzing a photographed human face, and feature amounts obtained by analyzing a human face during object operation. It has a recognition determination method for storing and determining the similarity with the stored feature amount and generating operation information of the object, and an object control means for controlling the object. This provides a method of performing an operation such as automatically selecting an object based on a face direction or the like.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below. An apparatus 1 according to the invention is shown in FIG. The apparatus comprises four parts: an image input unit 2, a face image processing unit 3, a recognition determination unit 4, and an object control unit 5.

[Image Input Unit] The image input unit 2 is a device for photographing a human face, and obtains an image using one ordinary TV camera. The input may be a black and white image, but in the case of a color input, it is converted to black and white and sent to the face image processing unit 3.

In the present embodiment, the processing is performed with a black and white image. However, if the face image processing unit 3 needs color information or the like, a color image may be output. The image input unit 2 may include a plurality of cameras, and may use not only a fixed input camera but also a camera capable of controlling focus (aperture), aperture, zoom, and the like. A camera rotatable in pan and tilt directions for changing the shooting direction of the camera may be used.

[Face Image Processing Unit] FIG. 2 shows an example of the configuration of the face image processing unit 3. The face image processing unit 6 in this embodiment includes a face detection unit 7, a face part detection unit 8, and a partial image generation unit 9.

In the face detecting section 7, the input image is
Affine transformation (translation, scaling, rotation) and cropping (cutout) are performed to generate a partial image of a specified size. Specifically, the input image is reduced in several stages, and an image of a size (N pixel × N pixel), which is the point of performing raster scan, is cut out from each reduced image (see FIG. 10). For the clipped partial image,
The degree of similarity between the prepared face image and the dictionary pattern is calculated. Here, the similarity is calculated using the subspace method.

The subspace method uses the collected samples as KL
Expansion (principal component analysis) is performed to obtain an orthonormal basis (eigenvector), and this is used as a dictionary pattern. Each test pattern is a dictionary pattern of each category (eigenvector)
The sum of the inner products of is calculated, and this is used as the similarity. At this time, the test pattern belongs to the category having the highest similarity. The eigenvector φ = (φ _i , ..., φ
The similarity s with the partial image G is obtained using _m ).

[0023]

(Equation 1)

A resolution and a position having the highest similarity are found in the extracted images, and it is determined that a face exists (see the lower center of FIG. 10). The face part detection unit 8 detects face parts such as eyes, nose, and mouth from the detected face area.
In the present embodiment, eyes (pupils) and nose (nostrils), which are face parts, are detected.

As a method for extracting these characteristic portions, as shown in FIG. 11 (b), first, as shown in FIG. 11 (b), the shape of a round region was extracted as a candidate for eyes and nose. Separation Mask (Osamu Yamaguchi, Kazuhiro Fukui, "Face Image Analysis Using Separation Features-Eye Pupil Detection-", Information Processing Society of Japan 52nd Annual Convention (2) -pp. 187-188, 1
996. ) Is used to detect a round area. Note that a conventional method may be used as a feature point extraction method, and any method may be used. From a plurality of round regions, a combination of four regions was determined to be the most likely face when considered as an eye / nose candidate.
Points are specified as eye-nose candidates. The determination of the face likeness is verified by comparison with a prepared face model. The face model may use a conventionally proposed method such as a method using density information or a structural feature value using a spring model.

The partial image generation unit 9 uses the detected position (feature point) of the face part as a reference as shown in FIG.
Cut out the face area. Based on the four points selected as feature points, the region is requantized from the image to generate a small image (normalized image) having a specified size. The size of the normalization here is a size of 15 × 15 pixels, as shown in FIG.
(D)) is set. The gray value at the position of the linear sum of the vectors is defined as the gray value of the image after the clipping (FIG. 11).
(E)). Note that the size of this cutout (15 × 15 pixels) is not limited to this. Further, the cutout portion is not limited to this.

When the cutout is performed using the position of the face part as described above, different feature amounts are obtained as shown in FIG. 12 depending on the difference between the face direction and the pupil position. FIG.
FIG. 12B is a schematic diagram of the gray value when looking at the bottom, FIG. 12C is a right diagram when looking at the right, and FIG. 12D is a schematic diagram of the gray value when looking at the left. As a result, a feature amount such as a face direction can be used without directly obtaining a line of sight.

By discriminating the characteristic amount of shading as shown in FIG. 12, it is possible to distinguish a screen having a size of about a 17-inch display into about nine divisions based on a difference in face direction. However, this requires calibration in advance. That is, it is necessary to acquire the feature amount of the face while viewing the corresponding divided screen and generate a dictionary in advance. Using this, simple menu selection and the like are also possible.

[Object Control Unit] The object control unit 5 comprises an object state management unit 15, an event management unit 16, and an object state change unit 17, as shown in FIG.

The object state management unit 15 manages state information on objects such as creation, management, and deletion of objects. The event management unit 16 stores information such as devices prepared for operating the object,
Handles the processing performed on the object, and all event information resulting from the processing performed by the object,
to manage. Note that an event represents operation information, operation content, and the like of an object in the system, and indicates a minimum unit of operation.

The object state changing unit 17 changes the state of each object, and performs control and processing related to the display of the object. In the present embodiment, in order to specifically describe a window manager in a computer window system as an example, as shown in FIG. 5, the object control unit 18 is a window control unit (window manager), and the object state management unit is a window control unit. The management unit 19 and the object state change unit are referred to as a window display change unit 21.

[Window control unit] Window control unit 1
8 is a function equivalent to a window manager in a normal window system, but includes the following three window management units 19, an event management unit 20, and a screen display change unit 21.
It is described as consisting of

<Window management unit> Window management unit 1
Numeral 9 manages attribute information such as the position and size of the displayed window, and how the windows overlap. When each window is newly created, the size (w, h), position (x, y), name (name), and ID number (idnumber) of the window are as follows ((x, y), (w, h), (name), (idnu)
mber)).

The (x, y), (w, h) of each window is used to detect the overlap between windows, and which window overlaps with another window is calculated and held.

<Event Management Unit> An event refers to a minimum unit of operation such as mouse movement, button operation, window operation, and key input in a window system.

An event is expressed as a set of (event type, object (window) ID where the event has occurred, event value (amount)).

The event management unit 20 performs a process for an event from a device such as a mouse or a keyboard, and also gives an instruction to change a screen display when an event occurs. For example, if there is a keyboard input from the user,
An event from the window system occurs, and the event is sent to the event management unit 20. The window to which the event is sent is processed by the event management unit 20. That is, it manages whether a window is a selection target (referred to as focus).

<Screen Display Changing Unit> The screen display changing unit 21 generates an event when the screen display is changed, such as displaying a window, drawing images, characters, figures, and the like in the window, and moving a mouse cursor. Change the window display when

For example, when the focus is moved by moving the mouse, the fact that the focus has been changed is changed by changing the color of the window frame. When the window is moved, the movement is brought to a designated position.

[Recognition determining section] Next, the recognition determining section 4 shown in FIG.
Will be described. FIG. 3 shows the configuration. The recognition determination unit 10 includes a dictionary generation unit 12, a recognition unit 11, a determination control unit 13,
Consists of

The dictionary generation unit 12 generates a dictionary pattern for recognition using the feature amount generated by the face image processing unit 3. Here, a dictionary pattern for recognition is generated using partial images of a plurality of cut out faces. The dictionary generation unit 12 performs a process as shown in FIG. 6 according to an instruction from the determination control unit 13. First, images are collected until an instruction is received from the next determination control unit (step 22). Then, when images of a certain constant are collected, a variance-covariance matrix from those images is constructed (step 23).
By performing KL expansion of the matrix, eigenvalues of the matrix,
An eigenvector is calculated (step 24). Specifically, a matrix calculation such as the yacobi method or LU decomposition may be performed. Next, the corresponding eigenvectors are rearranged in descending order of the eigenvalues, and only the top several eigenvectors are taken out and registered as dictionary patterns (step 2).
5). A plurality of dictionary patterns can be provided and can be deleted arbitrarily.

<Recognition Unit> The recognition unit 11 includes a dictionary generation unit 12
Using the dictionary pattern created by the above, it is determined to which dictionary pattern the image cut out separately is closest. The similarity is calculated using the subspace method described above.
The similarity s _i of the partial images is obtained from the i-th dictionary pattern (eigenvector) φ _i .

[0043]

(Equation 2)

[0044] In this case, for all s _i, is classified in the category of the dictionary pattern with the greatest s _i. <Judgment Control Unit> The judgment control unit 13 1) receives information from the object control unit 5, 2) controls recognition and dictionary generation, and 3) issues an instruction to the object control unit 5. As one embodiment of the judgment control unit 13, as shown in FIG. 7, it comprises an event arbitration unit 29, an event verification unit 27, and an event generation unit 28.

The event arbitration unit 29 receives an event from the object control unit 5, determines whether there is a related event, and distributes the event to each of the event verification unit 27 and the event generation unit 28. Here, the type of the event and the window ID where the event has occurred are checked so that only the event related to whether or not the window is being watched is selected. That is, (type, w
inID, value)
e, select using winID.

The event verification section 27 operates according to a flowchart shown in FIG. First, event information is received from the object control unit (window control unit) (step 30). If there is an event (Step 3
1) Check the ID of the window where the event occurred. If the event is from the target window ID (step 32), the dictionary generation unit is instructed to collect images for dictionary generation (step 33).

When the position information of the window itself is changed such as moving / deleting the window, the recognition unit is instructed to delete the dictionary pattern. The event generator operates according to the flowchart shown in FIG. Event information is received from the object control unit (step 37). If the received event is not a mouse event (step 38), and if the recognition result from the recognition unit is gazing at a certain window, the focus change is performed. Generate an associated event, such as an event, and send it to the object control unit.

The operation of each of these parts will be described according to an embodiment. First Embodiment (Window Manager) This embodiment describes an example in which the present invention is applied to a window system having a screen input (GUI) in a personal computer, a workstation, or the like.

The selection (focusing) of the window focus is taken as an example. Window focus refers to selecting a window to be operated, such as input from a keyboard, from a plurality of windows.

In the conventional window system, in order to perform window focus, when there are two windows as shown in FIGS. 14C and 14D, the mouse is moved into each of the windows and the mouse cursor is moved. By bringing the window inside the window, focusing on the window can be performed.

In this embodiment, initially, the focus of the window is focused on using the mouse, and the face image of the working person is acquired while the work is being performed in the window. After the dictionary patterns are created, recognition is performed using those dictionary patterns, and when the image is close to the face image when a certain window is being viewed, focusing is performed on the window.

An image of a human face is obtained by using a camera mounted near a display (display device) of a personal computer or a workstation as an input. In this embodiment, the camera is mounted near the lower side of the display, installed at an angle to look up, and captures the face.

Description will be made from a state where one window exists as shown in FIG. A new window is created as shown in FIG. Then the window is
Focusing is performed by the mouse as shown in FIG.

FIG. 15 shows a state transition diagram of the window. Nodes represented by ellipses indicate respective states, and arcs indicate event types indicating operation contents. n
ew is a new window creation, key push, ke
y release is keyboard input, mouse
e move is mouse movement, iconify Dei
“conify” indicates an instruction to convert a window into an icon or a window.

For example, (mouse move, 123,
(X, y)) indicates that the mouse has moved to the position of (x, y) in the window ID 123. FIG.
The change from (b) to FIG. 14 (c) is a transition to “focus state” after “window generation”. When the window is created, the window manager registers (ID, position, size, name) of the new window. Then, the window ID is sent to the judgment control unit of the recognition judgment unit.

When transitioning between the “focused state” and the “key input state”, that is, when the user is working in the focused window, the judgment control unit of the recognition judgment unit instructs the dictionary generation unit to transmit the dictionary. Instruct image collection to generate a pattern. In the case of key input, image acquisition is performed not only at the moment when the key is pressed but also during the time when the key is continuously pressed.

The dictionary generation unit calculates the window ID
When the number of collections reaches a certain constant, dictionary generation is performed as shown in FIG. When the dictionary pattern is generated, the dictionary generation unit notifies the determination control unit of the information of the dictionary pattern (a set of the dictionary and the window ID).

The judgment control section sends update information of the dictionary pattern to the recognition section, and the recognition section receives a new dictionary pattern from the dictionary generation section. The recognition unit sends the window ID to the determination unit as a result of the recognition using the dictionary pattern.

In this example, since there are two windows, it is sufficient to generate a recognition dictionary for each of the two windows. The judgment control unit receives the event information from the window control unit and detects whether there is a mouse movement, a button, or the like. If no event related to the mouse has occurred, an event is generated using the recognition result (window ID) from the recognition unit so that the focus is adjusted to the window of that ID, as in the case of moving the mouse. Send to control unit.

As a result, normally, as shown in FIG.
In addition to focusing with the mouse, FIG.
Focusing can be performed even if the mouse is not in the window, and key input can be continued.

The window control unit instructs the screen display changing unit to display on a screen that the focus has been changed to the focused window, and causes the window to bring the focus into focus.

Next, a case where a window is moved / deleted will be described. So far, we have described giving focus to a window when there is a window in the viewing direction. When the window is moved / deleted by mouse operation, the dictionary pattern used for focusing cannot be used, and the dictionary pattern needs to be updated.

First, when window movement / deletion is controlled by mouse operation, the event management unit detects window movement / deletion. The event management unit notifies the judgment control unit which window has been moved / deleted to the judgment control unit. The determination control unit instructs the recognition unit to delete the dictionary pattern used for identifying the window. Further, when an event such as a mouse focusing on the window ID has occurred with respect to the window ID, the determination control unit gives an instruction to the dictionary generation unit to generate an image to generate a new dictionary pattern.

The case where the window is iconified will be described. If it is iconized, the recognition unit
An instruction is sent from the judgment control unit to stop obtaining the similarity for the window. As a result, the iconized window is not given focus.
When the iconized window is returned to the original window, the dictionary is incorporated into the recognition unit again,
Perform recognition.

Embodiment 2 (Gaze Detection) As described above, the display screen is divided into about nine divisions based on the difference in the direction of the face by identifying the characteristic amount of light and shade as shown in FIG. Can be. In this case, calibration is required. That is, the feature amount of the face is acquired while viewing the corresponding split screen,
A dictionary must be created in advance.

However, by using the present invention, unlike the related art, the following calibration method using a mouse can be realized. As shown in FIG. 16, there are nine divided screens, and the mouse can be moved. When the mouse is moved and the user is looking at the mouse, the color is changed from FIG. 16A to FIG. 16B. This is not to change the color according to the position of the mouse, but to change the color deeply according to the number of captured images of the human face when looking at the division position. Next, when the mouse is moved as shown in FIG. 16C, the change of the upper right color stops, and the color changes according to the number of captured images when looking at the upper center portion. When the time elapses, the state changes to FIG. 16D, indicating that a larger number of sheets have been acquired than in the upper right corner. As shown in FIG. 16E, this is sequentially performed for all the division positions, and FIG.
When this state is reached, the feature amount of each face has been collected.

For realization, a window is allocated to each of the nine divisions, and a dictionary is generated for the window where the mouse is placed as described in the first embodiment.
At this time, an instruction is issued to the window control unit to change the color of the window according to the number of acquired images used for generating the dictionary. When the colors of all the windows have changed, the dictionary generation has been completed for all the windows, which has the advantage that humans can easily perceive.

Recognition is performed using the created dictionary, and the present invention can be applied to simple menu selection and a conventional gaze detection application. Of course, calibration of the type presented from the system side, as in the related art, may be performed.

Embodiment 3 (Remote Control) Using a home electric appliance as an example, consider application to channel selection by a remote control of a television. When the operation of changing the channel is performed by the remote controller, the channel is changed by the remote controller operation while looking at another location (A, B, C, D) different from the screen as shown in FIG. This corresponds to the event of the previous embodiment.

An image input unit is installed on a television, and a face image of a person watching the television is acquired. The face image processing unit performs the same processing. The judgment control unit associates the channel selection made while looking at a certain direction, and after the dictionary is generated, it is possible to change the channel only by looking at the direction. The object control unit requires a channel changing unit.

In this case, the position of the object (the position of the television) does not change, but the position of the human changes. In this case, it is possible to cope with a method of updating the dictionary and coping with it, and a method of generating a dictionary for each human position by taking the position of the human as an event type.

Embodiment 4 (Support for Other Media) As an example in which a device for voice recognition is added, consider a case in which home appliances and social systems are controlled using voice recognition.
In the case of a system that uses speech recognition, a system that uses only speech recognition often has incorrect recognition. This changes depending on whether the user is in a state of sending the command, and the recognition rate is reduced due to the directivity of a microphone installed in the system.

In some cases, the shape of the lips is added to the recognition.
The direction of the face is also important when acquiring it. Here, an example is shown in which the state information of another type of information (face information) at the time of voice registration is supplemented.

At the time of the first access (at the time of registration), the feature amount of the face of the user to be registered is acquired and registered in the dictionary. When there are a plurality of words to be registered, if each is treated as an event, a plurality of face state information can be registered.

In the case of performing recognition using voice recognition, even when the result of voice recognition is poor, more reliable recognition can be performed by considering together which words are close to the feature amount of the generated face. Will be possible.

A modified example will be described. In the first and second embodiments, a window is mainly described as an example. However, a window may be an object, and the contents of a target event may be changed as in the third and fourth embodiments. That is, the present invention may be applied not only to computers but also to household electric home appliances, automobiles, social systems and the like.

In the judgment control unit, the conventional event generation using a device such as a mouse and the event generation based on the face direction described in the present invention have priority over the event generation using a mouse device in the embodiment. However, the priorities may be reversed.

The recognition section may perform recognition several times for a certain period of time, and may use the dictionary category having the largest number of recognition times as the recognition result. Considering a case where a key input is performed on a window system, a case where the user is looking at the keyboard or a case where the user gazes at another window also occurs. In order to prevent this, a mechanism that stores information of past recognition and obtains the relationship between the point of regard and the feature amount may be provided. From this, it is possible to improve the accuracy of the dictionary by judging the general direction (whether the user is looking at the keyboard or looking at a window other than the collection target window), and removes those that are out of the collection target. .

In a window application, when there are many key inputs such as an editor, the position of the cursor may change frequently. In this case, the dictionary may be updated sequentially regardless of the movement of the window.
This control may be changed for each application.

In the above-described embodiment, in the above-described embodiment, the gray level value of the rectangular portion including the eyes and nose of the face is used as the feature value. May be similarly extracted as a gray value set, and a dictionary may be generated in consideration of pupil position fluctuation.

If further accuracy is required, replacement with or use in combination with a conventional visual axis detection device may be performed. The object control unit can be replaced with various systems, and the interface function based on the face direction can be easily extended and changed.

[0082]

According to the present invention, the control of an object, which has been controlled using a conventional device such as a mouse, is sequentially learned, and the control of the object is performed using information such as a face direction and a gaze direction. Can be changed and operated.

Further, there is no need to perform calibration in advance as in the case of conventional gaze detection, and control can be performed without using a mouse. Thereby, even if the subject does not learn special calibration, wasteful work can be omitted and efficient work can be performed.

[Brief description of the drawings]

Fig. 1 System configuration

FIG. 2 shows an embodiment of a face image processing unit.

FIG. 3 is an embodiment of a recognition determining unit.

FIG. 4 is an embodiment of an object control unit.

FIG. 5 shows an embodiment of a window control unit.

FIG. 6 is a flowchart of dictionary generation.

FIG. 7 is a configuration example of a judgment control unit.

FIG. 8 is a flowchart of an event verification unit.

FIG. 9 is a flowchart of an event generation unit.

FIG. 10 is an explanatory diagram of face detection.

FIG. 11 Method of feature point detection and segmentation

FIG. 12 is an explanatory diagram of a face direction and a face feature amount.

FIG. 13 is an explanatory diagram of a state when a mouse is used.

FIG. 14 is an explanatory diagram of a window focus.

FIG. 15 is a state transition diagram of a window state.

FIG. 16: Calibration method

FIG. 17 is an explanatory diagram of an embodiment in a television.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Object operating device 2 ... Image input part 3 ... Face image processing part 4 ... Recognition determination part 5 ... Object control part

Claims (6)

[Claims] An image input unit for inputting an image, a face image processing unit for analyzing a photographed human face, and a feature amount obtained by analyzing a face of a person who is operating an object. An object operation device, comprising: a recognition determination unit configured to determine a degree of similarity with a stored feature amount and generate operation information of an object; and an object control unit configured to control the object. 2. A method according to claim 1, wherein receiving the operation information of the specified object, the recognition determination unit stores and collects a feature amount obtained by analyzing a human face, and generates a dictionary of a dictionary pattern for identifying the feature amount. 2. The object operation device according to claim 1, further comprising a dictionary generation unit that performs the operation. 3. A recognition judging section having a recognition section for identifying a category having a similar degree of similarity by using a dictionary generated based on a feature amount obtained by analyzing a human face, and being associated with the category. 2. The object operation device according to claim 1, further comprising a determination control unit that generates operation information. 4. A recognition determining unit includes a dictionary generating unit for a feature amount obtained by analyzing a human face, and a recognizing unit for identifying the dictionary, and according to an operation state of a target object, 2. The object operation device according to claim 1, further comprising a determination control unit that controls switching between the two units. 5. The object operation device according to claim 1, wherein the recognition determination unit uses a subspace method in the dictionary generation unit and the recognition unit. 6. An image input means for inputting an image,
A face image processing means for analyzing a photographed human face and a feature amount obtained by analyzing the human face during the operation of the object are stored, a similarity with the stored feature amount is determined, and operation information of the object is determined. An object operation method, comprising: a recognition determining means for generating; and an object control means for controlling an object.