JP2005092419A - Information processing apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To execute desired processing upon an action preferred by an operator only by associating and registering arbitrary actions and instructions for every operator, when executing an instruction depending on an operator's action by analyzing an image photographing the operator. <P>SOLUTION: A CPU 1 analyzes an operator image photographed by a camera part 7 and refers to a face table to identify the operator, and refers to a gesture motion table to identify the operator's gesture. If the gesture made is registered in the gesture motion table, a command associated with the gesture is read from a gesture command table, and processing directed by the command is executed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an information processing apparatus and a program having a photographing function for photographing an operator.

In general, in an information processing apparatus such as a disc / notebook personal computer having a camera function for photographing an operator, an operator image (face image) obtained by photographing the operator is registered in advance instead of inputting a password at the time of login. The operator authentication is performed by comparing the registered image (face image).
Conventionally, for example, a system is known in which personal authentication is performed using a face image and a face motion (see Patent Document 1).
There is also known an information processing apparatus that recognizes a moving direction of a user based on an image obtained by photographing the user and outputs a command according to the moving direction (see Patent Document 2).

JP 2001-212112 A JP 2001-306236 A

 The system disclosed in Patent Document 1 described above is a technology that increases the safety factor at the time of personal authentication by considering the movement of the face in addition to the face image, and is premised only on login. The information processing apparatus disclosed in Patent Document 2 is a technique for improving operability by outputting a command corresponding to the moving direction of the user. However, the information processing apparatus is assumed to move in a predetermined direction. In addition, if the movement direction is the same, the same command is always output in a fixed manner.

  An object of the present invention is to execute an instruction command corresponding to an operator's action by analyzing an image captured by the operator, and to register an arbitrary action and an instruction command in association with each operator. It is to be able to execute a desired process with an action that suits the operator's preference.

The invention according to claim 1 is an information processing apparatus having a photographing function for photographing an operator, and stores and manages behavior information that is arbitrarily registered in advance for each operator and specifies the behavior information in association with the behavior information. Registration information storage means for storing and managing an instruction command for instructing execution of processing, image analysis means for analyzing the behavior of the operator based on an operator image obtained by photographing the operator, and analysis obtained by the image analysis means Recognizing means for recognizing whether the action of the operator is as registered content by comparing the result with the action information corresponding to the operator, and that the action as registered content is performed by this recognizing means A process execution means for reading out the instruction command associated with the action information and executing a specific process when it is recognized; It is characterized in.
Furthermore, a program for realizing the main functions shown in the invention described in claim 1 is provided to the computer (the invention described in claim 10).

The invention described in claim 1 may be as follows.
When the behavior information is registered for each operator, an operator image including a portion including a characteristic behavior is registered as behavior information by analyzing an operator image obtained by photographing the behavior of the operator (claim 2). Invention).

  The registered information storage means stores and manages a plurality of types of behavior information arbitrarily registered for each operator, and assigns different types of instruction commands arbitrarily registered in association with the plurality of types of behavior information for each behavior information. And when the recognition means recognizes that the action of the operator corresponds to any action information, the process execution means reads out the instruction command associated with the action information. The specific processing is executed (the invention according to claim 3).

  The analysis means analyzes an operator's action based on an operator image taken at power-on, and the processing execution means immediately after turning on the power when it is recognized that the operator's action is as registered. Is instructed to shift to the specific processing in accordance with the instruction command associated with the action information (the invention according to claim 4).

  The image analysis means pays attention to a hand portion in the operator image, identifies an operation change of the operator's hand by analyzing the image portion in time series, and the recognition means is obtained by the image analysis means. It is recognized whether or not the movement change of the given hand is in accordance with the registered content (the invention of claim 5).

  The image analysis means pays attention to the face portion in the operator image, specifies the moving direction of the operator by analyzing the image portion in time series, and the recognition means is obtained by the image analysis means. It is recognized whether or not the movement direction of the operator is in accordance with the registered content (the invention according to claim 6).

  The image analysis means focuses on the hand part in the operator image, identifies the number of fingers extended by analyzing the image part, and the recognition means recognizes the finger obtained by the image analysis means. It is recognized whether the number is as registered (invention of claim 7).

  The image analysis unit focuses on a face part and a hand part in an operator image, specifies a relative positional relationship between the face and the hand, and the recognition unit determines a relative position obtained by the image analysis unit. It is recognized whether or not the relationship is as registered (invention of claim 8).

  The image analysis means identifies the appearance state of the face by paying attention to the face portion in the operator image, and the recognition means determines whether or not the appearance state of the face obtained by the image analysis means is as registered content. (Invention of claim 9).

  According to the first aspect of the present invention, action information that is arbitrarily registered in advance for each operator is stored and managed, and an instruction command for instructing execution of the specific process is stored in association with the action information. In a managed state, when the operator's action is analyzed based on an operator image obtained by photographing the operator, and the analysis result is compared with the action information corresponding to the operator, the action of the operator is as registered Since the instruction command associated with this action information is read and the specific process is executed, the operator can simply register any action information and the instruction instruction in association with each operator. It is possible to execute a desired process with an action that suits the user's preference. In this case, different instructions can be executed for each individual even with the same action, and conversely, the same instruction can be executed for each individual with different actions, and the registration contents can be arbitrarily changed as necessary. The degree of freedom increases, and it is possible to expect a significant improvement in usability.

 According to the invention described in claim 2, in addition to having the same effect as that of the invention described in claim 1, the operator image obtained by photographing the action of the operator is analyzed when the action information is registered for each operator. Since the operator image including the part including the characteristic action is registered as the action information, the action information can be automatically registered only by taking any action in front of the camera.

  According to the invention described in claim 3, in addition to having the same effect as the invention described in claim 1 described above, a plurality of types of behavior information arbitrarily registered for each operator are stored and managed, and the plurality of types In a state where different types of instruction commands arbitrarily registered in association with the behavior information are stored and managed for each behavior information, when it is recognized that the behavior of the operator corresponds to any behavior information, Since the instruction command associated with the action information is read out and the specific process is executed, the operator can selectively execute the desired instruction instruction only by changing the type of action.

  According to the invention described in claim 4, in addition to having the same effect as the invention described in claim 1, the operator's action is analyzed based on the operator image photographed when the power is turned on, and the operator's action is registered. When it is recognized that the content is as it is, it is instructed to shift to the specific process according to the instruction command associated with the action information immediately after the power is turned on. Therefore, it can be expected to shorten the time from the power-on to the desired process and to simplify the operability.

  According to the invention described in claim 5, in addition to having the same effect as that of the invention described in claim 1, the operator pays attention to the hand portion in the operator image, and analyzes the image portion in time series to obtain the operator. The movement change (hand gesture) of the freely moving hand can be used as the action of the operator because the movement change of the hand is identified and it is recognized whether or not the movement change of the hand is as registered. It is possible to respond freely according to the operator's preference.

  According to the invention described in claim 6, in addition to having the same effect as that of the invention described in claim 1, the operator pays attention to the face portion in the operator image, and analyzes the image portion in time series to obtain the operator. The movement direction of the operator is specified, and whether or not the movement direction of the operator is the same as the registered content is recognized. For example, the movement in the up / down direction, the movement in the left / right direction, and the diagonal direction Movement and forward / rearward movement can be used as the action of the operator, and it is possible to freely respond to the preference of the operator.

  According to the seventh aspect of the invention, in addition to the same effect as the first aspect of the invention described above, the finger is extended by paying attention to the hand portion in the operator image and analyzing the image portion. Since the number is specified and it is recognized whether or not the number of fingers is as registered, it is possible to execute a desired process by a simple action of extending the finger. In addition to the number of fingers extended, if the direction of extension is taken into consideration, it is possible to freely respond to the preference of the operator.

  According to the invention described in claim 8, in addition to having the same effect as that of the invention described in claim 1, the relative positional relationship between the face and the hand is determined by paying attention to the face part and the hand part in the operator image. Since it is specified and whether or not this relative positional relationship is the same as the registered content, the position of the hand moving freely and the relative positional relationship with the face can be used as the action of the operator. , It is possible to respond freely according to the preference of the operator.

According to the ninth aspect of the invention, in addition to the same effect as the first aspect of the invention,
Focusing on the face part in the operator image, the appearance state of the face is specified, and it is determined whether or not the appearance state is as registered, so for example, the camera part is blocked with a hand or the like. The time (intentional image blocking time) or the number of appearances of the face can be used as the action of the operator, and a desired process can be instructed by a simple action.

(Example 1)
A first embodiment of the present invention will be described below with reference to FIGS.
FIG. 1 is a block diagram showing basic components of the information processing apparatus in this embodiment.
This information processing apparatus is a disc / notebook personal computer equipped with a video camera for photographing an operator, and by analyzing an operator image obtained by photographing the operator, an instruction command (in accordance with the gesture (hand movement)) Command) and an arbitrary gesture and a command are registered for each operator in association with each other, and a desired process can be executed with a gesture suitable for the operator's preference. That is, paying attention to the hand part in the operator image, analyzing the movement of the operator's hand by analyzing this image part in time series, and if the movement change of the hand is as registered, A registration command associated with this hand movement (gesture) is read out and a specific process is executed.
Before detailed description of the features of this embodiment, the hardware configuration of this embodiment will be described below.

  The CPU 1 is a central processing unit that controls the overall operation of the information processing apparatus in accordance with the operating system and various application software in the storage unit 2. The storage unit 2 is a fixed memory such as a hard disk, and has a program storage area and a data storage area. The program storage area in the storage unit 2 stores an application program for realizing the present embodiment in accordance with an operation procedure shown in FIGS. 7 and 8 described later, and the data storage area stores FIG. 2 described later. 4 stores data (face table, gesture operation table, gesture command table) shown in FIG. The program and data are loaded into the work memory 3 as necessary, and the data in the work memory 3 is saved in the storage unit 2. The storage medium 4 is a detachable memory such as a CD-ROM or DVD, and exchanges programs, image data, and the like with the storage unit 2 and the work memory 3.

On the other hand, an operation unit 5, a display unit 6, and a camera photographing unit 7 which are input / output peripheral devices are connected to the CPU 1 via a bus line, and the CPU 1 operates these input / output devices according to an input / output program. Take control.
The operation unit 5 is an input unit that constitutes a pointing device such as a keyboard, a touch panel, a mouse, or a touch input pen, and inputs character string data and various commands. The display unit 6 is a liquid crystal display unit that performs full-color display, a CRT display unit, a plasma display unit, or the like. The camera photographing unit 7 is a digital video camera equipped with a photographing lens, a lens / mirror block such as a mirror, an image sensor such as a CCD image sensor, a driving system thereof, and the like, and is attached to the entire surface of the apparatus main body, Are automatically captured, and a series of operator images (video images) are sequentially transferred to the image memory 8.

FIG. 2 is a diagram showing the configuration of the face table 11.
The face table 11 stores and manages authentication face images registered in advance for operator authentication, and can store a plurality of face images for each operator in order to increase the recognition rate. Yes. That is, for each operator (A, B, C,...), An “operator ID” and a plurality of “face patterns” are stored in association with each other, and the “operator ID” is an identification arbitrarily entered by the operator. The “face pattern” is information and is a face image for authentication photographed with different face orientations and expressions. The CPU 1 extracts a face part from an operator image photographed by the camera photographing unit 7 when the power is turned on, and compares the extracted image (face image) with each “face pattern” in the face table 11. An operator is specified, and authentication processing is performed to determine whether the operator is a legitimate operator.

FIG. 3 is a diagram showing the configuration of the gesture operation table 12.
The gesture operation table 12 stores and manages an operation pattern indicating an arbitrary gesture (hand movement) registered by the operator, and a series of operations expressing the gesture corresponding to the “gesture ID” for each gesture. Operation (1), operation (2), and operation (3) can be stored as patterns. In this embodiment, the fist movement change is illustrated as a gesture. The CPU 1 acquires a series of operator images (video images) photographed by the camera photographing unit 7, and the fists in the operator images are captured. By paying attention to the portion and analyzing the image portion in time series, an operator image (still image) including a characteristic fist position is extracted as an operation pattern and registered in the gesture operation table 12. ing.

FIG. 4 is a diagram showing the configuration of the gesture command table 13.
The gesture command table 13 stores and manages a plurality of types of gestures arbitrarily registered in advance for each operator, and stores and manages different types of commands arbitrarily registered in association with the plurality of types of gestures for each gesture. In the figure, “operator ID” is set in the horizontal direction, and “gesture ID” is set in the vertical direction, and each of the matrix-like intersections (“operator ID” and “gesture ID”) ( A “command (com1, com2, com3,...)” Is set in the grid portion. In this case, the correspondence between the gesture and the command is arbitrary. For example, a different command can be registered for each operator even with the same gesture, and conversely, the same command can be registered for each operator even with different gestures. Further, the contents of the gesture command table 13 can be arbitrarily changed as necessary.

The “command” registered in the gesture command table 13 is arbitrarily selected and designated by the operator from a command menu prepared in advance. For example, a specific mode (keyword search mode, initial condition setting) is set after the power is turned on. Command to instruct to move to the confirmation mode, etc., command to instruct the operator to read out and display personal data specific to the operator, command to instruct to save the data during work, the previous time Command for instructing the cursor to move to the editing position.
The CPU 1 searches the gesture command table 13 based on the “operator ID” specified by referring to the face table 11 and the “gesture ID” specified by referring to the gesture operation table 12, and the corresponding “ The “command” is read out and a specific process indicated by the command is executed.

FIG. 5 is a diagram for explaining gesture types classified as gestures according to changes in fist motion.
That is, the gesture type that can be analyzed in this embodiment is shown. As this gesture type, as shown in (Example 1), the fist is made to flow linearly in one direction so as to move diagonally from the upper right to the lower left. Type, as shown in (Example 2), a type in which the straight line changes direction so that the fist flows from the upper right to the lower left and then turns back to the lower right, and as shown in (Example 3), the fist is moved from right to left As shown in (Example 4), the type moves back and forth as it returns from the left to the right after flowing, and is roughly classified as the type that rotates as the fist rotates counterclockwise (counterclockwise).

FIG. 6 is a diagram illustrating a plurality of types of gestures classified as gestures according to changes in fist motion.
In other words, in addition to the above-mentioned (Example 1), there are gestures such as “Up → Down”, “Left → Right”, etc. In addition to the above-mentioned (Example 2), there are gestures such as “Up → Center → Right” and “Down → Center → Left”. In addition to (Example 3), for example, there are gestures such as “Left → Right → Left → Right”. ”Gesture. FIG. 6 shows a state in which an arbitrary command (instruction content) is registered linked to each gesture described above.

  Next, the operation concept of the information processing apparatus in the first embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. It is also possible to sequentially execute operations according to the program code transmitted via the transmission medium. This also applies to other embodiments described later, and in addition to the recording medium, operations specific to this embodiment can be executed using programs / data supplied externally via a transmission medium.

FIG. 7 is a flowchart showing an operation at the time of gesture registration.
First, when the gesture registration mode is instructed, the CPU 1 activates the camera photographing unit 7 (step A1). Then, the camera photographing unit 7 performs still photography of the operator in front of the apparatus main body and transfers it to the image memory 8. Here, the CPU 1 acquires an operator image (still image) from the image memory 8 (step A2), and extracts a face portion from the operator image (step A3). Then, the extracted image (face image) and each “face pattern” for each operator registered in the face table 11 are sequentially compared to identify the operator (step A4). In this case, after correcting the orientation and size of the face of the image, the shape, position, size, hairstyle, presence / absence and type of glasses, etc. of the face contour, eyes, mouth, nose, forehead, etc. are individually compared. Then, image comparison is performed by comprehensively comparing them (hereinafter the same). As a result, if there is no characteristically matching image (NO in step A5), the operator recognizes that it has not been registered, and proceeds to step A6 to display a confirmation message inquiring whether or not to perform operator registration.

 This message screen is provided with an OK button for instructing to perform operator registration. The presence / absence of operation of this OK button is checked (step A7). When the OK button is operated, a message is displayed prompting the operator to input an operator ID. In response to this, the operator ID input is captured (step A8), and the face orientation and expression are changed. After displaying the message for prompting what should be done, an operator image is acquired again and the face portion is extracted (step A9). Then, the inputted operator ID and each extracted operator image (face image) are associated with each other and newly registered in the face table 11 (step A10).

 Next, the process moves to a process of registering an arbitrary gesture or command corresponding to the operator newly registered as described above (steps A11 to A21). On the other hand, when the face image is compared in step A5 described above and recognized as a registered operator, the process proceeds to processing for registering an arbitrary gesture or command corresponding to the operator (step A11). ~ A21). First, the CPU 1 obtains a series of operator images (moving images) video-captured by the camera photographing unit 7 (step A11), and divides and extracts a plurality of operation patterns from the series of photographed moving images (step A12).

 That is, in order to identify a gesture performed by the operator in front of the camera, attention is paid to a fist portion in the operator image, and a characteristic fist position is included by analyzing this image portion in time series. An operator image (still image) is extracted as an operation pattern. For example, when a gesture is performed that causes the fist to flow from upper right to diagonally lower left, the movement pattern when the fist is positioned on the upper right of the face from a series of video shots, the fist positioned in front of the face At least three types of motion patterns are extracted, such as the motion pattern when the fist is positioned and the motion pattern when the fist is positioned diagonally to the lower left. Then, the gesture motion table 12 is searched based on the plurality of motion patterns (step A13), and the presence or absence of the same pattern is checked (step A14). Here, if it is the same as the registered gesture (YES in step A14), the “gesture ID” associated with the gesture is acquired from the gesture operation table 12 (step 15), but a new gesture that is not a registered gesture is acquired. If a gesture is performed (NO in step A14), a new “gesture ID” is generated (step A16), and this “gesture ID” is associated with the current gesture (a series of motion patterns). Is newly registered in the gesture motion table 12 (step A17).

Next, in a state where the command menu is read and displayed (step A18), when an arbitrary command is selected and specified from the command menu (step A19), the command menu is based on the operator ID and gesture ID. The gesture command table 13 is searched, and the command selected and specified is registered at the intersection of the operator ID and the gesture ID (step A20).
When the process for registering an arbitrary gesture or command is completed in this way, the process proceeds to the next step A21, where it is checked whether or not there is an operation for instructing the registration end, and the process returns to step A11 until the end instruction is given. Repeated (steps A11 to A21).

FIG. 8 is a flowchart showing the operation when the power is turned on.
First, when the power is turned on, the CPU 1 activates the camera photographing unit 7 (step B1), and then accesses the image memory 8 to acquire an operator image (still image) (step B2). A face part is extracted from the list (step B3). Then, the operator authenticates the operator by sequentially comparing the extracted image (face image) and each “face pattern” of each operator registered in the face table 11 (step B4). As a result, if there is no characteristically matching image (NO in step B5), it is recognized that the operator is not a regular operator, and the power is automatically turned off. However, if the image is a regular operator (YES in step B5). ), This “operator ID” is read from the face table 11 to identify the operator (step B6).

 Then, a series of operator images (moving images) taken by the camera photographing unit 7 are acquired (step B7), and a plurality of operations are performed by analyzing the gestures performed by the operator in front of the camera when power is turned on in time series. The pattern is divided and extracted (step B8), the gesture motion table 12 is searched based on the plurality of motion patterns, and the “gesture ID” corresponding to the hand motion performed this time by the operator is read and the gesture is specified ( Step B9). By searching the gesture command table 13 based on the “operator ID” and “gesture ID” specified in this way, the corresponding “command” is read (step B10), and the processing instructed by the command is executed. (Step B11). Such an operation is repeated every time the power is turned on.

  As described above, in the first embodiment, the CPU 1 identifies the operator with reference to the face table 11 while analyzing the operator image captured this time, and refers to the gesture operation table 12 to make the gesture of the operator. When a gesture as specified in the gesture operation table 12 is specified, the command associated with the gesture of the operator is read from the gesture command table 13 and instructed by this command. Since the process is executed, it is possible to execute a desired process with a gesture suitable for the operator's preference only by registering an arbitrary gesture and a command in association with each operator. In this case, different commands can be executed for each operator even with the same gesture, and conversely, the same command can be executed for each operator with different gestures. Furthermore, the registration contents of the gesture command table 13 can be arbitrarily changed as necessary, so that the degree of freedom is increased and a great improvement in usability can be expected.

In this case, by making the movement change (hand gesture) of the freely moving hand as the gesture of the operator, it is possible to respond freely according to the preference of the operator.
In addition, when registering the gesture for each operator, the operator image including the characteristic action is extracted and registered by analyzing the operator image obtained by photographing the operator's gesture. By simply making an arbitrary gesture in front of the camera, automatic gesture registration becomes possible.

The gesture command table 13 can store a plurality of types of gestures arbitrarily registered for each operator, and can store different types of commands arbitrarily registered in association with the plurality of types of gestures for each gesture. If the operator's hand movement corresponds to any registered gesture, the command associated with this gesture is read and the process is executed. By simply changing the type, you can selectively execute the desired command.
The movement of the operator's hand is analyzed based on the operator image photographed when the power is turned on. If the movement of the operator's hand is as registered in the gesture operation table 12, it is associated with the gesture immediately after the power is turned on. Since the process is shifted to the process according to the command, the process can be shifted to the desired process immediately after the power is turned on. Simplification of operability can be expected.

In the first embodiment described above, when registering a command, a command is selected from a command menu prepared in advance. However, an operator directly inputs an arbitrary command and registers it. You may make it do.
In the first embodiment described above, the movement of the fist is exemplified as the movement of the hand, but the movement of the palm, wrist, and forearm may be used.
(Example 2)

A second embodiment of the present invention will be described below with reference to FIGS.
In the first embodiment described above, attention is paid to the fist portion in the operator image, and the change in the hand of the operator (gesture) is specified by analyzing the image portion in time series. In the second embodiment, the operator's action is specified by paying attention to other parts. That is, it is possible to specify the movement direction of the operator by paying attention to the face portion in the operator image, to specify the number of fingers extended by paying attention to the hand portion in the operator image, Identify the relative positional relationship between the face and hand by focusing on the face and hand, and identify the time that the camera is blocked by the hand (intentional image blocking time) as the action of the operator It is a thing.
Here, the same or the same names in both embodiments are denoted by the same reference numerals, the description thereof will be omitted, and the following description will focus on the features of the second embodiment. .

FIG. 9 and FIG. 10 are diagrams illustrating a case where the moving direction of the operator is specified as the operator's action by paying attention to the face portion in the operator image.
That is, the type of movement direction that can be analyzed in this embodiment is shown. As the type of movement direction of the face, as shown in (Example 1) of FIG. As shown in (Example 2), the type of flowing in the right direction, as shown in (Example 3) of FIG. 10, the type of flowing in the upward direction, and as shown in (Example 4) of FIG. The type exists. An arbitrary command can be registered for each operator corresponding to each moving direction, and the CPU 1 executes an arbitrary command corresponding to the moving direction.
Note that the present invention is not limited to the case of moving the face with the apparatus main body fixed, and in a small information processing apparatus such as a notebook type, the apparatus main body may be moved with the face fixed. .

FIG. 11 is a diagram exemplifying a case where the number of fingers extended (number of fingers raised) is specified as the operator's action by paying attention to the hand portion in the operator image.
That is, (Example 1) is a state where one finger is raised, (Example 2) is a state where two fingers are raised, (Example 3) is a state where three fingers are raised, (Example 4) shows a state where five fingers are raised. In this way, an arbitrary command can be registered for each operator corresponding to the index with the finger raised, and the CPU 1 executes an arbitrary command corresponding to the index.

FIG. 12 is a diagram illustrating a case where the relative positional relationship between the face and the fist is specified as the action of the operator by focusing on the face portion and the fist portion in the operator image.
That is, (Example 1) is a state where the fist is moved and stopped at the position of the hair at the center of the face (position of hairline), (Example 2) is a state where the fist is moved near the left ear and stopped (Example 3) ) Shows a state where the fist is moved to the position of the jaw and stopped, and (Example 4) shows a state where the fist is moved to the position of the nose and stopped. An arbitrary command can be registered for each operator in correspondence with such a relative positional relationship, and the CPU 1 executes an arbitrary command according to the position of the fist.

FIG. 13 is a diagram showing an intentional image blocking state in which the camera portion is closed with a palm or a finger.
That is, paying attention to the face part in the operator image, the time interval from the first appearance of the face to the next appearance is counted as the intentional image blocking time, and this time is specified as the operator's action To do. An arbitrary command can be registered for each operator corresponding to such an image blocking time (for example, 1 second, 3 seconds), and the CPU 1 executes an arbitrary command according to the image blocking time. .

  In the second embodiment, a face table 11 similar to that in the first embodiment described above is provided. In this case, the registration to the face table 11 is performed when the operator registration button is operated, but is basically the same as the first embodiment, and the face portion extracted from the operator image, The entered operator ID is associated and registered in the face table 11. In the second embodiment, instead of the gesture operation table 12 and the gesture command table 13 shown in the first embodiment described above, the face movement command table 14, the finger command table 15, the relative command table 16, the blocking A time command table 17 is provided.

FIG. 14 is a diagram showing the configuration of the face movement command table 14.
The face movement command table 14 stores and manages commands arbitrarily registered corresponding to the movement direction of the operator's face for each operator. In the figure, the “operator ID” in the horizontal direction and the command in the vertical direction “Movement direction” is set, and “command” is set to each of the matrix-shaped intersections (mesh portions) constituted by the “operator ID” and “movement direction”. In this case, in the face movement command table 14, “left”, “right”, “lower”, “up” corresponding to the types of (Example 1) to (Example 4) shown in FIG. 9 and FIG. The “movement direction” is fixedly stored, but it is arbitrary whether the command registration is performed in association with all of these four types or the command registration is performed only for the type arbitrarily selected and designated from among them. .

FIG. 15 is a diagram showing the configuration of the finger command table 15.
The finger command table 15 stores and manages commands arbitrarily registered according to the number of fingers raised by the operator for each operator. In the figure, the “operator ID” in the horizontal direction and the command in the vertical direction , “Index” is set, and “command” is set in each of the matrix-shaped intersections (cells) constituted by the “operator ID” and “index”. In this case, the finger command table 15 has “1”, “2”, “3”, “5” “index” corresponding to the indexes of (Example 1) to (Example 4) shown in FIG. Is stored in a fixed manner, but it is optional to register a command in association with all of the indices or to register a command only to an index that is arbitrarily selected and designated from among them.

FIG. 16 is a diagram showing a configuration of the relative relationship command table 16.
The relative relationship command table 16 stores and manages commands arbitrarily registered corresponding to the relative positional relationship between the face and the fist for each operator. A “relative relationship” is set in the direction, and a “command” is set in each of the matrix-like intersections (grid portions) constituted by the “operator ID” and “relative relationship”. Yes. In this case, in the relative relationship command table 16, “center”, “left ear”, “chin”, “nose” corresponding to the indices of (Example 1) to (Example 4) shown in FIG. The “relative relationship” is stored in a fixed manner, but it is arbitrary whether the command registration is performed in association with all of the relative relationships, or the command registration is performed only for the relative relationship arbitrarily selected and designated from among them.

FIG. 17 is a diagram showing the configuration of the cutoff time command table 17.
The shut-off time command table 17 stores and manages commands arbitrarily registered corresponding to the intentional image shut-off time for blocking the camera part with a palm or a finger. “Operator ID”, “Image blocking time” is set in the vertical direction, and “Command” is set at each matrix-shaped crossing portion (grid portion) composed of “Operator ID” and “Image blocking time”. "Is set. In this case, the “image blocking time” of “1 second” and “3 seconds” is fixedly stored in the blocking time command table 17, but command registration is performed in association with all of the image blocking times. Whether the command registration is performed only during the image blocking time arbitrarily selected and designated from among them is arbitrary.

FIG. 18 is a flowchart showing an operation when an arbitrary command is registered in each of the command tables 14 to 17 for each operator.
First, when the command registration mode is instructed, the CPU 1 displays and outputs a table name menu (step C1). When a table to be registered is selected and designated (step C2), it is fixed in this designation table. Each action item memorized automatically is read out and displayed in a list on the registration screen (step C3). For example, when the face movement command table 14 is selected as a registration target, each “movement direction” is read and displayed as a list, and when the finger command table 15 is selected, each “index” is read. Is displayed in a list.

  This registration screen is provided with a command input area arranged corresponding to each action item, an operator ID input area, and a registration end button. Here, when an operator ID is entered in the registration screen (step C4), a command menu is displayed and output (step C5), and when an arbitrary command is selected and designated (step C6), this registration is performed. The command name is set in the corresponding command input area in the screen (step C7). Then, whether or not the registration end button is operated is checked (step C8). If the registration end button is not operated, the process returns to step 6 to repeat the command input. Here, when the registration end button is operated, the input information in the registration screen is registered in the corresponding command table (step C9).

FIG. 19 to FIG. 20 are flowcharts showing operations when a camera instruction button for instructing execution of a command is operated by analyzing an operator image.
First, the CPU 1 activates the camera photographing unit 7 in response to the operation of the camera instruction button (step D1). Then, the camera photographing unit 7 performs still photography of the operator in front of the apparatus main body and transfers it to the image memory 8. Here, the CPU 1 acquires a photographed image (still image) from the image memory 8 (step D2), extracts a face part from the photographed image (step D3), and checks the presence or absence of the face part (step D4). ).

  Here, if the face portion does not exist (NO in step D4), the operation of the camera instruction button is disabled (non-function), but if the face portion exists, the extracted face image and face table 11 are displayed. The operator is identified by sequentially comparing each “face pattern” registered for each operator, and this “operator ID” is read (step D5). Then, the image of the fist part is searched from the photographed image (step D6), and the presence or absence of the fist image is checked (step D7). If there is no fist image, the process proceeds to step D13. However, if the operator's fist is shown in the captured image, the relative relationship between the face image and the fist image is specified and extracted by image analysis (step D8). . Then, the relative relationship command table 16 is searched based on the “operator ID” and the “relative relationship” between the face and the fist (step D9), and the corresponding relative relationship corresponding to this operator is registered in the command table 16. If there is no corresponding, the process proceeds to step D13. If there is a corresponding, the intersection between the “operator ID” and “relative relationship” from the relative relationship command table 16 is determined. Is read (step D11), and the process instructed by this command is executed (step D12).

  On the other hand, when it is determined NO in step D7 or D10 described above, attention is paid to the movement of the operator's finger. That is, paying attention to the finger in the photographed image, the finger image is searched (step D13), and the presence or absence of the finger image is checked (step D14). Here, if there is no finger image, the process proceeds to step D18 in FIG. 20, but if the operator's finger is reflected in the captured image, the number of fingers standing is identified by image analysis (step D15). Then, the finger command table 15 is searched based on this “operator ID” and “index” (step D16), and it is checked whether or not the corresponding index corresponding to this operator is registered in the finger command table 15 (step D16). D17) If there is no match, the process proceeds to step D18 in FIG. 20. If there is a match, the command registered from the finger command table 15 at the intersection of the “operator ID” and “index” is stored. Is read (step D11), and the process instructed by this command is executed (step D12).

  Next, attention is paid to the movement direction of the operator's face. First, in step D18 of FIG. 20, a series of operator images (moving images) captured by the camera photographing unit 7 is acquired. Then, the presence / absence of movement of the face is checked by analyzing the series of captured moving images (step D19). If not, the process proceeds to step D23. If the face is moved, the moving direction is determined by image analysis. Is detected (step D20). Then, the face movement command table 14 is searched based on this “operator ID” and “movement direction” (step D21), and it is checked whether or not the corresponding movement direction is registered in the command table 14 corresponding to this operator. (Step D22) If there is no match, the process moves to Step D23, but if there is a match, it is registered from the face movement command table 14 at the intersection of the “operator ID” and “movement direction”. The command is read (step D11 in FIG. 19), and the process instructed by this command is executed (step D12).

  Next, in step D23, the presence or absence of face blocking is checked by analyzing a series of captured moving images. That is, it is checked whether or not the camera part is closed with a palm or a finger, and when face blocking is detected, an operation for measuring the image blocking time is started (step D24). Then, until the next appearance of the face (until the end of blocking is detected), the operation of counting the image blocking time is continued (steps D24 and D25). Here, when the end of blocking is detected, a face portion that appears at the end of blocking is extracted from a series of captured moving images (step D26), and this extracted image is compared with the face image immediately before blocking (step D27). Then, it is checked whether or not they match characteristically (whether they are the same person) (step D28). Here, on the condition that they are the same person, the blocking time command table 17 is searched based on this “operator ID” and “image blocking time”, and the corresponding image blocking time corresponding to this operator is displayed in the command table 17. Is registered (step D29), and if applicable, the command registered at the intersection between the “operator ID” and the “image cutoff time” is read from the cutoff time command table 17 ( In FIG. 11, step D11), the processing instructed by this command is executed (step D12).

  As described above, also in the second embodiment, as in the first embodiment described above, the face movement command table 14, the finger command table 15, and the relative relationship command table are associated with arbitrary actions and commands for each operator. 16, it is possible to execute a desired process with an action that suits the operator's preference simply by registering it in the cutoff time command table 17. In this case, different commands can be executed for each operator even in the same action, and conversely, the same command can be executed for each operator even in different actions, and it is possible to expect a significant improvement in usability.

In this case, paying attention to the face portion in the operator image, the moving direction of the operator is specified by analyzing the image portion in time series, and the moving direction of the operator is registered in the face moving command table 14. Since it is recognized whether or not the contents are correct, the operator can move up / down and left / right as actions of the operator. Is possible.
Further, paying attention to the hand portion in the operator image, the number of fingers stretched is identified by analyzing the image portion, and whether the number of fingers is in accordance with the contents registered in the finger command table 15 Since it is made to recognize whether or not, it is possible to execute a desired process by a simple action of extending a finger.

Paying attention to the face part and the hand part in the operator image, the relative positional relationship between the face and the hand is analyzed, and whether or not the relative position is the same as the contents registered in the relative command table 16. Therefore, the position of the hand that moves freely and the relative positional relationship with the face can be used as the action of the operator, and a flexible response according to the preference of the operator is possible.
Further, paying attention to the face portion in the operator image, the intentional image blocking time for blocking the camera portion with a hand or the like is counted, and whether this time is as described in the content registered in the blocking time command table 17. Since it is made to recognize whether or not, it is possible to instruct a desired process by a simple action.

 In the second embodiment described above, the up / down / left / right directions are exemplified as the movement direction of the operator's face. However, in addition to the four movement types, a total of eight types are included if including a type that flows in the up / down / left / right diagonal directions. It can be. Further, the movement of the face in the forward / backward direction, that is, the movement away / approaching from the camera may be included. Furthermore, the movement of the upper body is not limited to the face.

In the second embodiment described above, the number of fingers extended is detected, but the direction in which the fingers are extended may also be considered. Furthermore, you may make it include not only one hand but both hands.
In the second embodiment described above, the intentional image blocking time in which the camera portion is blocked by a hand or the like is detected, but the number of appearances of the face or the appearance frequency is detected. Also good.

  In addition, in the first and second embodiments described above, a disk type / notebook type personal computer equipped with a video camera is exemplified as the information processing apparatus, but a mobile phone, a PDA, or the like may be used. In addition to a stand-alone type information processing device, each component is physically separated into two or more cases, and a wired transmission path such as a communication line or a cable, or a wireless transmission path such as radio waves, microwaves, infrared rays, etc. It may be a distributed system that transmits and receives data through the network.

  On the other hand, a recording medium (for example, a CD-ROM, a RAM card, etc.) on which program codes for executing the above-described units are recorded may be provided to the computer. That is, a recording medium having a computer-readable program code for storing and managing behavior information that is arbitrarily registered in advance for each operator and instructing execution of specific processing in association with this behavior information A function for storing and managing the instruction command, a function for analyzing the action of the operator based on an operator image obtained by photographing the operator, and comparing the result of the image analysis with the action information corresponding to the operator. A function for recognizing whether or not the action is in accordance with the registered content, and when it is recognized that the action has been performed in accordance with the registered content, the instruction command associated with this action information is read and specified processing And a computer-readable recording medium storing a program for realizing the function for executing Unishi may be.

The block diagram which showed the basic component of information processing apparatus. The figure which showed the structure of the face table. The figure which showed the structure of the gesture operation | movement table 12. FIG. The figure which showed the structure of the gesture command table 13. FIG. The figure for demonstrating the gesture type classified according to the motion change of the fist as a gesture. The figure which illustrated a plurality of kinds of gestures classified according to the change in movement of the fist as gestures. The flowchart which showed the operation | movement at the time of gesture registration. The flowchart which showed the operation | movement at the time of power activation. The figure which illustrated the movement direction of the operator in 2nd Example. The figure which illustrated the movement direction of an operator like FIG. The figure which illustrated the number which has raised a finger in the 2nd example. The figure which illustrated the relative positional relationship of a face and a fist in the 2nd example. The figure which showed the case where the intentional image interruption which has blocked the camera part with a palm, a finger, etc. is performed in 2nd Example. The figure which showed the structure of the face movement command table 14 in 2nd Example. The figure which showed the structure of the finger command table 15 in 2nd Example. The figure which showed the structure of the relative relationship command table 16 in 2nd Example. The figure which showed the structure of the interruption | blocking time command table 17 in 2nd Example. The flowchart which showed the operation | movement in the case of registering arbitrary commands for each operator in each command table 14-17 in 2nd Example. 9 is a flowchart showing an operation when a camera instruction button is operated in the second embodiment. The flowchart which showed the operation | movement following FIG.

Explanation of symbols

1 CPU
DESCRIPTION OF SYMBOLS 2 Memory | storage part 4 Storage medium 5 Operation part 6 Display part 7 Camera imaging | photography part 11 Face table 12 Gesture action table 13 Gesture command table 14 Face movement command table 15 Finger command table 16 Relative relation command table 17 Shut-off time command table

Claims (10)

An information processing apparatus having a photographing function for photographing an operator,
Registered information storage means for storing and managing action information arbitrarily registered in advance for each operator, and storing and managing an instruction command for instructing execution of a specific process in association with the action information;
Image analyzing means for analyzing the behavior of the operator based on an operator image obtained by photographing the operator;
Recognizing means for recognizing whether or not the action of the operator is as registered, by comparing the analysis result obtained by the image analysis means and the action information corresponding to the operator;
A process execution means for reading out the instruction command associated with the action information and executing a specific process when it is recognized by the recognition means that the action according to the registered content has been performed;
An information processing apparatus comprising: When registering the action information for each operator, registering an operator image including a part including a characteristic action as action information by analyzing an operator image obtained by photographing the action of the operator.
2. The information processing apparatus according to claim 1, wherein the information processing apparatus is configured as described above. The registered information storage means stores and manages a plurality of types of behavior information arbitrarily registered for each operator, and assigns different types of instruction commands arbitrarily registered in association with the plurality of types of behavior information for each behavior information. Memory management,
When the recognition means recognizes that the action of the operator corresponds to any action information, the process execution means reads the instruction command associated with the action information and executes a specific process. ,
2. The information processing apparatus according to claim 1, wherein the information processing apparatus is configured as described above. The analysis means analyzes the operator's action based on the operator image taken at power-on,
The process execution means, when it is recognized that the action of the operator is as registered, directs the transition to the specific process according to the instruction command associated with the action information immediately after power-on,
2. The information processing apparatus according to claim 1, wherein the information processing apparatus is configured as described above. The image analysis means pays attention to a hand part in an operator image, specifies an operation change of the operator's hand by analyzing the image part in time series,
The recognizing means recognizes whether or not the hand movement obtained by the image analyzing means is as registered.
2. The information processing apparatus according to claim 1, wherein the information processing apparatus is configured as described above. The image analysis means pays attention to the face portion in the operator image, identifies the moving direction of the operator by analyzing the image portion in time series,
The recognizing means recognizes whether or not the moving direction of the operator obtained by the image analyzing means is as registered.
2. The information processing apparatus according to claim 1, wherein the information processing apparatus is configured as described above. The image analysis means focuses on a part of the hand in the operator image, specifies the number of fingers that are stretched by analyzing the image part,
The recognizing means recognizes whether or not the number of fingers obtained by the image analyzing means is as registered.
2. The information processing apparatus according to claim 1, wherein the information processing apparatus is configured as described above. The image analysis means focuses on a face part and a hand part in an operator image, specifies a relative positional relationship between the face and the hand,
The recognizing means recognizes whether or not the relative positional relationship obtained by the image analyzing means is as registered.
2. The information processing apparatus according to claim 1, wherein the information processing apparatus is configured as described above. The image analysis means focuses on the face portion in the operator image and identifies the appearance state of the face,
The recognizing means recognizes whether or not the appearance state of the face obtained by the image analyzing means is as registered.
2. The information processing apparatus according to claim 1, wherein the information processing apparatus is configured as described above. Against the computer,
A function for storing and managing action information arbitrarily registered in advance for each operator, and storing and managing an instruction command for instructing execution of a specific process in association with the action information;
A function of analyzing the behavior of the operator based on an operator image obtained by photographing the operator;
A function of recognizing whether or not the behavior of the operator is as registered, by comparing the result of the image analysis and the behavior information corresponding to the operator;
A function of reading the instruction command associated with the action information and executing a specific process when it is recognized that the action according to the registered content has been performed;
A program to realize

Images