JP2002116859A - Information processor, instruction recognition processing method and program storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an input interface whose operation is easy and whose operability is excellent for a user. SOLUTION: At the time of recognizing the change of the luminance level of picture data in instruction recognition areas 111-114 in an input picture obtained by image picking up the hand of a user by a CCD camera 8, prescribed scroll operations corresponding to commands preliminarily made correspond to the instruction recognition areas 111-114 are executed so that the scroll operations corresponding to the commands made correspond to the instruction recognition areas 111-114 can be immediately executed only by recognizing the movement of the hand of the user regardless of what kind of movement of the hand of the user in the instruction recognition areas 111-114. Thus, it is possible to provide the input interface whose responsiveness is high without any erroneous recognition.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus, an instruction recognition processing method, and a program storage medium, and is suitably applied to, for example, a notebook personal computer (hereinafter referred to as a notebook personal computer).

[0002]

2. Description of the Related Art Conventionally, a notebook personal computer comprises a main body, display means such as a liquid crystal display, and input means such as a keyboard or mouse for inputting predetermined commands and characters. A predetermined process is executed in response to the command, and the execution result is displayed on a display unit.

[0003]

In a notebook personal computer having such a configuration, an input means such as a keyboard or a mouse must be directly operated by an operation method for each input means. There is a problem in that it is necessary to memorize each operation method and a complicated operation is required.

[0004] In addition, if the input operation by the user is incorrect, the notebook personal computer performs an incorrect process, and is not easy to use.

The present invention has been made in view of the above points, and proposes an information processing apparatus, an instruction recognition processing method, and a program storage medium which can provide a user with an easy-to-operate and easy-to-use input interface. What you want to do.

[0006]

According to the present invention, a predetermined area in an image obtained by imaging an object to be recognized by an imaging means is specified, and a change in image data in the predetermined area is specified. When recognizing
By performing a predetermined process according to a command previously associated with the predetermined region, regardless of what kind of change in the image data in the predetermined region desired to be processed, Since the processing corresponding to the command associated with the predetermined area can be immediately executed just by recognizing the change, it is possible to provide a highly responsive input interface without erroneous recognition.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the drawings.

(1) External Configuration of Notebook Personal Computer In FIG. 1, reference numeral 1 denotes a notebook personal computer (hereinafter referred to as a notebook personal computer) as an information processing apparatus to which the present invention is applied. Body 2
And a display unit 3 attached to the main body 2 so as to be openable and closable.

The main body 2 has a plurality of operation keys 4 for inputting various characters, symbols, numbers, and the like on a top surface thereof, and a stick type pointing device (hereinafter, simply referred to as a stick) 5 used for moving a mouse cursor. Left click buttons 5A and 5B corresponding to a left button and a right button of a normal mouse, a center button 5C for operating a scroll bar without moving the mouse cursor to the scroll buttons, built-in speakers 6A and 6B,
Push-type power switch 7, CCD provided on display unit 3
(Charge Coupled Device) A shutter button 9 for a camera 8, a power lamp PL, a battery lamp BL, and a message lamp M each composed of an LED (Light Emitting Diode).
L and the like are provided.

The display section 3 has a TFT (Thin Fil) corresponding to, for example, an 8.9 type (1024 × 480 pixels) on its front surface.
m Transisitor) Liquid crystal display 1 consisting of color liquid crystal
0 is provided, and an image pickup section 11 provided with a CCD camera 8 as an image pickup means is provided at the upper end of the front center so as to be rotatable with respect to the display section 3.

In the image pickup section 11, the CCD camera 8 can be rotated at an angle of about 180 degrees from the front direction to the back direction of the display section 3 and positioned at an arbitrary angle. Focus adjustment when imaging a desired imaging target can be easily performed by rotating the adjustment ring 12 provided at the upper end of the imaging unit 11.

The display unit 3 is provided with microphones 13 on the front side and the back side near the left end of the image pickup unit 11, and covers a wide area from the front side to the back side of the display unit 3 via the microphone 13. It is designed to be able to collect sound across.

The display unit 3 further includes a liquid crystal display 10
The pawls 14 and 15 are provided near the left end and the right end, respectively, and holes 16 and 17 are provided at predetermined positions of the main body 2 corresponding to the pawls 14 and 15, respectively. In this state, the claws 14 and 15 are fitted into the corresponding holes 16 and 17, respectively.

On the other hand, when the front side of the display unit 3 closed by the main body 2 is lifted, the fitted state of the holes 16 and 17 and the claws 14 and 15 is released, and As a result, the display unit 3 can be developed from the main body 2.

The body 2 has an IrDA (In) on its right side.
frared Data Association) compliant infrared port 18,
Headphone terminal 19, microphone input terminal 2
0, USB (Universal Serial Bus) terminal 21, external power supply connector 22, external display output connector 2
3. A jog dial 24 and a modem terminal 25 for a modular jack are provided for inputting a command for executing a predetermined process by a rotation operation and a pressing operation of the rotary operation element.

On the other hand, as shown in FIG. 2, the main body 2 has an exhaust hole 26 and a PCMCIA (Personal Computer) on its left side.
Memory Card International Association) standard PC
(Personal Computer) card compatible with PC card slot 27 and 4-pin IEEE (Institute of E)
Electrical and Electronics Engineers) 1394 terminal 28 is provided.

Further, as shown in FIG. 3, the main body 2 is provided with a battery connector 29 on a rear side surface thereof, and a slide type removal lever 31 for detaching the battery pack 30 (FIG. 1) on the bottom surface, and the slide type removal lever 31. Lock lever 32 for locking the sliding operation of lever 31
And a reset switch 33 for interrupting the operation of the main body 2 and reconstructing the environment at power-on. The battery pack 30 is detachably connected to the battery connector 29.

(2) Circuit Configuration of Notebook Type Personal Computer Next, the circuit configuration of the notebook personal computer 1 will be described in detail with reference to FIG. In the main body 2 of the notebook personal computer 1, a CPU for controlling various functions of the main body 2 in an integrated manner
(Central Processing Unit) 50 is connected to the host bus 52, and the CPU 50 controls the RAM (Rand Rand).
om Access Memory) 53 is executed at a predetermined operating speed based on a system clock provided from the clock generator 60, thereby realizing various functions. I have.

A cache memory 51 is connected to the host bus 52 so as to cache data used by the CPU 50 and realize high-speed access.

The host bus 52 is connected to a PCI (Peripher
al Component Interconnect) bus 55 and host-PC
The PCI bus 55 is connected via a video controller 56 and an IEEE 1349
Interface 57, video capture processing chip 8
3 and the PC card interface 58 are connected.

Here, the host-PCI bridge 54
It controls the exchange of various data between the PU 50 and the video controller 56, the video capture processing chip 83, the IEEE 1349 interface 57, and the PC card interface 58, and controls the memory bus 5
9 for controlling the memory of the RAM 53 connected thereto.

The host-PCI bridge 54 includes a video controller 56 and an AGP (Accelerated Graphics P
(ort), so that image data can be transferred between the host-PCI bridge 54 and the video controller 56 at a high speed.

The video capture processing chip 83 includes an I ² C bus 82 (generally an SM (System
Management), and when image data captured by the CCD camera 8 is supplied via the I ² C bus 82, the image data is stored in a built-in frame memory (not shown). And store it in JPEG (Joint
Photographic Experts Group) After generating JPEG image data by performing image compression processing according to the standard,
The JPEG image data is stored in the frame memory again.

Then, the video capture processing chip 83
Transmits the JPEG image data stored in the frame memory to the RAM 53 using the bus master function in response to a request from the CPU 50, and then transfers the JPEG image (still image) data or the Motion JPEG image (moving image) data to the hard disk drive ( HDD) 67.

The video controller 56 outputs image data based on various kinds of application software supplied at appropriate times and image data picked up by the CCD camera 8 to the liquid crystal display 10 of the display unit 3,
A plurality of window screens can be displayed.

IEEE 1349 interface 57
Is directly connected to the IEEE 1394 terminal 28, and can be connected to another computer device or an external device such as a digital video camera via the IEEE 1394 terminal 28.

When an optional function is added, the PC card interface 58
7 is connected to a PC card (not shown) mounted on the PC 7 and, for example, a CD-ROM (Compact Dimm
sc-Read Only Memory) drive and DVD (Digital Ve
(rsatile Disc) drive so that it can be connected to an external device.

The PCI bus 55 is an ISA (Industrial S)
and a PCI-ISA bridge 66 and a PCI-ISA bridge 66.
The HDD 67 and the USB terminal 21 are connected to the bridge 66.

Here, the PCI-ISA bridge 66
DE (Integrated Drive Electronics) interface, configuration register, RTC (Real-T
imeClock) circuit, a USB interface, and the like, and controls the HDD 67 via the IDE interface based on a system clock supplied from the clock generator 60.

The hard disk of the HDD 67 has Windo
OS (Operating System) such as ws98 (trademark), e-mail program, auto pilot program, jog dial server program, jog dial driver,
Capture software, digital map software, and various other application software are stored, and are transferred to and loaded on the RAM 53 as needed in the course of startup processing.

The PCI-ISA bridge 66 is a U.S.A.
External devices such as a floppy (registered trademark) disk drive, a printer, and a USB mouse (not shown) connected via the B terminal 21 are controlled via a USB interface, and a modem 69 and a sound controller 70 connected to the ISA bus 65. Control.

The modem 69 is connected to an Internet service provider (hereinafter, referred to as a provider) from a modem terminal 25 via a public telephone line (not shown), and is connected to the Internet via the provider through a dial-up IP connection. ing.

The sound controller 70 generates audio data by digitally converting the audio signal collected by the microphone 13, outputs the audio data to the CPU 50, and converts the audio data supplied from the CPU 50 into an analog signal. , And outputs it to the outside via the built-in speaker 6.

The ISA bus 65 has an I / O (In / Ou).
t) The controller 73 is connected, receives power from an external power supply from the external power supply connector 22 via the power supply charging control circuit 85, and supplies power to each circuit when the power switch 7 is turned on. Here, the I / O controller 73 operates based on the system clock supplied from the clock generator 60 also.

The power supply charge control circuit 85 is provided with an I / O
The controller 73 controls charging of the battery pack 30 connected to the battery connector 29 (FIG. 3).

The I / O controller 73 includes a microcontroller, an I / O interface, a CPU, a ROM,
The flash memory 7 comprises a RAM or the like.
9 (Basic Input / Output Sys.)
tem) to control the input and output of data between the OS and application software and various peripheral devices such as the liquid crystal display 10 and the HDD 67.

The I / O controller 73 is connected to the infrared port 18 and can execute infrared communication with, for example, another computer.

Further, the I / O controller 73 is connected to an inversion switch 77, and when the imaging unit 11 is rotated by 180 degrees toward the back side of the liquid crystal display 10, the inversion switch 77 is turned on. -ISA
The CPU 50 is notified via the bridge 66 and the host-PCI bridge 54.

In addition, the I / O controller 73
It is connected to the full-press / half-press switch 78 and the main body 2
When the shutter button 9 provided on the upper surface of the camera is half-pressed, the full-press / half-press switch 78 is turned on to the half-pressed state, the CPU 50 is notified of this, and the shutter button 9 is fully pressed. The full-press / half-press switch 78 is turned on to the full-press state, and the CPU 5
Notify 0.

That is, the CPU 50 transfers the capture software from the hard disk of the HDD 67 to the RAM 53.
When the shutter button 9 is half-pressed by the user in a state where the camera is raised up, the still image mode is entered and the CC
The D camera 8 is controlled to freeze the still image. When the D camera 8 is fully pressed, the frozen still image data is captured and sent to the video controller 56.

On the other hand, when the shutter button 9 is fully depressed by the user without starting the capture software, the CPU 50 enters the moving image mode, captures a moving image for up to about 60 seconds, and outputs the moving image. The data is sent to the controller 56.

By the way, the RO of the I / O controller 73
M stores a wakeup program, a key input monitoring program, an LED control program, a jog dial state monitoring program, and various other control programs.

Here, the jog dial state monitoring program is a program used in conjunction with a jog dial server program stored in the hard disk of the HDD 67, and is for monitoring whether the jog dial 24 is rotated or pressed. It is.

The wake-up program is a PCI-I
When the current time supplied from the RTC circuit in the SA bridge 66 coincides with a preset start time, the CPU 50 is a program controlled to execute a predetermined process. Is a program for monitoring inputs from various key switches. The LED control program includes a power lamp PL,
This is a program for controlling lighting of various lamps such as a battery lamp BL and a message lamp ML (FIG. 1).

In the RAM of the I / O controller 73, an I / O register for a jog dial state monitoring program, a set time register for a wakeup program, a key input monitoring register for a key input monitoring program, and an LED control program An LED control register and other registers for various programs are provided.

The set time register stores the time information of the start time arbitrarily set by the user in advance for use in the wake-up program. Accordingly, the I / O controller 73 determines whether or not the current time supplied from the RTC circuit matches the arbitrarily set start time based on the wake-up program. Notify.

Thus, the CPU 50 starts up predetermined application software set in advance at the start time, and executes predetermined processing according to the application software.

The key input monitoring register stores operation key flags corresponding to input operations of the operation key 4, the stick 5, the left click button 5A, the right click button 5B, the center button 5C and the like.

Therefore, the I / O controller 73 operates based on the key input monitoring program to determine whether or not the pointing operation with the stick 5 or the clicking operation of the left click button 5A, right click button 5B and center button 5C has been performed. The determination is made based on the state of the key flag, and when a pointing operation or a click operation is performed, the fact is notified to the CPU 50.

Here, the pointing operation is an operation of moving the mouse cursor to a desired position on the screen by pressing the stick 5 up, down, left and right with a finger, and the clicking operation is the left click button 5A or the right click button. This is an operation to quickly press and release the click button 5B with a finger.

Thus, the CPU 50 executes a predetermined process in accordance with the movement of the mouse cursor and the click operation by the pointing operation.

The LED control register includes a power lamp P
A lighting flag indicating the lighting state of various lamps such as L, battery lamp BL, and message lamp ML is stored.

Therefore, the I / O controller 73 causes the CPU 50 to operate the HD
The e-mail program is started from the hard disk at D67, and when an e-mail is received according to the e-mail program, a lighting flag is stored, and the message lamp ML is turned on by controlling the LED 81 based on the lighting flag.

The I / O register for the jog dial state monitoring program stores a rotation operation flag and a pressing operation flag corresponding to a rotation operation and a pressing operation on the jog dial 24.

Therefore, when a user's desired menu item is selected from a plurality of menu items by rotating and pressing the jog dial 24 connected via the rotation detecting unit 88, the I / O controller 73 outputs the I / O controller 73. The rotation operation flag and the pressing operation flag stored in the O register are set, and the CPU 50 is notified to that effect.

In accordance with the jog dial server program read from the HDD 67 and started on the RAM 53, the CPU 50 executes a predetermined process corresponding to the menu item determined by the rotation operation and the press operation of the jog dial 24, and executes the menu item. Start the application software corresponding to.

Here, the I / O controller 73 operates constantly under the control of the power supply / charge control circuit 85 even when the power switch 7 is off and the OS is not running. The user can start application software or a script file desired by the user by pressing the jog dial 24 in the power saving state or when the power is off.

[0058] Note that the I / O controller 73 is also connected to the ^{I 2} C bus 82, supplies the various setting parameters for the CCD camera 8 set by operating keys 4 or the jog dial 24 through the ^{I 2} C bus 82 Thus, the brightness and contrast of the CCD camera 8 are adjusted.

(3) Command Recognition Process In addition to the above configuration, the notebook computer 1 includes an application program called a cyber gesture program for recognizing a predetermined command corresponding to the movement of the user's hand captured by the CCD camera 8. From the hard disk, recognizes the movement of the user's hand captured by the CCD camera 8 based on the cyber gesture program, and executes a predetermined process according to the recognition result on a window screen based on the application program. Has been made.

For example, as shown in FIG. 5, the notebook computer 1 has scroll bars 101 and 102 at the right end and the lower end of the window screen 99 displayed on the liquid crystal display 10 based on a predetermined application program.
Is operated with the up key, the down key, the left key, and the right key of the operation key 4 to scroll the display image 99A of the window screen 99 up, down, left, and right. In the above, the CPU 50 recognizes a command corresponding to the movement of the hand of the user picked up by the CCD camera 8 based on the cyber gesture program, so that the above-described scrolling operation can be performed without touching the operation keys 4 under the control of the CPU 50. To be able to run on.

That is, the CPU 50 of the notebook computer 1
As shown in FIG. 6, a command corresponding to the movement of the user's hand is recognized according to a command recognition processing procedure based on the cyber gesture program of the routine RT1, and the process first proceeds to step SP1.

In step SP 1, the CPU 50 starts a cyber gesture program from the hard disk of the HDD 67 in response to a user operation, and superimposes the command recognition screen 100 generated based on the cyber gesture program on the display image 99 A in the window screen 99. Then, the process proceeds to the next step SP2.

Here, the instruction recognition screen 100 is 164 ×
It is configured with a screen size of 136 pixels (pixels), and is formed with a screen size extremely smaller than the screen size of the liquid crystal display 10 (1024 × 480 pixels). The hidden area of the instruction recognition screen 100 with respect to the window screen 99 is made as small as possible so that the displayed image 99A is not hidden.

As shown in FIG. 7, the command recognition screen 100 has "CYBERGESTURE" at the upper end of the command recognition screen 100 indicating that it is a cyber gesture program.
(Sony Corporation trademark) title character portion 103, option button 104 for performing detailed setting of functions, help button 105, minimize button 106, and close button 1
07 is provided.

Further, the command recognition screen 100 is provided with command recognition areas 111 to 114, which are rectangular frames for recognizing four types of commands, in the image display area 110. A command for scrolling up the display image 99A on the screen 99 is recognized, a command for scrolling down the display image 99A in the command recognition area 112 is recognized, and the display image 99A is moved left in the command recognition area 113. The command for scrolling in the direction is recognized, and the command for scrolling the display image 99A rightward in the command recognition area 114 is recognized.

In step SP2, the CPU 50 takes an image of the user located in front of the display unit 3 (FIG. 1) with the CCD camera 8 of the imaging unit 11, and recognizes the input image obtained as a result of the movement of the user's hand. And input the input image into the image display area 110 of the instruction recognition screen 100.
And moves to the next step SP3.

In step SP3, the CPU 50 applies a mask image 116 of a mask pattern as shown in FIG. 8 corresponding to the command recognition areas 111 to 114 of the image display area 110 to the input image fetched from the CCD camera 8 in step SP2. To perform a mask process, and then proceeds to the next subroutine SRT2.

The mask image 116 is displayed in the image display area 1
Corresponding to the instruction recognition areas 111 to 114 of FIG. 7 (FIG. 7), the shield area 116A and the non-shield areas 116B to 116E are left so that only the instruction recognition areas 111 to 114 are left.
Has a mask pattern formed thereon.

Therefore, the CPU 50 can obtain image data relating to the movement of the hand of the user only from the command recognition areas 111 to 114 of the input image by superimposing the mask image 116 on the input image and synthesizing the mask image. . Incidentally, the CPU 50 displays the input image on which the mask image 116 is superimposed in the image display area 110 of the command recognition screen 100, although the input image on which the mask image 116 is superimposed is used for hand movement recognition. Instead, the input image that has not been subjected to the mask processing is displayed in the image display area 110 of the command recognition screen 100.

As shown in FIG. 9, the subroutine SRT2
In step SP11, the CPU 50 divides the fetched input image into a plurality of 16 × 16 pixel macroblocks, and proceeds to the next step SP12.

At step SP12, the CPU 50
A motion vector for each macroblock is calculated based on a change in the luminance level between the current frame and the previous frame in each macroblock, and the flow advances to next step SP13.

At step SP13, the CPU 50
A plurality of macroblocks having motion vectors in the same direction are collected, the center point of the motion area is calculated, and the routine goes to the next step SP14.

In this case, the CPU 50 sets the mask image 116
Since the input image is subjected to the mask processing, macroblocks to be subjected to data processing can be obtained only from the instruction recognition areas 111 to 114 in the input image, so that the necessary amount of data processing is minimized. I have.

At step SP14, the CPU 50
It is determined whether or not the motion area calculated in step SP13 exists in the input image. If a negative result is obtained here, this means that a motion area which is a set of macroblocks having motion vectors in the same direction does not exist in the input image. At this time, the CPU 50 proceeds to step SP16. Then, the processing after step SP11 in the next frame is repeated.

On the other hand, if an affirmative result is obtained in step SP14, this means that a motion area which is a set of macroblocks having motion vectors in the same direction exists in the input image. At this time, the CPU 50
Moves to the next step SP15.

At step SP15, the CPU 50
It is determined whether or not the motion area is equal to or larger than a predetermined size.
If a negative result is obtained here, it indicates that the motion area is not equal to or larger than the predetermined size, that is, it cannot be determined that the movement area is the user's hand.
Then, the process proceeds to step 7 and the processes after step SP11 in the next frame are repeated.

On the other hand, if an affirmative result is obtained in step SP15, this indicates that the motion area can be determined to be equal to or larger than a predetermined size, that is, the user's hand. As shown in FIG. 10, it can be recognized that the user's hand is held over the instruction recognition area 113, and thereafter, the process returns to the routine RT1 again and proceeds to the next step SP4.

In step SP4, the CPU 50 issues a command (in this case, an instruction to scroll the display image 99A of the window screen 99 to the left) in accordance with the command recognition area 113 held by the user based on the cyber gesture program. API (Application Progra
The scroll operation of the display image 99A according to the command is performed by supplying the jog dial server program to the application program via the mming interface), and the process proceeds to the next step SP5.

Here, the API is a program interface open to the application program by the OS, and the application program basically performs all processes via the API. Incidentally, the API of a general OS at present is in the form of a function, and an application program calls an API function by designating appropriate arguments (parameters).

In step SP5, the CPU 50 determines whether or not the center point of the hand movement area recognized in the instruction recognition area 113 has moved between the current frame and the previous frame. If a negative result is obtained here, this indicates that the user has not moved at all despite the hand being held, and at this time, the CPU 50 executes the subroutine SR
Returning to T2, the above processing is repeated.

On the other hand, if a positive result is obtained in step SP5, this means that the user's hand recognized in the instruction recognition area 113 has been moved again in the instruction recognition area 113. At this time, the CPU 50 returns to step SP4 again and repeats the above-described processing, thereby supplying a command to the application program again according to the movement of the user's hand, thereby continuing the scroll operation of the display image 99A corresponding to the command. Have been made to perform.

In this embodiment, the torso of the user always enters the command recognition area 112. However, unless the torso moves, the window screen 99 is displayed.
No command is supplied to the application program to scroll the displayed image 99A downward.

As described above, the CPU 50 of the notebook personal computer 1
Performs mask processing on an input image captured by the CCD camera 8 and assigns commands corresponding to the command recognition areas 111 to 114 to the command recognition areas 111 to 114.
When the user's hand is held over the command recognition area 4,
By executing the scroll operation of the display image 99A corresponding to the command on the window screen 99 based on the application program by recognizing the command corresponding to 1 to 114 and supplying the command to the application program via the jog dial server program. It is made to be able to do.

Thus, the user can handle a desired scrolling operation simply by holding his / her hand over any of the command recognition areas 111 to 114 via the CCD camera 8 of the imaging unit 11 without directly operating the operation keys 8. By giving the given command, the display image 99A of the window screen 99 can be easily moved in a desired direction.

(4) Operation and Effect In the above configuration, the CPU 50 of the notebook computer 1
Performs a mask process on the input image captured by the CCD camera 8 with a mask image 116, and outputs a command recognition area 111
When the user recognizes that the user's hand is held over any one of the command recognition areas 114 to 114, a command previously assigned to the instruction recognition area is supplied to the application program. A desired scrolling operation can be executed in a non-contact manner by merely holding the user's hand over one of the instruction recognition areas 111 to 114 without performing an operation.

The CPU 50 of the notebook personal computer 1 associates predetermined commands with the command recognition areas 111 to 114 in advance, so that it is not necessary to recognize how the user's hand has been moved. Area 1
Since the corresponding command can be supplied to the application program only by recognizing that the hand has been held over 11 to 114, there is no erroneous recognition as compared with the case of recognizing the gesture of the user's hand, and it is accurate and responsive. High scroll operation can be executed.

According to the above configuration, when the notebook computer 1 recognizes the movement of the user's hand in any of the command recognition areas 111 to 114 provided in the image display area 110 of the command recognition screen 100, By immediately supplying the command previously associated with the recognized command recognition area to the application program, it is possible to instantly recognize the user's command and accurately and reliably execute a scroll operation according to the command. Can be.

(5) Other Embodiments In the above-described embodiment, mask processing is performed using the mask image 116 so that the instruction recognition screen 100
Command recognition areas 11 at the top, bottom, left and right of the image display area 110
Although the case where 1 to 114 are provided has been described,
The present invention is not limited to this, and a mask image 171 having a mask pattern of a shielded area 171A and non-shielded areas 171B to 171F as shown in FIG. 11A, and a shielded area 172A and a masked area 172A as shown in FIG. By performing mask processing using a mask image 172 formed of a mask pattern of the non-shielded areas 172B to 171H surrounding the user, command recognition areas formed of various patterns may be provided.

When mask processing is performed using the disk image 172 as shown in FIG. 11B, FIG.
As shown in FIG. 0, it is possible to avoid a situation in which the user's torso always enters the command recognition area 112, and there is no possibility of recognizing the movement in the torso, so that there is no erroneous recognition and more accurate Can recognize the user's command.

In the above-described embodiment, a case has been described in which commands corresponding to scroll operations in up, down, left, and right directions are assigned to the command recognition areas 111 to 114 as predetermined areas. The present invention is not limited to this, and commands corresponding to other various operations may be assigned.

Further, in the above embodiment, the command recognition area 111 is recognized by the CPU 50 as recognition means.
Although the case where the user's hand is held over based on the change in the brightness level of the image data in .about.114 has been described, the present invention is not limited to this, and various other methods such as block matching are used. May recognize that the user's hand is held.

Further, in the above-described embodiment, by masking the input image using the mask image 116, image data necessary only for recognizing hand movement is obtained from only the command recognition areas 111 to 114. However, the present invention is not limited to this.
Mask image 11 by CPU 50 as an area specifying means
Command recognition area 1 from the input image without using
Only the image data of the area corresponding to 11 to 114 may be specified (designated) and acquired.

Further, in the above-described embodiment, the case where the CPU 50 as the execution means recognizes the user's hand based on the cyber gesture program stored in advance in the hard disk or ROM of the HDD 67 and executes the scroll operation is described. However, the present invention is not limited to this. By installing a program storage medium in which a cyber gesture program is stored in the notebook computer 1, it is possible to recognize the above-mentioned user's hand and execute the scroll operation. Is also good.

As described above, a program storage medium used to install the cyber gesture program for executing the above-described series of processes in the notebook personal computer 1 and make it executable in the notebook personal computer 1 includes, for example, a floppy disk, CD-ROM (Co
mpact Disc-Read Only Memory), DVD (Digital Vers
The present invention may be realized not only by package media such as an atile disc, but also by a semiconductor memory or a magnetic disk in which a cyber gesture program is temporarily or permanently stored. As a means for storing the cyber gesture program in these program storage media, a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting may be used, and various communication interfaces such as a router and a modem may be used. It may be stored.

Further, in the above-described embodiment, the case where the input image picked up by the CCD camera 8 in the image pickup unit 11 integrally formed with the display unit 3 is taken as the image pickup means has been described. However, the present invention is not limited to this, and an input image captured by various other image capturing means separate from the notebook computer 1 may be captured via the IEEE 1394 terminal 28 or the like.

Further, in the above-described embodiment, the case where the information processing apparatus of the present invention is applied to the notebook personal computer 1 has been described. However, a desktop personal computer or a PDA (Personal Digital Assis
tant) may be applied to various other information processing devices.

[0097]

As described above, according to the present invention, when a predetermined area in an image obtained by imaging an object to be recognized by an imaging means is specified, and a change in image data in the predetermined area is recognized, By performing a predetermined process according to a command previously associated with the predetermined region, regardless of what kind of change in the image data in the predetermined region desired to be processed,
Just by recognizing the change, it is possible to immediately execute a process corresponding to the command associated with the predetermined area, so that a highly responsive input interface without erroneous recognition can be provided. An information processing apparatus, a command recognition processing method, and a program storage medium that can provide an input interface that is easy to operate and easy to use can be realized.

[Brief description of the drawings]

FIG. 1 is a schematic perspective view showing an overall configuration of a notebook personal computer according to a first embodiment of the present invention.

FIG. 2 is a schematic diagram illustrating a configuration of a left side surface of a main body.

FIG. 3 is a schematic diagram illustrating a configuration of a rear side surface and a bottom surface of a main body.

FIG. 4 is a block diagram illustrating a circuit configuration of a notebook personal computer.

FIG. 5 is a schematic diagram illustrating an instruction recognition screen displayed over the window screen.

FIG. 6 is a flowchart illustrating a window control processing procedure based on motion detection in the instruction recognition area.

FIG. 7 is a schematic diagram illustrating a configuration of a command recognition screen.

FIG. 8 is a schematic diagram illustrating a mask image.

FIG. 9 is a flowchart illustrating an acquisition processing procedure regarding hand position information.

FIG. 10 is a schematic diagram illustrating a state when a hand is held over an instruction recognition area.

FIG. 11 is a schematic diagram illustrating types of mask images according to another embodiment.

[Explanation of symbols]

1 ... Notebook type personal computer, 2 ...
Main unit, 3 display unit, 4 operation keys, 8 CCD camera, 10 liquid crystal display, 11 imaging unit, 5
0: CPU, 53: RAM, 100: Command recognition screen, 110: Image display area, 111 to 114: Command recognition area, 116, 171, 172: Mask image.

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 7/20 G06T 7/20 A (72) Inventor Keigo Ihara 6-7-35 Kita Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (72) Inventor Takahiko Sueyoshi 6-35 Kita-Shinagawa, Shinagawa-ku, Tokyo F-term (reference) 5B057 CA08 CA12 CA16 CB08 CB12 CB16 CC01 CE09 DC22 5B087 AA09 CC33 DD03 DE02 5E501 AA03 AC15 BA05 CB14 EA14 EB01 FA14 FA45 FB21 FB32 FB43 5L096 AA09 BA18 CA02 CA14 DA05 EA35 HA02

Claims (12)

[Claims] 1. An area specifying means for specifying a predetermined area in an image obtained by imaging an object to be recognized by an imaging means; a recognition means for recognizing a change in image data in the predetermined area; An information processing apparatus comprising: an execution unit configured to execute a predetermined process according to a command associated with the predetermined area in advance when recognizing a change in the image data. 2. The apparatus according to claim 1, wherein said area specifying means superimposes a mask image formed of a shielding pattern corresponding to the predetermined area on the image so as to leave only the predetermined area. Information processing device. 3. The image processing apparatus according to claim 2, wherein the area specifying means superimposes the mask image formed of the shielding pattern such that the recognition target imaged by the image pickup means and the predetermined area do not overlap with each other. The information processing apparatus according to claim 2, wherein 4. The apparatus according to claim 1, wherein said recognizing means recognizes a luminance level difference between a current frame and a previous frame of the image data as a change in the image data.
An information processing apparatus according to claim 1. 5. An area specifying step for specifying a predetermined area in an image obtained by imaging an object to be recognized by an imaging unit; a recognition step of recognizing a change in image data in the predetermined area; An execution step of executing a predetermined process according to a command previously associated with the predetermined area when the change in the image data is recognized. 6. The area specifying step according to claim 5, wherein a mask image composed of a shielding pattern corresponding to the predetermined area is superimposed on the image so as to leave only the predetermined area. Instruction recognition processing method. 7. In the area specifying step, the mask image formed of the shielding pattern such that the recognition target imaged in the image capturing step does not overlap the predetermined area is superimposed on the image and synthesized. The instruction recognition processing method according to claim 6, wherein 8. The method according to claim 5, wherein in the recognizing step, a luminance level difference between a current frame and a previous frame of the image data is recognized as a change in the image data. 9. An area specifying step of specifying a predetermined area in an image obtained by imaging a recognition target by an imaging unit; a recognition step of recognizing a change in image data in the predetermined area; Executing a predetermined process according to a command associated with the predetermined area in advance when the change in the image data is recognized. A program storage medium to be executed. 10. The method according to claim 9, wherein in the area specifying step, a mask image formed of a shielding pattern corresponding to the predetermined area is superimposed on the image so as to leave only the predetermined area. Program storage medium. 11. The method according to claim 11, wherein in the area specifying step, the mask image formed of the shielding pattern such that the recognition target imaged in the imaging step does not overlap the predetermined area is superimposed on the image. Claim 1
0. The program storage medium according to 0. 12. The program storage medium according to claim 9, wherein, in the recognizing step, a luminance level difference between a current frame and a previous frame of the image data is recognized as a change in the image data.