JP2012243172A - Information input device and method for electronic equipment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine an input operation area in which an input operation is performable with the hand in accordance with the posture of the face, the position of the face and the movement of the face as the characteristics of the face, and to achieve the selection of an input face or the input of information to the selected input face in accordance with the posture and movement of the hand as the characteristics of the face and the characteristics of the hand.SOLUTION: The information input device includes: image pickup means; face characteristic determination means for extracting face information from the video of the image pickup means, and for determining the characteristic information of the face; a hand characteristic determination means for determining the characteristic information of the hand from the input operation area of the video of the image pickup means; indicator state determination means for determining the current indicator state on the basis of the face characteristic information and the hand characteristic information; operation state determination means for determining an operation state from an operation procedure consisting of the combination of the input operation area, the input face, the indicator state, a time when the indicator state continues, and the operation instruction; and operation procedure execution means for executing the operation instruction of the determination of the layout position of the input operation area or the selection of the input face or the generation of the input information according to the determined operation state.

Description

  The present invention relates to an information input apparatus and method for an electronic device. More specifically, an electronic device capable of inputting input information set by the operator according to the movement of the face, the appearance of the hand in the input operation area set in advance with respect to the face position, and the movement of the hand. The present invention relates to a device information input apparatus and method.

  As an input device for user-friendly electronic equipment that can set the operation procedure and the appearance of the finger to perform input operations so that it is easy for the operator to input, an input device that uses video captured by the camera as input information is proposed Has been.

  For example, when this input device is applied to a personal computer, the input surface can be selected and switched according to the finger shape and movement that can be arbitrarily set, and this affects the other input surfaces with respect to the selected input surface. The applicant of the present application has already filed an information input device that can input information equivalent to characters, equivalent to a mouse, equivalent to a numeric keypad, equivalent to shortcut keys, and the like.

  Although it can be said that the user-friendly information input device can be provided by the input device previously filed by the applicant of the present application, by subsequent research and development, the position coordinate of the face is first obtained and then the input operation area of the hand is determined, The applicant of the present application has found that the input operation can be further diversified by performing the input operation with both the face and the hand operating in the input operation area.

  The present invention determines an input operation area in which an input operation can be performed by a hand that functions as a secondary indicator based on the position of the face that functions as the main indicator of the operator who performs the input operation in an input operation to an electronic device. It is an object of the present invention to provide an input device using at least two indicators that can set input instructions in consideration of both face and hand states and movements and realize a variety of input operations. Is.

According to the invention, as claimed in claim 1,
Imaging means for sequentially capturing images of a predetermined area of the input operation surface;
The main indicator information is extracted from the image from the imaging means, the main indicator shape and the position coordinates of the main indicator are determined from the main indicator information, and the current position coordinates of the main indicator on the image Based on the previous position coordinates stored, the main indicator for determining whether or not the main indicator is moving is determined, and the input operation surface is determined from the current position coordinates of the main indicator. Main indicator characteristic determination means for calculating the two-dimensional coordinates of the current main indicator on the input surface formed at a predetermined position;
When an input operation area defined as a predetermined area of the input operation surface is determined by the main indicator shape of the main indicator characteristic determination means and the position coordinates of the main indicator, the input from the imaging means Subordinate indicator characteristic determining means for extracting the auxiliary indicator information around the main indicator from the video of the operation area, and determining the shape of the auxiliary indicator from the auxiliary indicator information;
The indicator state determining means for determining the indicator state based on the difference between the main indicator and the main indicator from the main indicator characteristic determining means and the following indicator shape from the secondary indicator characteristic determining means; ,
At least one or more operations comprising a combination of the input operation area, the input surface, the indicator state, and the operation instruction, which can be determined and changed in advance according to the indicator state determined by the indicator state determination means. An operation state determination means for determining an operation state from an operation state determination table associated with a procedure;
According to the operation state determined by the operation state determination means, the content of the operation instruction for determining the position of the input operation area, selecting the input surface, or generating the input information is the instruction for determining the position of the input operation area. If there is, confirm the arrangement position of the input operation area according to the instruction to confirm the arrangement position of the input operation area,
If the content of the operation instruction is an input surface selection instruction, change the currently selected input surface according to the input surface selection instruction,
If the content of the operation instruction is an instruction to generate input information, on the currently selected input surface,
Generating input information in response to an instruction to generate the input surface and the input information; or
An operation procedure execution means for generating input information from the input surface, an instruction for generating the input information, and the two-dimensional coordinates of the main indicator;
An information input device for an electronic device is provided.

According to the present invention, as described in claim 2, in the input device according to claim 1,
The subordinate indicator characteristic determining means determines an input operation area defined as a predetermined area on the input operation surface based on the main indicator shape of the main indicator characteristic determining means and the position coordinates of the main indicator. When the information on the input operation area is extracted from the image of the input operation area from the imaging means, the position of the subordinate indicator on the input operation area The coordinates are determined, and the difference status of the subordinate indicator for determining whether or not the subordinate indicator is moving is determined from the current position coordinates of the subordinate indicator and the stored previous position coordinates. And calculating the two-dimensional coordinates of the current subsidiary indicator on the input surface formed at a predetermined position on the input operation surface from the position coordinates of the current subsidiary indicator,
When the input operation area is fixed by the pointer state determination means, the difference between the main indicator shape and the main pointer from the main pointer characteristic determination means and the secondary pointer characteristic determination means The status of the indicator is determined based on the difference between the shape of the subordinate indicator and the subordinate indicator.
In the operation procedure execution means, when the content of the operation instruction is an instruction to generate input information, on the currently selected input surface,
Generating input information in response to an instruction to generate the input surface and the input information; or
Generate input information from the input surface, the input information generation instruction and the two-dimensional coordinates of the main indicator, or
Generating input information from the input surface and the instruction for generating the input information and the two-dimensional coordinates of the subordinate indicator;
An information input device for an electronic device is provided.

  According to the present invention, as described in claim 3, in the input device according to claim 1, the operation state determination table of the operation state determination means includes the input operation region, the input surface, and the indicator state. There is provided an information input device for an electronic device characterized in that at least one set of operation procedures including a combination of a time during which the indicator state continues and an operation instruction is associated.

According to the present invention, as described in claim 4, in the input device according to claim 1, the operation procedure performing means is:
An input operation region definition table that associates an input operation region that is predetermined and can be changed with respect to a positioning instruction of the input operation region and a position coordinate of a main indicator;
If the content of the operation instruction of the operation state determination means is confirmation of the arrangement position of the input operation area, the arrangement position of the input operation area is confirmed from the input operation area definition table,
An input information table in which a plurality of input information that is predetermined and changeable with respect to the input surface, the input information generation instruction, and the two-dimensional coordinates of the indicator are associated;
If the content of the operation instruction of the operation state determination means is an instruction to generate input information, on the currently selected input surface,
Generating input information determined by an operator in response to the input surface and the input information generation instruction; or
An information input device for an electronic device is provided, wherein input information is generated from the input information table from the input surface, an instruction for generating the input information, and the two-dimensional coordinates of the main indicator.

According to the invention, as claimed in claim 5,
An imaging process for sequentially capturing images of a predetermined area of the input operation surface;
The main indicator information is extracted from the video from the imaging process, the main indicator shape and the position coordinates of the main indicator are determined from the main indicator information, and the current position coordinates of the main indicator on the video Based on the previous position coordinates stored, the main indicator for determining whether or not the main indicator is moving is determined, and the input operation surface is determined from the current position coordinates of the main indicator. A main indicator characteristic determination process for calculating the two-dimensional coordinates of the current main indicator on the input surface formed at a predetermined position;
When an input operation area defined as a predetermined area on the input operation surface is determined by the main indicator shape and the position coordinates of the main indicator in the main indicator characteristic determination process, input from the imaging process Subordinate indicator characteristic determination process for extracting the subordinate indicator information around the main indicator from the operation region image, and determining the shape of the subordinate indicator from the subordinate indicator information;
The indicator state determination process for determining the indicator state based on the difference between the main indicator and the main indicator from the main indicator characteristic determination process and the subordinate indicator shape from the sub indicator characteristic determination process. ,
At least one or more operations comprising a combination of the input operation area, the input surface, the indicator state, and the operation instruction, which can be determined and changed in advance according to the indicator state determined by the indicator state determination process. An operation state determination process for determining an operation state from an operation state determination table associated with a procedure;
According to the operation state determined by the operation state determination process, the content of the operation instruction for determining the position of the input operation area or selecting the input surface or generating the input information is the instruction for determining the position of the input operation area. If there is, confirm the arrangement position of the input operation area according to the instruction to confirm the arrangement position of the input operation area,
If the content of the operation instruction is an input surface selection instruction, change the currently selected input surface according to the input surface selection instruction,
If the content of the operation instruction is an instruction to generate input information, on the currently selected input surface,
Generating input information in response to an instruction to generate the input surface and the input information; or
An operation procedure execution process for generating input information from the input surface, the input information generation instruction, and the two-dimensional coordinates of the main indicator;
An information input method for an electronic device is provided.

In the input device according to the first aspect of the present invention, based on the video imaged by the imaging means, the main indicator characteristic determining means determines the appearance and difference status of the main indicator, and the subordinate indicator characteristic determining means is the main. The appearance of the subordinate indicator is determined from the video in the input operation area arranged according to the position coordinates of the indicator.
Then, the indicator state determining means determines the indicator state based on the arranged input operation area from the main indicator and the difference indicator and the following indicator, and the operation state determining means The operation instruction according to the area, the input surface, and the indicator state is determined, and the operation procedure execution means determines the position of the input operation area according to the operation instruction, selects the input surface, and input information of the selected input surface. Therefore, the input operation can be set for each input operation area. Therefore, diversification of input operations can be achieved.

In the input device according to claim 2, the subordinate indicator characteristic determination means determines the difference status of the subordinate indicator in addition to the figure of the subordinate indicator, and the position coordinates of the current subordinate indicator Calculating the two-dimensional coordinates of the current subordinate indicator on the input surface formed at a predetermined position on the input operation surface;
When the input state is determined by the indicator state determining means, the difference between the main indicator shape from the main indicator characteristic determining means and the main indicator and the subordinate indicator characteristic determining means From the difference between the figure of the indicator and the following indicator, the indicator state is determined,
In the operation procedure execution means, when the content of the operation instruction is an instruction to generate input information, on the currently selected input surface,
Since the input information can be generated from the input surface and the input information generation instruction and the following two-dimensional coordinates of the indicator, the input operation can be diversified.

  According to another aspect of the input device of the present invention, the operation state determination table of the operation state determination means includes at least one set of combinations of the input operation region, the input surface, the indicator state, the duration of the indicator state, and the operation instruction. Since the operation procedures are associated with each other, the input operation can be performed in consideration of the time during which the indicator state continues, and the input operation can be diversified.

In the input device according to claim 4, the operation procedure execution means includes:
An input operation region definition table that associates an input operation region that is predetermined and changeable with respect to a position instruction of a main indicator and a determination instruction of an arrangement position of the input operation region;
If the content of the operation instruction of the operation state confirmation means is confirmation of the arrangement position of the input operation area, the arrangement position of the input operation area is confirmed from the input operation area definition table,
An input information table that associates a plurality of input information that is predetermined and changeable with respect to the input surface, the input information generation instruction, and the two-dimensional coordinates of the indicator;
If the content of the operation instruction of the operation state determination means is an instruction to generate input information, on the currently selected input surface,
Generate input information defined by the operator in response to the input surface and input information generation instruction, or
Since the input information is generated from the input information table from the input surface, the input information generation instruction, and the two-dimensional coordinates of the main indicator, the input operation procedure can be easily changed.

In the input method of claim 5 of the present invention,
Based on the video imaged in the imaging process, the main indicator's appearance and difference status are determined in the main indicator characteristic determination process, and the input arranged by the position coordinates of the main indicator in the subsequent indicator characteristic determination process The appearance of the subordinate indicator is determined from the image in the operation area.
Then, in the indicator state determination process, the indicator state based on the arranged input operation area is determined based on the main indicator, the difference status, and the subordinate indicator, and in the operation state determination process, the input operation The operation instruction according to the area, the input surface and the indicator state is confirmed, and according to the operation procedure, the arrangement position of the input operation area is determined according to the operation instruction, the input surface is selected, and the input information of the selected input surface Therefore, the input operation can be set for each input operation area. Therefore, diversification of input operations can be achieved.

FIG. 1 is an overall view of Embodiment 1 of an information input device according to the present invention.
FIG. 2 is a transition diagram of the input surface of the first embodiment of the information input device according to the present invention.
FIG. 3 is a flowchart of the first embodiment of the information input device according to the present invention.
FIG. 4 is an example of the indicator state table of the first embodiment of the information input device according to the present invention.
FIG. 5 is an example of the operation state determination table of the first embodiment of the information input device according to the present invention.
FIG. 6 is an overall view of Embodiment 2 of the information input device according to the present invention.
FIG. 7 is a transition diagram of the input surface of the embodiment 2 of the information input device according to the present invention.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Throughout the drawings, the same reference numerals or symbols denote the same or equivalent elements throughout the specification.

  First, a personal computer functioning as an input device of the invention (hereinafter referred to as a PC), in which a pointing operation or a scrolling operation of a personal computer or a function installed in the personal computer is activated in front of a display. Will be described with reference to FIGS.

First, the physical configuration will be described with reference to FIG.
Reference numeral 1 denotes a PC functioning as an input device, and a CCD (Charge Coupled Device) camera 3 (hereinafter referred to as a “coupled device”) that shoots an operator located in front of the display 2 during operation. An imaging device 9 having a camera 3) is provided and connected to the PC 1.

  The dotted line area 4a in FIG. 1 is demarcated at the operator's standing position. The input information for operating the PC 1, the information equivalent to the mouse of the PC 1, and the functions installed in the personal computer are derived from the image of the camera 3. This is an area for inputting input information to be activated. In FIG. 1, for convenience of explanation, an operator is positioned beside the display 2 and it appears that the imaging device 9 is photographing this. However, in reality, the imaging device 9 on the display 2 is connected to the display 2. The field of view is directed forward, and the operator has a positional relationship such that the operator stands almost in front of the display 2. At that time, in order to facilitate photographing, an irradiating means for illuminating the upper body centering on the face of the operator to be imaged may be provided integrally with the imaging device 9, for example.

   The area 4a includes an input operation area 4a1 for performing a pointing operation and a scroll operation on the right side of the operator's face, an input operation area 4a2 for performing a scroll operation in the downward direction of the operator's face, and the operator's face. On the left side of the face, an input operation area 4a3 for starting up Internet Explorer and mail software is provided. Further, the input operation area 4a1 to the input operation area 4a3 can be arbitrarily set in advance in the area 4a.

Next, an outline of the operation procedure of the input device will be described with reference to FIG.
The input device defines an input operation area definition in which three positions of the input operation area 4a1 to the input operation area 4a3 are determined in advance with respect to the position of the face based on the position coordinates of the face recognized from the image of the camera 3 in the area 4a. This is determined with reference to table 174.

As shown in FIG. 2, the input operation area 4a1 is equivalent to an input surface (OP-No) in which an input operation cannot be performed, an input surface (OP-P) for inputting information equivalent to the pointing operation of the PC 1, and a scroll operation. An input surface (OP-S) for inputting information is provided.
The input operation area 4a2 includes an input surface (OP-No) where input operation cannot be performed and an input surface (OP-S) for inputting information equivalent to the scroll operation.
The input operation area 4a3 includes an input surface (OP-No) in which an input operation cannot be performed and an input surface (OP-F) for inputting input information for activating a predetermined function. In the first embodiment, input information for starting Internet Explorer and mail software can be input.

Which input operation area from the input operation area 4a1 to the input operation area 4a3 is used depends on which input operation area the hand that is the subordinate indicator is found after the face that is the main indicator is recognized. Determined by
If a hand is placed on the right side of the face, hand information is found in the input operation area 4a1. When the hand placed in the input operation area 4a1 raises one finger (see FIG. 1) and stops for 2 seconds, the pointing input surface (OP-P) is selected. When the hand placed in the input operation area 4a1 raises two fingers and stops for 2 seconds, the scroll input surface (OP-S) is selected.
If a hand is placed below the face, hand information is found in the input operation area 4a2. As soon as the hand information is found, the scroll input surface (OP-S) is selected. The shape of the hand at this time may be any shape.
If a hand is placed on the left side of the face, hand information is found in the input operation area 4a3. When the hand placed in the input operation area 4a3 is stationary for 2 seconds, an input surface (OP-F) for activating a predetermined function is selected. At this time, the shape of the hand may be any shape.

When the input screen (OP-P) is selected, the mouse pointer on the display 2 is moved by turning the face in the direction of the position of the icon to be operated displayed on the display 2 (see 6d in FIG. 1). The input is determined by bending and stretching one finger.
When the input screen (OP-S) is selected, the screen is scrolled by turning the face in the direction of scrolling (see 6d in FIG. 1).
When the input screen (OP-F) is selected, input information for activating a predetermined function by moving a stationary hand to the right or moving to the left is input.

  Next, the overall operation of the input device according to the first embodiment configured as described above will be described with reference to the flowchart of FIG. When the PC 1 is activated, the control program for the input device is activated, and the processing shown in FIG. 3 is started.

  First, it is determined whether or not it is a video capture timing (S010). The input device captures video from the camera 3 at a predetermined time interval (in this embodiment, at an interval of 100 milliseconds), and executes the following processing (after S020). If it is not the capture timing (S010: NO), it waits until the capture timing is reached.

  When the capture timing is reached (S010: YES), the video from the camera 3 is captured (S020).

  Next, face information is extracted from the image, and the face appearance and face difference status are determined from the face information. The appearance of the face is classified according to whether or not the face can be recognized. The difference situation determines the position coordinates of the center of the face on the current video in order to determine whether the face is moving. Next, a face difference situation is determined from the difference situation between the current face center position coordinates and the face center position coordinates of the previous face extraction information stored. When the input plane is selected, the two-dimensional coordinates of the face on the selected input plane are calculated from the position coordinates of the center of the face (S031).

  Next, when the arrangement position of the input operation area is confirmed, hand information is extracted from the input operation area of the video, and the difference between the hand shape and the hand is determined from the hand information. The figure of the hand indicates whether or not the hand is present. When the hand is present, the shape of the hand is determined as necessary. The shape of the hand determines and classifies the gap between the fingers, determines whether the finger is one or two, or which direction the finger is facing, and the hand position coordinate is the tip of the finger Is used (S032). In addition, since the extraction of the hand immediately after the position of the input operation area is determined does not know which input operation area has the hand information, it is necessary to determine whether or not the hand information exists for each input operation area. However, once hand information is extracted, hand information may be extracted from the input operation area in which hand information is extracted from the previous video.

  For example, the hand extraction immediately after the arrangement position of the input operation area is determined performs the existence determination on which area of the input operation area 4a3 from the input operation area 4a1 the hand information exists. If hand information is found, then hand information may be extracted from the input operation area 4a2.

  Next, if hand information has been extracted, the indicator state is determined from the face appearance and face difference status, and the hand appearance and hand difference status. For example, if a stationary face can be recognized and the hand is stationary with one finger extended on the right side of the face, the indicator state is “10010” (the indicator state table 173 in FIG. Reference) (S040). If hand information is not extracted, the indicator state is determined from the face appearance and the difference between the faces (see pointing finger state “00010” in FIG. 4).

Next, in the operation state determination means (S050), the operation state of the input device is determined based on the pointing finger state.
For example, an operation state determination table in which an instruction for determining the arrangement position of the input operation area is registered after the face is stationary (indicator state “00010”) after 3 seconds have passed without the arrangement position of the input operation area being determined. Data 171 (see P-1 in FIG. 5) is determined. When this data is confirmed, an instruction to confirm the arrangement position of the input operation area is issued.

  Next, when only one finger is stretched on the right side of the face and finger information in a stationary state (indicator state “10010”) is found, the input surface (OP-NO) of the input operation area 4a1 is selected immediately. If the data (see P-2 in FIG. 5) of the operation state determination table 171 to be confirmed is confirmed and then the state of one finger is stationary for 2 seconds or more, the OP-P input surface for pointing operation is The data (see P-3 and P-4 in FIG. 5) of the operation state determination table 171 that is selected and is in a state in which a pointing operation can be performed according to the vertical and horizontal movements of the face (see 6d in FIG. 1) is confirmed. To do.

  Alternatively, when two fingers are stretched on the right side of the face and finger information in a stationary state (indicator state “11110”) is found, an operation for selecting the input surface (OP-NO) of the input operation area 4a1 is performed. When the data of the state determination table 171 (see P-7 in FIG. 5) is confirmed, and then the state of the two fingers is stationary for 2 seconds or longer, the input surface of the OP-S for the scroll operation is selected. Then, the data (see P-8 in FIG. 5) of the operation state determination table 171 that enables the screen to be scrolled in accordance with the up / down / left / right movement of the face is determined. In this state, when the face moves up, down, left and right (indicator state “11111”), the data (see P-9 in FIG. 5) of the operation state determination table 171 that scrolls in the direction in which the face has moved is fixed.

Next, according to the operation instruction of the data in the operation state determination table 171 determined by the operation state determination unit by the operation procedure execution unit (S060), the determination of the arrangement position of the input operation area, the selection of the input surface, and the input Generate information.
For example, the input device is prepared in advance so that three input operation areas are arranged on the right side, the lower side, and the left side of the face, and when an instruction to confirm the arrangement position of the input operation area is issued, the input device is prepared in advance. Three input operation areas are arranged. If the operation instruction is an input surface selection instruction, the input surface is selected according to the operation instruction content. If the operation instruction is an input information generation instruction, the input information is generated according to the operation instruction content.

When any one of the above processes is completed in the operation procedure execution means (S060), control is transferred to a determination process for determining whether or not it is the next image acquisition timing in S010.
The input operation area 4a1 based on the detection of the hand located on the right side of the face has been described in particular. However, for the input operation area 4a2 and the input operation area 4a3, an indicator state table and an operation state determination table are provided for each input operation area. Needless to say, the contents of the table can be changed if necessary so as to be set in advance and be easy for the operator to use.

  By repeating the processing procedure described above with reference to the flowchart, it is possible to input necessary input information to the computer without using a conventional input device such as a keyboard or a mouse. Various input information can be input by combining both the face and hand indicators.

In the above embodiment,
S010 to S020 in FIG. 3 correspond to the imaging means and process of the present invention. Also,
S031 in FIG. 3 corresponds to the main indicator characteristic determination means and process of the present invention. Also,
S032 in FIG. 3 corresponds to the indicator characteristic determination means and process according to the present invention. Also,
S040 in FIG. 3 corresponds to the indicator state determination means and process of the present invention. Also,
S050 in FIG. 3 corresponds to the operation state determination means and process of the present invention. Also,
S060 in FIG. 3 corresponds to the operating procedure execution means and process of the present invention.

  Next, a PC functioning as an input device according to the present invention will be described with reference to FIGS. 6 to 7 as Embodiment 2 in which emergency information is input at a work place and the input information is displayed on a display in a monitoring room. To do.

First, the physical configuration will be described with reference to FIG.
Reference numeral 1 is a PC that functions as an input device, and is provided with an imaging device 9 including a camera 3 that constantly photographs an operator of the office 4e during operation. In the input operation area, the input operation areas 4a4 are arranged on both sides of the face.

Next, an outline of the operation procedure of the input device will be described with reference to FIG.
When the input device recognizes a stationary face, an input operation area 4a4 is arranged (see FIG. 6). Thereafter, when the hand is raised on both sides of the face, an input surface (OP-No) in which an input operation cannot be performed is selected from the image of the camera 3 in the input operation area 4a4 (see FIG. 6). Thereafter, when hand information is continuously extracted, an input surface (OP-E) for inputting input information for displaying emergency information on the display 2 is selected.

  For example, if you put your hands on both sides of the face and rest for 3 seconds, the input surface (OP-E) will be selected, and if you keep your hands on both sides of the face, input information will be generated to indicate “emergency” And displayed on the display 2 installed in the monitoring room 4f. After that, if both hands are spread to both sides, the “risk level” is determined in advance corresponding to the spread size, thereby generating input information of the risk level and displaying it on the display 2. It should be noted that the shape of the hand may be any shape. The hand position coordinate is the uppermost position of the hand.

  The preferred embodiments of the present invention and specific applications thereof have been described above with reference to the drawings. However, the technical scope of the present invention is not limited to these, and departs from the scope of the claims described below. Without mentioning, some examples have been described in the present specification, but it goes without saying that various changes and modifications can be made.

For example, on the input surface (OP-P) of the first embodiment, the appearance of the face is determined whether the face is recognized and whether it is a smile, and the click operation of the selected icon is determined based on whether it is a smile. It is also possible to respond by moving the face up and down within a predetermined time.
Further, the position of the input operation area may be different depending on the result of face recognition, when smiling and when not smiling.

  The present invention can be used as an information input device for various electronic devices such as computers, electronic musical instruments, game devices, and audio devices.

DESCRIPTION OF SYMBOLS 1 Personal computer 2 Display 3 Camera 4a1 Input operation area | region 4a2 arrange | positioned on the right side of a face Input operation area | region 4a3 arrange | positioned below the face Input operation area | region 4a4 arrange | positioned on the left side of a face Input operation arrange | positioned on both sides of a face Area 6 Hand 6c Face 6d Action to turn the face left and right and up and down 4e Office 4f Monitoring room 9 Imaging device

Claims (5)

An information input device for electronic equipment, the device comprising:
Imaging means for sequentially capturing images of a predetermined area of the input operation surface;
The main indicator information is extracted from the image from the imaging means, the main indicator shape and the position coordinates of the main indicator are determined from the main indicator information, and the current position coordinates of the main indicator on the image Based on the previous position coordinates stored, the main indicator for determining whether or not the main indicator is moving is determined, and the input operation surface is determined from the current position coordinates of the main indicator. Main indicator characteristic determination means for calculating the two-dimensional coordinates of the current main indicator on the input surface formed at a predetermined position;
When an input operation area defined as a predetermined area of the input operation surface is determined by the main indicator shape of the main indicator characteristic determination means and the position coordinates of the main indicator, the input from the imaging means Subordinate indicator characteristic determining means for extracting the auxiliary indicator information around the main indicator from the video of the operation area, and determining the shape of the auxiliary indicator from the auxiliary indicator information;
The indicator state determining means for determining the indicator state based on the difference between the main indicator and the main indicator from the main indicator characteristic determining means and the following indicator shape from the secondary indicator characteristic determining means; ,
At least one or more operations comprising a combination of the input operation area, the input surface, the indicator state, and the operation instruction, which can be determined and changed in advance according to the indicator state determined by the indicator state determination means. An operation state determination means for determining an operation state from an operation state determination table associated with a procedure;
According to the operation state determined by the operation state determination means, the content of the operation instruction for determining the position of the input operation area, selecting the input surface, or generating the input information is the instruction for determining the position of the input operation area. If there is, confirm the arrangement position of the input operation area according to the instruction to confirm the arrangement position of the input operation area,
If the content of the operation instruction is an input surface selection instruction, change the currently selected input surface according to the input surface selection instruction,
If the content of the operation instruction is an instruction to generate input information, on the currently selected input surface,
Generating input information in response to an instruction to generate the input surface and the input information; or
An operation procedure execution means for generating input information from the input surface, an instruction for generating the input information, and the two-dimensional coordinates of the main indicator;
An information input device for electronic equipment, comprising: The information input device according to claim 1, wherein the subordinate indicator characteristic determining unit is configured to determine the input operation surface based on the main indicator characteristic of the main indicator characteristic determining unit and the position coordinates of the main indicator. When the input operation area determined as the predetermined area is fixed, subordinate indicator information is extracted from the image of the input operation area from the imaging means, and the subordinate indicator information is subtracted from the subordinate indicator information. Determine the figure and the position coordinates of the secondary indicator on the input operation area, and determine whether the secondary indicator is moving based on the current position coordinates of the secondary indicator and the previous position coordinates stored. The difference state of the subordinate indicator for determination is determined, and the current subordinate indicator on the input surface formed at a predetermined position on the input operation surface from the position coordinates of the subordinate indicator is determined. Calculate 2D coordinates,
When the input operation area is fixed by the pointer state determination means, the difference between the main indicator shape and the main pointer from the main pointer characteristic determination means and the secondary pointer characteristic determination means The status of the indicator is determined based on the difference between the shape of the subordinate indicator and the subordinate indicator.
In the operation procedure execution means, when the content of the operation instruction is an instruction to generate input information, on the currently selected input surface,
Generating input information in response to an instruction to generate the input surface and the input information; or
Generate input information from the input surface, the input information generation instruction and the two-dimensional coordinates of the main indicator, or
Generating input information from the input surface and the instruction for generating the input information and the two-dimensional coordinates of the subordinate indicator;
An information input device for an electronic device.   2. The information input device according to claim 1, wherein the operation state determination table of the operation state determination unit includes the input operation region, the input surface, the indicator state, a time during which the indicator state continues, and an operation instruction. An information input device for electronic equipment, wherein at least one set of operation procedures consisting of combinations of the above is associated. The information input device according to claim 1, wherein the operation procedure execution unit includes:
An input operation region definition table that associates an input operation region that is predetermined and can be changed with respect to a positioning instruction of the input operation region and a position coordinate of a main indicator;
If the content of the operation instruction of the operation state determination means is confirmation of the arrangement position of the input operation area, the arrangement position of the input operation area is confirmed from the input operation area definition table,
An input information table in which a plurality of input information that is predetermined and changeable with respect to the input surface, the input information generation instruction, and the two-dimensional coordinates of the indicator are associated;
If the content of the operation instruction of the operation state determination means is an instruction to generate input information, on the currently selected input surface,
Generating input information determined by an operator in response to the input surface and the input information generation instruction; or
Generate input information from the input information table from the input surface, the input information generation instruction and the two-dimensional coordinates of the main indicator.
An information input device for an electronic device. A method of inputting information to an electronic device, the method comprising:
An imaging process for sequentially capturing images of a predetermined area of the input operation surface;
The main indicator information is extracted from the video from the imaging process, the main indicator shape and the position coordinates of the main indicator are determined from the main indicator information, and the current position coordinates of the main indicator on the video Based on the previous position coordinates stored, the main indicator for determining whether or not the main indicator is moving is determined, and the input operation surface is determined from the current position coordinates of the main indicator. A main indicator characteristic determination process for calculating the two-dimensional coordinates of the current main indicator on the input surface formed at a predetermined position;
When an input operation area defined as a predetermined area on the input operation surface is determined by the main indicator shape and the position coordinates of the main indicator in the main indicator characteristic determination process, input from the imaging process Subordinate indicator characteristic determination process for extracting the subordinate indicator information around the main indicator from the operation region image, and determining the shape of the subordinate indicator from the subordinate indicator information;
The indicator state determination process for determining the indicator state based on the difference between the main indicator and the main indicator from the main indicator characteristic determination process and the subordinate indicator shape from the sub indicator characteristic determination process. ,
At least one or more operations comprising a combination of the input operation area, the input surface, the indicator state, and the operation instruction, which can be determined and changed in advance according to the indicator state determined by the indicator state determination process. An operation state determination process for determining an operation state from an operation state determination table associated with a procedure;
According to the operation state determined by the operation state determination process, the content of the operation instruction for determining the position of the input operation area or selecting the input surface or generating the input information is the instruction for determining the position of the input operation area. If there is, confirm the arrangement position of the input operation area according to the instruction to confirm the arrangement position of the input operation area,
If the content of the operation instruction is an input surface selection instruction, change the currently selected input surface according to the input surface selection instruction,
If the content of the operation instruction is an instruction to generate input information, on the currently selected input surface,
Generating input information in response to an instruction to generate the input surface and the input information; or
An operation procedure execution process for generating input information from the input surface, the input information generation instruction, and the two-dimensional coordinates of the main indicator;
An information input method for electronic equipment, comprising: