JP5852928B2 - Operation input device, operation input method and program - Google Patents

Description

  The present invention relates to an action input device, an action input method, and a program for converting a trajectory of a user's finger movement into an input signal.

  Various input devices have been developed for operating means of a pointing device in electronic equipment. As the device was downsized, it changed to a mouse that required a certain amount of flat space, and an analog stick and jog dial that could be used while moving were installed. Today, it can be said that a touch panel is common as an operation means in a smartphone or a slate terminal.

  However, when the touch panel is used, it is necessary to directly touch the liquid crystal with a finger, so that there is a problem that the liquid crystal becomes dirty or damaged. In addition, since input can be performed only at points on the touch panel, there is a limitation that only planar input can be detected unless the touch panel has a special shape.

  On the other hand, as a technique that has recently attracted attention, there is a technique in which a distance from a user is determined by an infrared sensor or the like and converted into an input signal. In this method, it is possible to detect the three-dimensional arm tilt and the pointing direction, as well as the stationary state, as well as dynamic actions such as waving and bowing, as input to the device, and specific events Can be generated.

  For example, each distance between the three quantum infrared sensors A, B, C and the finger is calculated from the output signal intensity of the three quantum infrared sensors A, B, C arranged in a right isosceles triangle shape, A position detection device that calculates three-dimensional coordinates representing the position of a finger from each distance has been clarified (Patent Document 1).

Japanese Patent Publication No. 2007-80214

  However, in the position detection device disclosed in Patent Document 1, it is naturally necessary to attach a plurality of infrared sensors to the terminal. Moreover, since accuracy will fall if the distance of each sensor is too close, it is necessary to ensure a certain physical size according to the accuracy of a sensor.

  In order to solve the drawbacks of the detection method described above, the present invention detects a user's fingerprint from a captured image, and if the locus matches a preset operation pattern, converts it to an input signal corresponding to the operation pattern. It is an object to provide an action input device, an action input method, and a program.

  The present invention provides the following solutions.

The invention according to the first feature is an operation input device that detects an operation of a user's finger and converts it into an input signal,
Imaging means for acquiring a predetermined range being photographed as an image;
A fingerprint detection means for analyzing the image acquired by the imaging means and detecting the fingerprint of the user;
When the fingerprint detected by the fingerprint detection means and a fingerprint registered in advance are identified and a fingerprint of a specific user is identified, a fingerprint identification means that performs a user authentication function as security is further provided. ,
A fingerprint action storage means for storing a trajectory of the movement of the finger by temporally changing the fingerprint detected by the fingerprint detection means by an operation of the user's finger;
From the trajectory stored by the fingerprint operation storage means, extract the operation pattern from a plurality of types of operation patterns represented by a step indicating a predetermined finger movement or a combination of a plurality of types of steps. Input signal changing means for converting into an input signal previously associated with the extracted operation pattern;
Equipped with a,
When the plurality of operation patterns are extracted, the input signal changing unit determines the operation pattern having the highest priority as the operation pattern and converts it into an input signal previously associated with the operation pattern. to provide a gesture input device you.

According to the first aspect of the invention, the operation input device that detects the movement of the user's finger and converts it into an input signal acquires a predetermined range being shot as an image, analyzes the acquired image, When the fingerprint of the user is detected, whether the detected fingerprint matches a pre-registered fingerprint is identified, and the fingerprint of the specific user is identified, the user authentication function as security is further fulfilled The detected fingerprint changes with time by the operation of the user's finger, so that the trajectory of the operation of the finger is memorized, and a step or a plurality of types indicating preset finger movements from the memorized trajectory of extracting a plurality of kinds of operation pattern represented by a combination of the steps, when converted into the associated in advance with the input signal to the extracted operation patterns, and extracts a plurality of operation patterns, What is the highest priority among these, as an operating pattern that converts into associated in advance with the input signal to the operation pattern.

In the invention according to the second feature, in the imaging means, the predetermined range is imaged three-dimensionally rather than planarly,
In the fingerprint operation storage means, and wherein the sterically storing operation of the fingers, to provide a gesture input device is the invention according to the first aspect.

According to the invention of the second aspect, an invention according to the first aspect gestures input device, three-dimensionally imaged rather than planar predetermined range, three-dimensionally storing the operation of a finger.

  These above-mentioned inventions are categories of operation input devices, but even if they are categories of methods and programs, there are similar operations and effects according to the categories.

The invention according to the third feature is
An operation input method executed by an operation input device that detects an operation of a user's finger and converts it into an input signal,
Obtaining a predetermined range being photographed as an image;
Analyzing the acquired image and detecting the user's fingerprint;
Identifying whether the detected fingerprint and a pre-registered fingerprint match, and when a specific user's fingerprint is identified, further performing a user authentication function as security;
Storing the locus of movement of the detected finger by temporally changing the detected fingerprint by an operation of the user's finger;
From the stored trajectory, from among a plurality of types of operation patterns represented by a combination of the steps of the step or a plurality of types showing the movement of the preset finger, and extracting the operation pattern, extracted the operation pattern Converting to an input signal previously associated with
Equipped with a,
In the step of converting to the input signal, when a plurality of the operation patterns are extracted, the highest priority among them is determined as the operation pattern, and the input signal previously associated with the operation pattern It provides a gesture input method that converts into.

The invention according to the fourth feature is as follows:
In the operation input device that detects the movement of the user's finger and converts it into an input signal,
Obtaining a predetermined range being photographed as an image;
Analyzing the acquired image and detecting the fingerprint of the user;
Identifying whether the detected fingerprint matches a pre-registered fingerprint, and if a specific user's fingerprint is identified, further performing a user authentication function as security,
Storing the locus of movement of the detected finger by temporally changing the detected fingerprint by the user's finger operation;
From the stored trajectory, from among a plurality of types of operation patterns represented by a combination of the steps of the step or a plurality of types showing the movement of the preset finger, and extracting the operation pattern, extracted the operation pattern Converting to an input signal previously associated with
And execute
In the step of converting to the input signal, when a plurality of the operation patterns are extracted, the highest priority among them is determined as the operation pattern, and the input signal previously associated with the operation pattern providing because gesture input device program to convert.

  According to the present invention, an action input device, an action input method for detecting a user's fingerprint from a captured image and converting the detected locus into an input signal corresponding to the action pattern when the locus matches the preset action pattern, And it becomes possible to provide a program.

FIG. 1 is a conceptual diagram for explaining an outline of the action input device 1. FIG. 2 shows an example of converting an operation into an input signal using the operation input device 1. FIG. 3 is a flowchart showing the procedure of the operation input process executed by the operation input device 1. FIG. 4 is a conceptual diagram showing a system configuration of the action input device 1. FIG. 5 is a functional block diagram of the action input device 1. FIG. 6 is a flowchart showing a procedure of fingerprint detection and position storage processing executed by the action input device 1. FIG. 7 is a flowchart showing the procedure of the locus input signal conversion process executed by the action input device 1. FIG. 8 is a diagram showing a three-dimensional coordinate system with the operation input device 1 as the origin. FIG. 9 shows a part of the fingerprint position table in the fingerprint position database 50. FIG. 10 shows a part of the operation pattern table in the operation pattern database 51.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. This is merely an example, and the technical scope of the present invention is not limited to this.

  The outline of the present invention will be described. The operation input device 1 acquires images captured by the camera 101 and the camera 102 when the user's finger traces 201a to 201b and 201c and draws a locus as indicated by the arrows 202 and 203, for example. The user's fingerprint is detected in the image, and the locus is stored in the fingerprint position database 50. Thereafter, the operation input device 1 extracts an operation pattern that matches the locus of the user's finger from the operation pattern database 51, and extracts an input signal associated with the operation pattern. Then, the user's finger trajectory is converted into an input signal by acting in the same manner as the extracted input signal is actually input.

[Outline of Message Sending System]
Here, based on FIG.1, FIG2 and FIG.3, the outline | summary of the operation input apparatus 1 which is suitable embodiment of this invention is demonstrated. Here, the operation input device 1 may be an information terminal, and the operation input device 1 itself may receive the converted input signal. The operation input device 1 has a communication function, and the operation input device 1 itself is merely an input. The device may behave as a device, and the converted input signal may be transmitted to another device that is communicably connected. In the following embodiments, the former case will be described.

  FIG. 1 is a conceptual diagram for explaining an outline of the action input device 1. Here, the action input device 1 has a shape imitating glasses, and is used by a person wearing it on the face. The operation input device 1 does not necessarily have a shape imitating glasses, and may be a device such as a smartphone or an information terminal as long as it has a function described in detail below.

  The action input device 1 includes a camera 101 for photographing the same range as the right eye field of view and a camera 102 for photographing the same range as the left eye field of view. A storage unit 103 near the center stores a control unit 11 that controls the operation input device 1, a storage unit 13 that stores information, and a communication unit 14 that communicates with other devices. . Details of the control unit 11, the storage unit 13, and the communication unit 14 will be described later.

  The storage unit 13 includes a fingerprint position database 50 for storing a user's fingerprint position and an operation pattern database 51 for converting an operation pattern into an input signal. Further, the storage unit 13 stores the feature amount of the user's fingerprint. The storage unit 13 may include an input signal ID 52 stored in the operation pattern database 51 and an input signal database 52 in which corresponding input signals are stored.

  The lens 104 may be mere glass, with or without power, or may be a liquid crystal that can display information superimposed on the field of view. In addition, the operation input device 1 may include a projector and project information on the lens 104 to display the information superimposed on the field of view. The behavior of the information terminal after the operation is converted into the input signal is not included in the scope of the technical idea of the present invention, and thus will not be described in detail here.

  FIG. 2 shows an example of converting an operation into an input signal using the operation input device 1. The near side of the image is the near side for the user, and the chord portion of the glasses is long, so a part thereof is omitted. The fingerprint 201a, the fingerprint 201b, and the fingerprint 201c are not the display by the operation input device 1, but the trajectory of their fingers that the user is looking through the glasses. The arrows 202 and 203 are not actually visible and are described for the sake of explanation.

  Now, when the user moves the finger of the left hand from the left to the right in front of the left eye, the finger traces the fingerprint 201a, the fingerprint 201b, and the fingerprint 201c with the passage of time. The situation is photographed by the camera 102, and the user's fingerprint is moved from left to right for the user, that is, the locus of the user's finger can be captured. When this left-to-right movement is associated with the same input signal as the left-to-right swipe on the touch panel, the user causes the camera 102 to capture the finger trajectory, so that the user can perform the same behavior as the swipe on the touch panel. An event can be generated in the action input device 1.

  FIG. 3 is a flowchart showing the procedure of the operation input process executed by the operation input device 1. The procedure of the action input process will be described below.

  First, the action input device 1 acquires the shooting ranges of the cameras 101 and 102 as images (step S01). Here, the number of cameras may be one or more. However, as will be described later, when there is only one camera, the accuracy of receiving a three-dimensional operation as an input decreases.

  Next, the operation input device 1 detects the user's fingerprint from the acquired image (step S02), calculates the position of the fingerprint with reference to the operation input device 1 from the position of the fingerprint in the image, and obtains the fingerprint position database. 50 (step S03). Steps S01 to S03 are called fingerprint detection and position storage processing.

  Next, the operation input device 1 reads the position of the fingerprint stored for a predetermined period from the present time from the fingerprint position database 50, regards it as a continuous trajectory, and extracts a matching motion pattern from the motion pattern database 51 ( Step S04). Then, assuming that an input signal corresponding to the matched operation pattern has been input, the trajectory is converted into an input signal, which is received by the device 1 itself or transmitted to another device that is communicably connected. (Step S05). Steps S04 and S05 are referred to as a locus input signal conversion process. The above is the procedure of the operation input process executed by the operation input device 1.

  The action input device 1 converts the user's action into an input signal by executing the action input process described above. The above is the outline of the action input device 1 which is a preferred embodiment of the present invention.

[System configuration of the operation input device 1]
FIG. 4 is a conceptual diagram showing a system configuration of the action input device 1. When the operation input device 1 is an information terminal and the operation input device 1 itself receives the converted input signal, there is no portion indicated by a dotted line in FIG. 4, and only the operation input device 1 is used. It is possible to execute processing.

  On the other hand, when the operation input device 1 behaves as a simple input device, the operation input device 1 has a communication function and is connected to a portion indicated by a dotted line in FIG. The input signal converted to the terminal 301 is transmitted. Here, the communication between the operation input device 1 and the information terminal 301 may be wireless such as WiFi (Wireless Fidelity) compliant with IEEE 802.11 or Bluetooth, or may be preferential connection. The information terminal 301 receives an input signal from the action input device 1 and behaves in the same manner as when there is an input to the input unit of the information terminal 301.

[Description of each function]
Based on FIG. 5, the structure of the operation input device 1 is demonstrated. The operation input device 1 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like as the control unit 11, and a camera that captures a predetermined range as an image as the input / output unit 12. In addition to a photographing device such as a liquid crystal monitor for displaying images and text messages, and a display unit such as a projection screen. Similarly, the operation input device 1 includes a data storage unit such as a hard disk, a semiconductor memory, a recording medium, or a memory card as the storage unit 13 for storing data and files. The storage unit 13 includes a fingerprint position database 50, an operation pattern database 51, and an input signal database 52. In addition, since the operation input device 1 functions as an input device of another device, the communication unit 14 is a device for enabling communication with another device, for example, WiFi (Wireless Fidelity) compliant with IEEE 802.11. Provide a compatible device.

  In the operation input device 1, the control unit 11 reads a predetermined program, thereby realizing the captured image acquisition module 15 in cooperation with the input / output unit 12. Further, in the operation input device 1, the control unit 11 reads a predetermined program so that the fingerprint detection module 16, the coordinate calculation module 17, the fingerprint position storage module 18, and the trajectory conversion module 19 cooperate with the storage unit 13. Realize. In the operation input device 1, the control unit 11 reads a predetermined program, thereby realizing the input signal notification module 20 in cooperation with the communication unit 14.

[Fingerprint detection and location storage processing]
FIG. 6 is a flowchart showing the procedure of fingerprint detection and position storage processing executed by the action input device 1. The fingerprint detection and position storage processing will be described.

  First, the captured image acquisition module 15 of the action input device 1 acquires the imaging ranges of the camera 101 and the camera 102 as images (step S11). In the case of an apparatus simulating eyeglasses as in the present embodiment, the shooting range of both cameras substantially matches the user's field of view.

  The fingerprint detection module 16 of the action input device 1 extracts a feature amount from each image in order to detect the user's fingerprint included in the acquired image (step S12). Here, the feature amount is a value to be compared when pattern recognition is performed using a user's fingerprint stored in advance as a pattern from the received image. As a specific process, the fingerprint detection module 16 equally divides the acquired image vertically and horizontally into a predetermined number, and calculates the average value of the color of the pixels in each part.

  Next, the fingerprint detection module 16 reads the feature quantity of the user's fingerprint stored in the storage unit 13 included in the action input device 1 (step 13). Here, when a general fingerprint feature value is read instead of a specific user's fingerprint, the fingerprint can be converted into an input signal regardless of the person or the finger. Further, when the feature amount of the fingerprint of a specific user is read, it can serve as a user authentication function as security. Then, when the fingerprint feature amount for each finger is read, a combination of a plurality of finger actions may be converted into an input signal by performing the process described later for each finger.

  The fingerprint detection module 16 detects a fingerprint from the acquired image by using the read fingerprint feature amount (step S14). That is, a fingerprint portion in an image is detected by performing pattern recognition of the fingerprint feature amount from the image feature amount. At this time, if the finger is tilted with respect to the camera, the fingerprint is represented as a projection in the image, and detection may not be successful. To solve this problem, it is possible to improve the fingerprint detection accuracy by capturing the object in the image in three dimensions by comprehensively considering the feature values in each camera.

  As a result of step S14, when a fingerprint is detected from the image (step S15: “YES”), the following processing is subsequently performed. If no fingerprint is detected from the image (step S15: “NO”), the following processing is not performed, and the fingerprint detection and position storage processing ends.

  When a fingerprint is detected by the fingerprint detection module 16, a straight line connecting the fingerprint and the camera can be expressed as an expression from the position in the fingerprint image regardless of the size of the finger. When the number of cameras is two or more, the coordinate calculation module 17 solves the linear equation for each camera as a simultaneous equation and calculates the position of the fingerprint in the three-dimensional coordinate system centered on the operation input device 1 ( Step S16). In calculating the solution, a calculation algorithm stored in the storage unit 13 may be read and used. At the same time, the coordinate calculation module 17 may calculate the orientation of the finger by obtaining two or more positions of the fingerprint. Here, the position of the fingerprint is a predetermined position of the fingerprint.

  When there is one camera, only one equation can be obtained and the above method cannot be used. Therefore, the coordinate calculation module 17 calculates the distance between the camera and the finger based on the size of the finger stored in advance, calculates the direction based on the distortion of the fingerprint in the image, or 2 instead of the three-dimensional coordinate system. The position of the finger in the dimensional coordinate system is calculated.

  FIG. 8 is a diagram showing a three-dimensional coordinate system with the operation input device 1 as the origin. When the operation input device 1 is worn, the right hand direction of the user is the positive direction of the x axis. Further, the overhead of the user is the positive direction of the y axis, and the vertical direction is the negative direction. Further, when the finger position is captured in a three-dimensional coordinate system, the forward direction for the user is the positive z-axis direction. In addition, the rotation angles with respect to the x-axis, y-axis, and z-axis can be expressed as Roll, Pitch, and Yaw, respectively. In this embodiment, the right-handed screw direction is positive with respect to the axis, and is expressed in a range of −180 degrees to +180 degrees. By using Roll, Pitch, and Yaw, the direction from the base of the finger to the direction of the fingertip can be numerically described, and a more complicated operation can be expressed. In the present embodiment, an orthogonal coordinate system is adopted, but a polar coordinate system or the like may be used, and the method of representing the position and posture of the finger is not limited to this format.

  Thereafter, the fingerprint position storage module 18 of the action input device 1 stores the calculated x, y, z, Roll, Pitch, and Yaw in the fingerprint position database 50 together with the date and time when the image is acquired (step S17). FIG. 9 shows a part of the fingerprint position table in the fingerprint position database 50.

  The above is the procedure of fingerprint detection and position storage processing executed by the action input device 1.

[Track input signal conversion processing]
FIG. 7 is a flowchart showing the procedure of the locus input signal conversion process executed by the action input device 1. The locus input signal conversion processing will be described.

  First, the trajectory conversion module 19 of the action input device 1 reads the stored fingerprint position from the fingerprint position database 50 (step S21). Normally, input is performed in units of a few seconds, so that only the position of the fingerprint from the present to a few seconds before may be read.

  Here, the trajectory conversion module 19 determines whether the change of each coordinate component of the fingerprint is equal to or less than a predetermined threshold value or no fingerprint in the image in a predetermined period, so that the user's finger is stopped. Determine if it is in a state. If it is not in the stop state (step S22: “NO”), it is determined that the user's finger is in the middle of drawing the locus, the following processing is not performed, and the locus input signal conversion processing ends. In the case of the stop state (step S22: "YES"), the following processing is performed in order to convert the trajectory up to that point into an input signal. Note that the absence of a fingerprint in an image at a certain time can be determined by the absence of a record of that time in the fingerprint position database 50.

  The trajectory conversion module 19 regards the read fingerprint position as a continuous trajectory, and extracts an operation pattern that matches the trajectory from the operation pattern database 51 (step S23). If the motion pattern is not extracted (step S24: “NO”), the user determines that the input is not valid, the following processing is not performed, and the input signal conversion processing of the trajectory ends. When one or more operation patterns are extracted (step S24: “YES”), the following processing is performed subsequently.

  FIG. 10 shows a part of the operation pattern table of the operation pattern database 51. In this embodiment, the operation pattern is represented by a combination of steps of movement corresponding to the x-axis, y-axis, and z-axis. Each operation pattern is associated with an input signal. Note that one operation pattern does not prevent being combined with two or more input signals.

  For example, when the x-coordinate of the fingerprint increases with time, it is assumed that “RIGHT” is moved in this step, and when the y-coordinate is increased thereafter, “UP” is moved. . In such a case, the pattern whose pattern ID is 2 matches. A threshold value may be set for each change.

  In FIG. 10, when two or more movements are necessary in each step, they are connected by a comma. For example, in the case of “RIGHT, UP”, the x coordinate and the y coordinate are satisfied only when they are increased at the same timing. When there is no condition at each step, it is represented by a character string “NULL”.

  By the way, in the extraction method, it may be more convenient to extract a pattern that satisfies at least a condition rather than an operation pattern in which the operations of all steps are completely the same. For example, when there is no pattern on the operation pattern table in which step 1 is “RIGHT” and step 2 is “UP, LEFT”, when the fingerprint moves, whether the pattern ID is the operation pattern 2 or 3 It is desirable that it is determined that it is an input error.

  In preparation for such a case, the trajectory conversion module 19 first extracts a pattern satisfying at least a condition, and if a plurality of motion patterns are extracted, the trajectory conversion module 19 matches the one with the highest priority among them. The operation pattern may be determined (step S25). The priority is considered to be higher as the operation is more complicated and as the risk of the user when it is erroneously input is smaller. In the former example, when the pattern ID matches both the operation patterns of 3 and 11, the more complicated 11 has a higher priority. In the latter example, the operation pattern associated with the cancel button has a higher priority than the operation pattern associated with the power button and the determination key.

  Next, the trajectory conversion module 19 specifies an input signal associated with the determined operation pattern (step S26). Corresponding input signal IDs may be stored in the operation pattern database 51 as in the present embodiment, or instructions themselves to be executed by the device may be stored in binary format. When stored in the ID format, the trajectory conversion module 19 refers to the input signal database 52 in the storage unit 13 and converts the input signal ID into an actual input signal. Note that the input signal may represent a mouse movement or a specific key press, or an input signal corresponding to a pinch or swipe on the touch panel. Further, the input signal may be represented by a function that takes the change of each coordinate component of the fingerprint as a variable.

  Finally, the input signal notification module 20 of the action input device 1 notifies the target of the input signal specified by the trajectory conversion module 19 (step S27). When the notification target is the action input device 1 itself, as an example, by transmitting a message to the operating system of the action input device 1, it behaves in the same manner as when the same input is received in the input unit. At this time, it is not necessary to operate the communication unit 14. When the operation input device 1 is operating as an input device of another information terminal, the specified input signal is transmitted to the information terminal.

  The above is the procedure of the locus input signal conversion process executed by the operation input device 1.

  The means and functions described above are realized by a computer (including a CPU, an information processing apparatus, and various terminals) reading and executing a predetermined program. The program is provided in a form recorded on a computer-readable recording medium such as a flexible disk, CD (CD-ROM, etc.), DVD (DVD-ROM, DVD-RAM, etc.), for example. In this case, the computer reads the program from the recording medium, transfers it to the internal storage device or the external storage device, stores it, and executes it. The program may be recorded in advance in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided from the storage device to a computer via a communication line.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to these embodiment mentioned above. The effects described in the embodiments of the present invention are only the most preferable effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. is not.

  DESCRIPTION OF SYMBOLS 1 Message transmission system, 10 User use terminal, 100 Visual field image analysis server, 150 Feature-value database, 151 Person information database, 200 Message transmission server

Claims (4)

An operation input device that detects the movement of a user's finger and converts it into an input signal,
Imaging means for acquiring a predetermined range being photographed as an image;
A fingerprint detection means for analyzing the image acquired by the imaging means and detecting the fingerprint of the user;
When the fingerprint detected by the fingerprint detection means and a fingerprint registered in advance are identified and a fingerprint of a specific user is identified, a fingerprint identification means that performs a user authentication function as security is further provided. ,
A fingerprint action storage means for storing a trajectory of the movement of the finger by temporally changing the fingerprint detected by the fingerprint detection means by an operation of the user's finger;
From the trajectory stored by the fingerprint operation storage means, extract the operation pattern from a plurality of types of operation patterns represented by a step indicating a predetermined finger movement or a combination of a plurality of types of steps. Input signal changing means for converting into an input signal previously associated with the extracted operation pattern;
With
When the plurality of operation patterns are extracted, the input signal changing unit determines the operation pattern having the highest priority as the operation pattern and converts it into an input signal previously associated with the operation pattern. The input device that performs. In the imaging means, the predetermined range is imaged three-dimensionally rather than planarly,
2. The operation input device according to claim 1 , wherein the fingerprint motion storage means memorizes a finger motion three-dimensionally. An operation input method executed by an operation input device that detects an operation of a user's finger and converts it into an input signal,
Obtaining a predetermined range being photographed as an image;
Analyzing the acquired image and detecting the user's fingerprint;
Identifying whether the detected fingerprint and a pre-registered fingerprint match, and when a specific user's fingerprint is identified, further performing a user authentication function as security;
Storing the locus of movement of the detected finger by temporally changing the detected fingerprint by an operation of the user's finger;
From the stored trajectory, the motion pattern is extracted from a plurality of types of motion patterns represented by a step indicating a predetermined finger movement or a combination of a plurality of types of the steps, and the extracted motion pattern Converting to a pre-associated input signal;
With
In the step of converting to the input signal, when a plurality of the operation patterns are extracted, the highest priority among them is determined as the operation pattern, and the input signal previously associated with the operation pattern The input method to convert to. In the operation input device that detects the movement of the user's finger and converts it into an input signal,
Obtaining a predetermined range being photographed as an image;
Analyzing the acquired image and detecting the fingerprint of the user;
Identifying whether the detected fingerprint matches a pre-registered fingerprint, and if a specific user's fingerprint is identified, further performing a user authentication function as security,
Storing the locus of movement of the detected finger by temporally changing the detected fingerprint by the user's finger operation;
From the stored trajectory, the motion pattern is extracted from a plurality of types of motion patterns represented by a step indicating a predetermined finger movement or a combination of a plurality of types of the steps, and the extracted motion pattern Converting to a pre-associated input signal;
And execute
In the step of converting to the input signal, when a plurality of the operation patterns are extracted, the highest priority among them is determined as the operation pattern, and the input signal previously associated with the operation pattern Program for input device to convert to.

Images