KR101258969B1 - Input device using non-contact user interface - Google Patents

Abstract

PURPOSE: An input device having a non-contact user interface is provided to easily execute a key under an individual section if a part of a user body is positioned in a specific individual section of a virtual keyboard unit. CONSTITUTION: An input device having a non-contact user interface includes a keyboard unit(20), a virtual keyboard unit(30), a hand motion detecting unit(40), a position calculating unit(50), and a control unit(60). The keyboard unit has more than one letter information and indicates the letter information in an individual section. The virtual keyboard unit is upwardly separated from the keyboard unit leaving a predetermined gap. The virtual keyboard forms an area for the individual section to correspond to the individual section of the keyboard unit on a flat surface. The hand motion detecting unit is formed in a predetermined position above the virtual keyboard unit. The hand motion detecting unit detects a user hand motion positioned on the virtual keyboard unit by photographing the user hand motion. The position calculating unit calculates the position of the user hand motion on the virtual keyboard unit using a vector value of the user hand motion detected in the hand motion detecting unit. If individual section information of the virtual keyboard unit is drawn from the location of the user hand motion, the individual section information of the keyboard unit corresponding to the individual section information of the virtual keyboard unit is checked, and the control unit outputs a control signal executing the letter information of the individual section of the keyboard unit. [Reference numerals] (20) Keyboard unit; (30) Virtual keyboard unit; (40) Hand motion detecting unit; (50) Position calculating unit; (60) Control unit

Description

Input device using contactless user interface {INPUT DEVICE USING NON-CONTACT USER INTERFACE}

The present invention relates to an input device using a non-contact user interface, and more particularly, to allow a user to access a portion of his or her body without directly contacting a keyboard portion displayed for performing a specific purpose, and approaching the vicinity of the keyboard portion. It relates to an input device using a non-contact user interface that can easily select the keyboard of the individual compartment.

Recently, in order to cope with the increase in financial transactions, an automatic teller machine (ATM) has been installed in public places so that users can execute financial transactions directly without the help of financial institutions. Through various devices such as banking and telephony banking, users can process financial transactions directly. Cash dispenser as described above is operated even after the operation of the financial institution is very convenient for the user.

In addition, as population density rapidly increases around large cities, apartments or skyscrapers are rapidly being generated. In such an apartment or a high-rise building, several elevators are installed in each building to facilitate the movement of people between floors.

As described above, in order to select a task to be processed among the menu items of the ATM, or to select a switch or a button displaying a floor to be moved in an elevator, the user may apply a force higher than a predetermined level through any part of his / her body. Currently, a method of selecting a corresponding switch or the like by applying a button or touching the touch screen is mainly used.

However, the method of selecting a switch by applying a force above a certain level to the corresponding switch, etc., requires the user to contact any part of the body with the corresponding switch or button, and also requires the effort to apply a force above a certain level. Cause.

In addition, the user selects the switch by touching any part of the body including a finger to the switch without applying a certain force, so that any part of the body must directly touch the switch, By contacting one's body with a switch used by people, problems such as the spread of viruses and contamination of the body occur through the switch.

That is, in the related art, a predetermined key button is installed on the surface of the terminal and directly pressed on the rubber pad or input by scrolling, inputting on a two-dimensional floor plane or wall, or power supply ON / OFF. It has been used as a method of inputting a constant voltage, a constant resistance type touch screen using a touch panel only with physical external buttons, and data and execution keys for voice recognition through human voice. In addition, an analog electric signal is output based on the distance between the reflector and the infrared sensor. The analog electric signal is digitized through a digital conversion process, and may be processed by a virtual keyboard or a mouse with a key value corresponding to each value. In recent years, as a gesture-based user interface, the use of various sensors for interaction has also responded to the needs of users.

However, data and execution methods as voice recognition through voice other than the classical method on the two-dimensional input are unsuitable for use in public due to the weakness of noise, which makes it possible to distinguish a specific voice from multiple users. Stable user voice recognition technology is required, and the gesture-based interface input method is inconvenient to use multiple cameras and expensive equipment due to the light source interference problem, or the user must interact with the direct equipment. The challenge was to require more technology in a separate external device to address the changes.

The present invention has been made to solve the above problems, the present invention comprises a virtual keyboard portion to form an area of the individual partition in a plane corresponding to the keyboard portion, the body part of the user moving on the virtual keyboard portion is virtual An object of the present invention is to provide an input device using a non-contact user interface that can easily execute the keyboard of an individual compartment below even if located in a specific individual compartment of the keyboard unit.

In order to solve the above technical problem, in one preferred embodiment of the present invention, the input device using the contactless user interface, the keyboard unit having one or more character information, and displays the character information in a separate section ( 20); A virtual keyboard part 30 spaced apart from each other by a predetermined interval directly above the keyboard part 20 to form an area of the individual partitions in a plane corresponding to the individual partitions of the keyboard part 20; A hand motion detection unit (40) provided at a predetermined position above the virtual keyboard unit (30) to detect a user's hand motion entering the virtual keyboard unit (30) by photographing; A position calculation unit (50) for calculating the position of the hand gesture on the virtual keyboard unit (30) from the vector value of the hand gesture detected by the hand gesture detecting unit (40); And when the individual partition information of the virtual keyboard unit 30 is derived from the calculated position of the hand gesture, after checking the individual partition information of the keyboard unit 20 corresponding to the individual partition information of the virtual keyboard unit 30, And a control unit 60 for outputting a control signal for executing character information of the individual sections of the keyboard unit 20.

In addition, the vending unit 20 may include any one of an input unit of an elevator, an ATM, or a vending machine.

In addition, the hand motion detection unit 40 preferably includes at least one or more of the optical proximity sensor or the illumination sensor.

In addition, the vector value may be arithmetic processed by data-forming an image change of the hand movement in a virtual matrix form divided into X, Y, and Z axis regions.

As described above, the input device using the non-contact user interface of the present invention does not directly touch any part of its body to perform a specific purpose. You can easily choose the keyboard.

Accordingly, the present invention can prevent the spread of various viruses and body contamination that can be transmitted by unconsciously contacting a keyboard directly contacted by an unspecified number of people.

In addition, the present invention can prevent the hassle of having to remove the worn protective equipment in order to contact the keyboard when the user wears a protective equipment of the hand such as gloves.

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments of the invention and, together with the description of the invention, It should not be construed as limited.
1 is a perspective view of an input device using a contactless user interface according to an embodiment of the present invention.
2 illustrates a configuration of an input device using a contactless user interface according to an embodiment of the present invention.
FIG. 3 illustrates an image configuration in a virtual matrix form for gesture recognition according to an embodiment of the present invention.

It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention in any way, and the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. There may be a plurality of embodiments of the present invention, and overlapping descriptions may be omitted in the description.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

1 is a perspective view of an input device using a contactless user interface according to an embodiment of the present invention, Figure 2 shows the configuration of the input device using a contactless user interface according to an embodiment of the present invention, Figure 3 According to an embodiment of the present invention, there is shown an image configuration in a virtual matrix form for motion recognition.

1 and 2, in an embodiment of the present invention, an input device using a non-contact user interface includes: a keyboard unit 20 having one or more character information and displaying the character information in separate sections; A virtual keyboard part 30 spaced apart from each other by a predetermined interval directly above the keyboard part 20 to form an area of the individual partitions in a plane corresponding to the individual partitions of the keyboard part 20; A hand motion detection unit (40) provided at a predetermined position above the virtual keyboard unit (30) to detect a user's hand motion entering the virtual keyboard unit (30) by photographing; A position calculation unit (50) for calculating the position of the hand gesture on the virtual keyboard unit (30) from the vector value of the hand gesture detected by the hand gesture detecting unit (40); And when the individual partition information of the virtual keyboard unit 30 is derived from the calculated position of the hand gesture, after checking the individual partition information of the keyboard unit 20 corresponding to the individual partition information of the virtual keyboard unit 30, And a control unit 60 for outputting a control signal for executing character information of the individual sections of the keyboard unit 20.

Conventionally, when selecting a menu item such as an elevator button or an ATM, the user has a trouble in contacting his or her body with a button or a switch used by an unspecified number of people, and also spreads a virus through a button or the like. And problems such as contamination of the body may occur.

However, the input device using the non-contact user interface of the present invention as described above does not directly touch any part of the user's body, which is displayed to perform a specific purpose, and is approached even if the user approaches the keyboard only. There is an effect that the keyboard of the compartment can be easily selected.

The keyboard unit 20 may display one or more character information. Character information may include all Korean, English, numbers, special characters, and the like. The keyboard unit 20 may be displayed in individual sections provided with respective character information.

The individual compartments may be used to perform specific purposes such as elevator buttons or switches indicating the number of floors to be moved, individual menu items for ATM processing, and an input unit of a product vending machine such as a beverage. It may be a selection item of various devices that can input through the movement of a part of the user's body.

Therefore, the keyboard unit 20 according to an embodiment of the present invention may include any one of an input unit of an elevator, an ATM, or a vending machine.

Referring to FIG. 1, the virtual keyboard part 30 mobilized in the present invention may be formed to be spaced apart by a predetermined distance directly above the keyboard part 20. The virtual keyboard portion 30 may be formed as a region of separate virtual individual partitions in a plane corresponding to the individual partitions of the keyboard portion 20. That is, the present invention may be formed up and down with the virtual keyboard portion 30 having the same shape as the keyboard portion 20.

Accordingly, the present invention does not directly contact a part of the body with the keyboard 20 using the virtual keyboard 30, but only close to the keyboard 20, that is, contact with the virtual keyboard 30. Alternatively, the individual compartments to be selected of the keyboard unit 20 can be easily selected through touch.

The distance d between the keyboard portion 20 and the virtual keyboard portion 30 may be freely selected within 0 <d ≦ 10 (unit: cm) in consideration of the user's convenience. However, the virtual keyboard unit 30 may be configured in consideration of the size of individual compartments of the keyboard unit 20 or the interference with other compartments, the accuracy of selection, the user's confusion with an existing touch keyboard, or the prevention of errors in use. It is preferable that it is formed within a short range of ˜3 cm.

The hand motion detecting unit 40 may include a camera capable of detecting a user's hand motion at a predetermined position above the virtual keyboard unit 30. Therefore, the camera may capture the hand gesture of the user entering the virtual keyboard 30.

The hand motion detection unit 40 preferably includes at least one of an optical proximity sensor and an illumination sensor. That is, the camera not only photographs the user's hand movement on the virtual keyboard unit 30, but also detects the position and / or direction of the user's hand through the one sensor or a plurality of combined sensors, and detects the result. To the position calculation unit 50.

In particular, the hand motion detection unit 40 may measure the motion of the hand or finger located on the virtual keyboard 30 to measure the vector value of the motion.

The camera is preferably a depth-sensing camera. Depth detection cameras can detect the movement speed, direction, and trajectory of a hand or finger movement through cognitive characteristics and sensors.

The hand motion detector 40 detects a motion of an object (hand or finger) photographed by a camera and measures a motion (motion) vector from a reference position. Thereafter, the position calculating unit 50 calculates the position of the object (hand or finger) on the virtual keyboard unit 30 from the vector value of the hand gesture detected by the hand gesture detecting unit 40.

The method of recognizing the first finger movement signal using the motion detection camera of the hand motion detection unit 40 flows from each direction (east, west, south, north) in a predetermined input area through an infrared image sensor or pyroelectric sensor. The detection signal inputted by the finger movement can be accepted as an initial stage of internal CPU processing, and the digital 0 signal can be changed to the signal value of 1 to switch from the initial standby state of all sensors or the depth sensing camera to the operating state. .

The hand motion detecting unit 40 includes an optical proximity sensor, an illuminance sensor, a geomagnetic sensor, and the like, for an input signal obtained by using an input device using position information on coordinates of each X, Y, and Z axes in a three-dimensional space. The analogue value is expressed from the standard of.

First of all, it is possible to detect the proximity and distality of the optical proximity sensor and the subject, and the LED configuration of the receiver to the photo transistor and the emitter is based on the operation including the optical sensor and the optical encoder. Up to three or more infrared lights are used to respond to the input due to the detection of the phase difference of the reflection relative to the input of the LED reflected light derived from the movement of the finger.

The recognition angle of the proximity sensor sets the initial reference angle to operate within a certain angle range (e.g., 100 ° ± 10 °) of the left and right, and the recognition distance is the distance from the finger movement (e.g. within 1 ~ 10cm) and the output voltage signal. Can be generated at a constant voltage (e.g. 1.5V) and converted to digital analogue voltage.

The illuminance sensor works with the proximity sensor to change the response of the light intensity to a voltage to maintain a data-formatted configuration of units of measurement intervals (eg 100ms) to minimize sensitivity variations.

Geomagnetic sensor measures the magnetic strength of three axes (X, Y, Z) and outputs it as an analog signal.However, the direction of the camera in terms of input in four directions (East, West, South, North) Data values for left and right upside down within the input voltage range (eg, 2.6V to 3.6V) can be presented for an input reference that is completely different from the input reference.

A method of processing information in three-dimensional space using a depth-sensing camera is a three-dimensional vector operation that recognizes the movement of the left and right movement of the finger to two-dimensional X and Y-axis positions and the pressing position for execution by the depth of the Z-axis. It can be performed by three-axis vector operation.

Among the various methods of vector operation, the decomposition of X, Y coordinates using the vector dot product and the generated normal vector to one side of the space and the space resulting from the vector value of Z orthogonal to the two vectors of X and Y by the cross product operation To have a result of that value.

Referring to FIG. 3, the virtual matrix image configuration of the square model for motion recognition includes (numbers up to 9 including 0 and * and # symbols) on the X axis (numbers) on the Y axis (numbers) on the Z axis (numbers). The image change of the finger movement in the smallest spatial region is converted into data and processed as a vector operation.

As shown in FIG. 3, the basic image filter configuration for the calculation of the minimum motion image change is divided into 48 pieces of 4 × 3 × 4 with respect to the X, Y, and Z axes, and the vector for the image change of the finger motion. Based on arithmetic processing. That is, the vector value may be composed of three-dimensional data for each axis of X, Y, and Z.

Accordingly, the vector value according to an embodiment of the present invention may be arithmetic processed by data-changing the image change of the hand movement in a virtual matrix form divided into X, Y, and Z axis regions.

The more detailed data can be classified, the better the perception sensitivity can be. However, proper division is desirable in consideration of a certain load on the main CPU by repetitive computation.

In particular, assuming a square model of the smallest unit in a three-dimensional space, the top is divided into upper, lower, left, and right directions by X and Y axes, respectively, and the Z axis is recognized as the distance in the direction of the depth sensing camera. Proximity, distal, etc. can be expressed as Z-axis, and can be expressed by setting the maximum and minimum values, respectively. For example, the left side is the minimum value of the X axis, the right side is the maximum value of the X axis, the upper side is the maximum value of the Y axis, and the lower side is the minimum value of the Y axis. In the case of proximity to the camera direction, the Z-axis can be set to represent the minimum value and the distal to the maximum value.

The software program for performing the operation can be based on the implementation of the function for vector operation by distinguishing from the image processing of the right hand finger movement and the left hand finger movement of most of the reference user.

Illustrates the movement of a finger with a single point of recognition, computes the movement trajectory when moving, and the vector V (X, Y, Z) indicating the movement position is converted as a function of the movement time t, and the vector operation The derivative of the scalar variable t of function V is the same as for the scalar function.

(Eg quoting the definition of a vector operation).

Since the user's finger movement is in three-dimensional space rather than in two-dimensional plane on the reference axis X, Y axis, it can be arbitrarily changed, and the error according to the input because the Y axis and Z axis can be changed in conjunction with the X axis. The filtering and scaling process may be additionally performed in order to consider the support of image stabilization, which may involve comparison and control of the input data.

The position calculation unit 50 compares and analyzes the digital signal obtained by conversion from the sensors (optical proximity sensor, illuminance sensor and geomagnetic sensor), the predetermined reference value of the database DB, and the calculation processing value extracted from the depth sensing camera. Can be processed.

More specifically, based on a hypothetical matrix filter of a square model, finger motions are captured through video several times, and the actual measurement data is extracted several times using an application program consisting of a hardware emulator. In other words, software and hardware emulator composed of application based on sensor motion data is used to measure the motion of finger movement obtained on each X, Y, Z axis several times and organize the data by each location. do.

Visual image programs can be created to understand and implement the sensor's behavior with respect to finger movements, and graph changes in the actual measurement data of the sensors.

In particular, the hardware emulator may have an advantage of improving the cognitive sensitivity by adding more detailed functions to determine the type, function, and speed of the sensor while constructing the data to make the data responded to the sensors more sensitive to various finger movements. .

In summary, the program tool is developed to be selected according to the user's definition based on the characteristics of each sensor, and the database is constructed by searching, calculating, comparing and analyzing several times repeatedly based on the initial recognition point of the depth detection camera. The closest point is displayed by comparing it with the video image obtained through the preprogrammed program and the actual depth detection camera.

Using the data obtained from the sensor, it checks for duplicates through a series of processes such as searching, extracting, calculating and comparing the input signal in 3D space, including memory and CPU for storing 3D space information. In addition, the control unit removes a portion which is repeated more than a certain time, and makes use of the characteristics of each sensor to analyze and analyze the data to construct a design input database in order to minimize the error value and to be processed based on the reference.

The communication between the transmitter and the receiver in the data transmission unit is based on RS-232 serial communication. In some cases, if the communication speed is increased or the distance is restricted, it is resistant to noise and environmental effects can be transmitted as far as possible. RS-422 or RS-485 can be used when supplementation is needed. Depending on the application, you can consider using 10/100 Base Ethernet or ISM Band wireless LAN as a freely available frequency band.

The movement of the finger in the input device is recognized as a user interface, and the response of the final recognition process through the position calculating unit and the control unit may be a liquid crystal display, a TV, a game machine, a PC monitor, or the like. However, the present invention is not limited to the above example, and is input at a predetermined distance from the ground including the support in front of the screen in a predetermined input area, and the digital data value and depth converted through the proximity sensor, the illuminance sensor, or the geomagnetic sensor. The sensing camera may be computed and processed through a virtual matrix image filter to compare the extracted values with each other and deliver correction values as final data so that a user may perform desired input information or data.

When the individual partition information of the virtual keyboard unit 30 is derived from the calculated position of the hand gesture, the controller 60 confirms the individual partition information of the keyboard unit 20 corresponding to the individual partition information of the virtual keyboard unit 30. Thereafter, a control signal for executing character information of the individual sections of the keyboard portion 20 is output.

As described above, the present invention is different from the input method of inputting a touch screen of a constant voltage and a constant resistance type using a mouse or a touch panel that presses a button on a plane or scrolls, and moves a finger as a user interface in a three-dimensional space. Depth detection speed and trajectory of the camera Through the cognitive characteristics and sensors of the camera, it is possible to implement a new concept input device technology that recognizes the user's input information by comparing and analyzing the digital numerical information of the analog signal.

In addition, the present invention calculates the trajectory range of the movement of the moving finger passing through the image filter through the depth-sensing camera to minimize the error range shown in the motion recognition-based technology in the conventional three-dimensional space, motion sensor By comparing and analyzing the database of the digitally converted numerical value of the user can more accurately recognize the desired input information.

Finally, the user's finger movements in the three-dimensional space are controlled by passing the final data while controlling the data within the databaseized range by minimizing and correcting the numerical error value through several iterations before the user's input development. It may be characterized by the recognition or display of the desired signal in the type of.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It will be apparent to those skilled in the art that various changes and modifications may be made without departing from the scope of the present invention as defined by the appended claims and their equivalents.

10: input device using a contactless user interface
20: keyboard portion 30: virtual keyboard portion
40: hand motion detection unit 50: position calculation unit
60: control unit

Claims (4)

In the input device using a contactless user interface,
A keyboard unit including at least one switch or button for inputting information into any one of an elevator, an ATM, and a vending machine used in a public place;
A virtual keyboard part serving as a reference for calculating a hand position by forming an area of an individual partition so as to correspond to an area of a switch or button included in the keyboard part at a height spaced vertically by the keyboard part;
A hand motion detector arranged adjacent to the keyboard to detect a hand motion entering an area of an individual compartment of the virtual keyboard to generate a vector value, and to photograph a finger movement;
A data transfer unit which provides information generated by the hand motion detection unit to the outside through wireless communication;
A position calculation unit for calculating a hand gesture position on an area of the individual section based on the vector value and the photographed image provided from the data transfer unit; And
And a controller for matching a hand motion position on an area of an individual compartment provided by the position calculator with the keyboard to output a control signal for executing information corresponding to a switch or a button of the keyboard. delete The method of claim 1,
The hand motion detection unit 40 is an input device using a non-contact user interface, characterized in that at least one of the optical proximity sensor or the illumination sensor. The method of claim 1,
The vector value is an input device using a non-contact user interface, characterized in that the image processing of the hand motion movement in the virtual matrix form divided into the X, Y, Z axis region is processed by the data.

Images