WO2012030153A2

WO2012030153A2 - Non-contact type input device

Info

Publication number: WO2012030153A2
Application number: PCT/KR2011/006428
Authority: WO
Inventors: 유병문; 황두성
Original assignee: 주식회사 엘앤와이비젼
Priority date: 2010-09-02
Filing date: 2011-08-31
Publication date: 2012-03-08
Also published as: WO2012030153A3

Abstract

The present invention relates to a non-contact type input device. The non-contact type input device includes a main body having a predefined virtual input space area at one side; a physical display device installed at another side of the main body; at least two cameras installed at the main body to face the virtual input space area and generate a capture image capturing a position or movement of a touch means in the virtual input space area; and an image processing unit for calculating position coordinates of the touch means by analyzing the capture image and converting the position coordinates of the touch means into projected position coordinates on the display device. According to the present invention, without any physical contact with an actual input device, a user may input a key value or signature. Therefore, disease (germs) infection of a user due to an indirect contact in crowded places such as hospitals, banks, marts, and restaurants may be prevented.

Description

Contactless input device

The present invention relates to a contactless input device, and more particularly, when a user's finger touches an invisible virtual input surface or signs in a virtual input space area, the user touches an actual physical display device to make an input. As the key value is input or the signature is displayed on the display device, it is possible to prevent personal information exposure such as disease transmission or fingerprint due to contact, and to use for signature inquiry during electronic payment by using signature characteristic information for each user. It is about.

With the rapid development of electronic and information communication technology, there are no fields where electronic devices such as computers are not utilized, and the touch means (devices) of the electronic devices also vary.

Input devices, such as touch screens, keyboards, keypads, mice, push buttons, light pens, digital tablets, and trackballs, which are widely used at present, are a touch type in which a user inputs data through direct physical contact with the input device.

Keypad related to the present invention is used in various fields, such as digital door lock (ATM), automatic teller machine (ATM), credit authorization terminal (Credit Authorization Terminal), a button type and a touch screen using a physical button ( Touch screen) is used and all are touch type.

However, such a touch keypad 1) As a user enters data through physical contact with the keypad serves as a medium for transmitting diseases (germs) caused by indirect contact in places where many mobile customers such as hospitals, banks, marts, restaurants, etc. , 2) there is a risk of exposing information such as a password due to wear due to frequent mechanical contact, and 3) there is a problem of exposing personal information such as fingerprints of a user to the input device.

In relation to the contactless keypad, Korean Patent No. 10-0894544 projects an optical plane including a keypad pattern in the air, and when a finger is positioned at a specific position of the projected optical plane, the position of the finger using the IR light source unit and the IR receiver unit It is possible to use a small and semi-permanent use without a mechanical key input device by detecting the key input is made, and a technology that can be applied to a door lock (digital home appliance) keypad, etc. is disclosed, Figure 1 A schematic diagram is disclosed.

Referring to FIG. 1, the space touch screen device according to the registered patent includes a pattern projection unit 10, an IR light source unit 20, an IR sensing unit 30, a signal processing unit 40, and an optical plane 50. .

The pattern projection unit 10 generates a character or an image and displays it in the air, and the optical plane 50 is an image projected in the air through the pattern projection unit 10 to receive input from a user. It may be a keypad, a still image, or the like.

The IR light source unit 20 irradiates an IR optical signal to detect a touch signal provided from a user in the optical plane 50, and independently modulates IR light for each channel to correspond to each input button displayed on the optical plane 50. The signal is generated and provided to the optical plane 50, thereby maintaining the optical plane 50 in an input ready state.

The IR detector 30 amplifies and demodulates and demodulates the reflected IR optical signal generated by touching a hollow keypad and an image provided on the optical plane 50 to the signal processor 40, and the signal processor 40 transmits the IR signal. The IR optical signal transmitted from the sensing unit 30 is calculated and processed to find an input button touched by the user, and the output signal for the input signal is transmitted to the pattern projection unit 10.

The registered patent can be used semi-permanently because it does not require a mechanical key input device by providing a non-contact keypad, and can be applied to a door lock or a keypad of a digital home appliance, but a projection device for generating an optical plane 50 is required. In addition, since the number of IR light source units 20 and IR detectors 30 corresponding to each input button displayed on the optical plane 50 is required, the manufacturing cost is high and it is difficult to miniaturize.

Meanwhile, as the modern society has progressed to a credit society and an information society, payment methods such as credit card, electronic money, and electronic payment have become commonplace, and user authentication methods for verifying user identity of a payment system are also various. Information for the electronic payment system to verify the user's identity may use information such as the user's advice, iris, DNA information, password, signature, etc.User's password or signature due to the convenience of information acquisition or technology implementation The method is widely used.

In the password input method, a user inputs a password by touching a button on the keypad or a button on the touch screen. In the method using a signature, the user inputs his or her signature on the touch screen using a touch pen or a pointed object. That's the way it is.

Touch signature pads, which use a touch pen or pointed object to enter a user's signature on the touchscreen, allow the user to enter data through physical contact with an input device, such as a touch pen, for hospitals, banks, marts and restaurants. There is a risk that mobile customers are exposed to the risk of transmitting diseases (germs) by indirect contact in many places, and the life of the device is shortened due to wear caused by mechanical contact between the touch pen and touchsterin. have.

The present invention has been made to solve the problems of the conventional contact keypad as described above, by using two or more cameras to form an invisible input space area on the upper portion of the physical display device contactless key selection input or signature This makes it possible to prevent personal information such as disease transmission or fingerprints caused by contact.

Another object of the present invention is to enable the user to use the identity check when making an electronic payment using the characteristic information when the user signs in the virtual three-dimensional signature space area.

According to a preferred embodiment of the present invention for achieving the above object, a main body having a predefined virtual input space area on one side, a physical display device installed on the other side of the main body portion, toward the virtual input space area Two or more cameras installed in the main body to generate a captured image photographing the position or movement of the touch means in the virtual input space region and the captured image to analyze the position coordinates of the touch means; And an image processing unit for converting the position coordinates of the touch means into projection position coordinates on the display device according to the set mapping information between the virtual input space area and the display device.

According to the present invention, it is possible for a user to input a key value or a signature without any physical contact with an actual input device, so that the user's disease due to indirect contact at a place where a lot of mobile customers such as hospitals, banks, marts, restaurants, etc. ) It is effective to prevent the transmission.

In addition, since the present invention is made in a non-contact manner, there is no mechanical friction between the touch pen and the touch screen in the keypad or signature pad, thereby extending the mechanical life.

In addition, when the input device is a signature pad, the user can use the characteristic information when the user signs in the virtual three-dimensional signature space area so that the user can use it for identity check during electronic payment.

1 is a configuration diagram of a space touch screen device according to Korean Patent No. 10-0894544.

Fig. 2 is a block diagram of a contactless keypad device according to the first embodiment of the present invention.

3 is a two-dimensional block diagram of the contactless keypad device according to the first embodiment of the present invention.

4 is a flowchart illustrating a process of performing virtual keypad setting in the contactless keypad device according to the first embodiment.

FIG. 5 is a flowchart illustrating a process of performing a method of detecting a contactless key input using the contactless keypad device according to the first embodiment.

6 is a block diagram illustrating a contactless signature pad according to the second embodiment.

7 is a perspective view showing a schematic configuration of a contactless signature pad according to a second embodiment.

8 illustrates a signature pattern on a virtual signature space area and a signature pattern projected onto a display device.

9 shows an example of the installation state of the frame and the display device.

10 illustrates a principle of generating mapping information using a calibration tool.

11 is a flowchart illustrating a process of generating mapping information using a calibration tool.

12 is a flowchart illustrating a process of receiving a signature in a contactless manner from the contactless signature pad according to the second embodiment and expressing the signature.

Hereinafter, with reference to the accompanying drawings will be described in detail preferred embodiments of the present invention. In the present embodiment, as a contactless input device, a keypad will be described as an example in the first embodiment and a signature pad in the second embodiment, but the technical concept of the present invention is not limited thereto. Of course it includes all of the non-contact input device that can be made contactless input by detecting the position coordinates in the input space of the.

First embodiment: contactless keypad

2 is a block diagram of a contactless keypad device according to a first embodiment of the present invention, Figure 3 is a two-dimensional block diagram of a contactless keypad device according to a first embodiment of the present invention.

2 and 3, the contactless keypad device according to the first embodiment of the present invention is the body 1, those installed on the body 1, the physical keypad 100, the camera 200, The interface unit 300 includes an image processing unit 400, a storage unit 500, an output unit 600, and a control unit 700.

The physical keypad 100 is for visually showing the keypad shape, and may be a physical key input device such as a physical button or an LCD screen. If the visual keypad is provided to the user, the physical keypad 100 may be printed in various forms such as paper with a keypad pattern printed thereon. Means are available.

The camera 200 is installed on the upper portion of the physical keypad 100 to capture an image of the upper portion of the physical keypad 100. The two cameras are combined to generate a stereo image.

The interface unit 300 is for connection with a server (not shown) connected with the keypad device.

The image processor 400 detects the position of the touch means such as the user's finger from the image photographed by the camera 200 and corresponds to the position of the touch means on the virtual keypad area 800 which describes the position of the detected touch means. A function of determining the key 810 is performed.

The storage unit 500 includes a virtual keypad area 800 and a virtual keypad area, which are virtual input spaces located in a predetermined area above the physical keypad 100, information about the size of the physical keypad 100 and the keys 110. Position information of the key 810 in the 800 and information for converting the virtual keypad area photographed by the camera 200 into the actual virtual keypad area 800 are stored.

Here, the virtual keypad area 800 is a virtually defined area corresponding to the physical keypad 100 on the upper predetermined height h of the physical keypad 100 for detecting a contactless key input of the touch means. It is invisible.

The output unit 600 generates sound or light to give a key touch when the touched key 810 on the virtual keypad area 800 is determined by the image processor 500. When it is determined that a specific key 810 on the image is touched, a sound or light is generated to change the color or brightness of the key 110 on the corresponding physical keypad 100 or give a feeling of touch.

The control unit 700 is a part in charge of the overall control of the contactless keypad device using the camera according to the present invention.

In addition, the dam 900 may be installed at one side of the main body 1 for stability of the background when the stereo image is acquired.

Referring to FIG. 4, as shown, first, information of the physical keypad 100 is obtained and stored (S400). The physical keypad information may include the size of the physical keypad 100, location information of keys, and the like.

When the information of the physical keypad 100 is obtained, the camera 200 may be touched simultaneously or continuously by touching a plurality of points of the physical keypad 100 using a calibration tool having a marker formed at a point corresponding to the height of the virtual keypad area. The marker is detected from the stereo image captured by the controller to determine and store information about the virtual keypad area 800 (S410).

Next, the spatial correlation of the camera 200 with respect to the physical keypad 100 or the weather keypad area 800 is calculated and stored using the detected marker information (S420). Here, the spatial correlation may be the relative position or relative angle of the camera 200 with respect to the virtual keypad area 800 or the physical keypad 100.

Finally, the conversion information is generated using the marker information obtained by touching a plurality of points of the physical keypad 100 using the calibration tool and the marker information obtained from the image information acquired from the camera 200. Save (S430). Here, the transform information is a matrix for transforming between the actual coordinate information and the coordinates when viewed by the camera 200, and it is well known to those skilled in the art to generate the transform matrix and its inverse transform matrix, and thus detailed description thereof will be omitted. .

If a virtual keypad area 800 located in a predetermined area on the upper part of the physical keypad 100 is defined through the process of FIG. 4, a process of detecting a non-contact key input as shown in FIG. 5 is performed.

The image of the upper portion of the physical keypad 100 is continuously photographed by the camera 200 to determine whether the touch means is detected.

If the touch means is detected, when the touch means is located in the virtual keypad area 800 based on the captured image of the camera 200, the touch area of the touch means is detected (S500).

When the touch area of the touch means is detected (S500), the touch position on the virtual keypad area 800 is determined based on the detected area (S510).

Based on the detected position on the virtual keypad area of the touch means, the key 810 on the nearest virtual keypad area 800 is found (S520), and the distance between the touch position and the corresponding key 810 is compared and the distance is a reference value. If it is less than or equal to recognize the selected key (S530), by selecting the corresponding key to transmit a key signal to the server, in response to this process to change the color or brightness of the key 110 on the physical keypad 100 or touch Generates sound or light to give a feeling of (S540).

Second Embodiment: Contactless Signature Pad

6 is a block diagram illustrating a contactless signature pad according to a second embodiment, and FIG. 7 is a perspective view illustrating a schematic configuration of a contactless signature pad according to a second embodiment.

6 and 7, the contactless signature pad according to the second embodiment may be largely signed by a user through a virtual signature space area S as a virtual input space, and the input signature may correspond to the display area of the display device. It is composed of a non-contact signature unit (1) for converting the position coordinate to be provided to the display device 2 and a physical display device (2) to visually express the user's signature.

The non-contact signature unit 1 includes a main body 40 having an invisible virtual signature space area S formed on one side, two or more cameras 10 installed on one side of the main body 40, and a virtual signature space area S on one side. And an image processor 20 for converting the signature pattern into position coordinates of the display device 2 and an output unit 30 for generating sound or light to give a touch when a signature is input.

The two or more cameras 10 are installed in the main body 40 so as to face the virtual signature space area S, and photograph the movement of the signature input means in the virtual signature space area S. The two cameras are combined. Generates a captured image. The main body 40 is formed to surround four surfaces of the virtual signature space area S, and the two or more cameras 10 are installed on one of four surfaces surrounding the virtual signature space area S, so that two or more cameras ( In 10), the image can be obtained stably. The frame 40 is formed so as to surround the four sides of the virtual signature space area S to acquire an image more stably. The virtual signature space area S is not surrounded by the main body 40 and two or more cameras are provided. (10) It is, of course, also possible to be formed on the open space in front.

The image processing unit 20 is installed inside the main body 40, and continuously analyzes the captured images generated by the two or more cameras 10 to continuously calculate the position coordinates of the signature input means, and sets a predetermined virtual signature space area. Signature pattern data obtained by converting the position coordinates of the signature input means calculated according to the mapping information between the display apparatuses into the projection position coordinates on the display apparatus 2 is generated.

Although not shown, the image processing unit 20 may include mapping means for performing mapping processing between the physical display device 2 and the virtual signature space area S, and signature input means from images acquired from two or more cameras 10. Position determination means for extracting the position and position determination means for determining the position of the signature input means in the virtual signature space area (S) by using the position information of the two or more cameras 10 may be further included.

When the signature is input, that is, when the entry of the signature input means is detected in the virtual signature space area S, the lamp 31 or the lamp 31 expressing the signature initiation or signature input means detection as light or the sound is input. And a speaker 32 for outputting.

The display device 2 is a device for displaying the signature pattern received from the image processing unit 20, and the display unit 21 is formed at one side. The display unit 21 may be an LCD, and since the signature is made in a non-contact manner, it does not need to be a touch screen such as a conventional signature pad.

The non-contact signature unit 1 and the display device 2 may be formed integrally, but more preferably formed to be separated from each other for convenience of use. In the latter case, it is necessary to combine the wires or wirelessly between the two.

In addition, in the present exemplary embodiment, the image processing unit 20 and the output unit 30 are illustrated in the main body 40. However, the image processing unit 20 and the output unit 30 are installed in the display device 2 or installed in a separate host device. Of course, the generated captured image may be analyzed by the host device to map the location coordinates and then transmitted to the display device 2.

As shown in FIG. 8, when the user places a finger 60 in the virtual signature space area S and signs in the space, the signature image is captured by two or more cameras 10, and the image processing unit ( A signature pattern is generated by detecting a continuous change of position of the finger 60 at 20), and the generated signature pattern is projected on the display unit 21 of the display device 2 and displayed. At this time, as will be described later, the thickness of the signature pattern 70 is varied according to the height of the finger 60 (z value in the virtual signature space area S) during the signature operation so as to be similar to the actual signature of the user. It is desirable to.

9 shows an example of the installation state of the main body and the display device.

One of the advantages of the contactless signature pad according to the second embodiment is that the main body 40 can be installed separately from the display device 2. The virtual signature space area S is defined in the open space formed in the main body 40 through the calibration process using the calibration tool 50, and the virtual signature space area S and the actual physical display device 2 are defined. This is because the mapping information is set in advance. Therefore, according to the present invention, the main body 40 does not need to be installed integrally with the display device 2, and can be installed and used at any position or angle that is convenient for use.

As an example of such a free installation, in FIG. 9, the display device 2 is vertically installed, and the main body, that is, the non-contact signature part 1, is installed at an angle of 45 ° in front of the display device 2, so that the user can hold the finger 60. FIG. 2 shows an installation example that makes it easy to sign using a reference and a user can easily see the signature pattern displayed on the display device 2.

In addition, various types of arrangements are possible, such as arranging the display device 2 horizontally, placing the non-contact signature portion 1 at any height therefrom, or arranging the two devices in parallel and horizontally.

FIG. 10 illustrates a principle of generating mapping information using a calibration tool, and FIG. 11 is a flowchart illustrating a process of generating mapping information using a calibration tool.

As shown in FIG. 10, the calibration tool 50 used in the present invention has a structure similar to a cube, and a marker 51 is formed at eight vertices of the cube, and the markers 51 are connected to the connecting rod 52. Is connected, the lower end of the calibration tool 50 is formed with a leg portion 53 to be spaced apart from the bottom by a predetermined distance. The height of the leg 53 may be set equal to the distance between the display device 2 and the virtual signature space area.

Referring to FIG. 11, the mapping information generation process is described. First, the calibration tool 50 of FIG. 10 is installed to contact four vertices of the display apparatus 2 (S600). At this time, the cube area corresponding to the virtual signature space area by the leg portion 53 of the calibration tool 50 is positioned to be spaced apart from the display device 2 by a predetermined height.

Next, the calibration tool 50 is photographed using two or more cameras 10 (S610), and the virtual signature space area S that is invisible is determined from the photographed images photographed by the two or more cameras 10. (S620). That is, the position information of the eight markers 51 is obtained from the captured image, and the cubic space formed by the eight markers 51 is defined as the virtual signature space area S.

When the virtual signature space area S is determined, two or more cameras for the physical display device 2 or the virtual signature space area S are obtained using marker information obtained from an image captured by the two or more cameras 10. The relative position of 10) is calculated (S630).

Finally, transformation information is generated by using the position information of the marker 51 of the preset calibration tool 50 and the position information of the marker 51 extracted from the image information acquired from the two or more cameras 10, and then a virtual three-dimensional image. The mapping relationship between the signature space area and the two-dimensional display apparatus 2 is determined (S640).

Referring to FIG. 12, the image of the virtual signature space area S is continuously photographed by two or more cameras 10 to determine whether a signature input means such as a finger is detected (S700).

If the signature input means is detected, the touch area of the signature input means is extracted and the position of the signature input means is detected in the touch region of the extracted signature input means (S710). Detecting the position of the signature input means is performed for the purpose of tracking the change in the coordinates of the user's finger to generate the signature pattern.

When the position of the signature input means is detected, a three-dimensional touch position is calculated using a trigonometry (S720). That is, a vector is created that connects the positions of the two or more cameras 10 and the positions of the signature input means on the virtual signature space area S, and the vectors of the two vectors connecting the positions of the two cameras and the positions of the signature input means. The intersection point is recognized as a touch position on the virtual signature space area S. FIG.

When the three-dimensional touch position is calculated, the user's finger position (x, y coordinate) determined in the invisible virtual signature space area S is determined by the projection position on the display device 2 using the mapping information calculated in the setting step. It is determined (S730).

Then, the thickness of the signature displayed on the display device 2 is determined by the height position (z coordinate) of the user's finger (S740). That is, as the height position of the user's finger, i.e., the z-coordinate is smaller, the user's finger is located deeper in the virtual signature space area S, so that the signature pattern is expressed thicker. Since the height position of the user's finger is continuously variable during the signature operation, the thickness of the signature pattern is variably expressed accordingly, so that a signature pattern which is very similar to the actual signature of the user can be obtained.

When the continuous position coordinates of the signature pattern are obtained, a smoothing process is performed as a post-process to prevent discontinuity due to a mathematical error generated in the process of digitizing the continuously projected finger position and thickness information (S750). Smoothing converts the continuously projected position coordinates and thickness information into digital information, and filters the plurality of adjacent points to form a line connecting a plurality of points. It is done by changing it according to the form.

When the smoothing process is completed, the signature pattern data converted according to the mapping information is transmitted to the display device 2 and displayed on the display unit 21 (S760).

When the signature pattern is displayed, signature characteristic information is generated (S770), and the generated signature characteristic information is transmitted to the electronic payment system and used for a user identity inquiry (S780). The signature characteristic information may include information such as signature pattern, signature height, signature speed, signature direction information, and the like, and the image processor 20 analyzes the photographed image to calculate such information and generates signature characteristic information therefrom.

In order to be able to query the user's identity using the signature characteristic information, information about the user's signature, that is, signature pattern information or signature characteristic information, should be stored in advance in the electronic payment system.

The present invention is an invention that allows the input of the keypad input or the signature pad in a non-contact manner, it is a very useful invention that can be applied to the keypad or signature pad used in banks, card merchants and the like.

Claims

A main body having a predefined virtual input space region on one side;

A physical display device installed at the other side of the main body;

Two or more cameras installed in the main body so as to face the virtual input space area and generating a captured image photographing the position or movement of the touch means in the virtual input space area; And

An image processor configured to calculate the position coordinates of the touch means by analyzing the photographed image, and convert the position coordinates of the touch means into projection position coordinates on the display apparatus according to preset mapping information between the virtual input space region and the display apparatus; Non-contact input device comprising a.
The method of claim 1,

And a storage unit which stores information about the physical display device, information about a virtual input space area, and conversion information for converting the virtual input space area photographed by the camera into a display area of an actual display device. Contactless input device.
The method of claim 1,

The display device is a keypad having a key pattern, and the image processor detects a touch area of the touch means from the captured image, and determines a key corresponding to the touch area of the touch means on a virtual input space area. Contactless input device.
The method of claim 3, wherein

And an output unit for generating sound or light to give a key touch when the touched key on the virtual input space area is determined by the image processor.
The method of claim 3, wherein

One side of the physical keypad is a contactless input device, characterized in that the dam having a height corresponding to the height of the virtual keypad is installed.
The method of claim 3, wherein

The virtual keypad area and the conversion information

Detecting the marker from the image taken by the camera while touching the physical keypad using a calibration tool formed with a marker to determine a virtual keypad area as a virtual input space area, and using the information of the detected marker The relative position or relative angle of the camera with respect to the virtual keypad area or the physical keypad is calculated, and converted using the marker information extracted from the image information obtained from the camera and the marker information when the physical keypad is touched with a calibration tool. Non-contact keypad device using a camera, characterized in that calculated by generating and storing information.
The method of claim 1,

The image processing unit continuously calculates the position coordinates of the touch means, and converts the calculated position coordinates of the touch means into projection position coordinates on the display device according to preset mapping information between the virtual input space area and the display device. And generating a converted signature pattern data, wherein the display apparatus is a sign pad displaying a signature pattern received from the image processor.
The method of claim 7, wherein

The image processor

And a height information of the signature input means is continuously calculated from the photographed image, and the height information is converted into thickness information of a signature pattern.
The method of claim 7, wherein

The image processing unit converts the continuously projected position coordinates and thickness information into digital information, and performs filtering on a plurality of adjacent points to smooth the signature pattern.
The method of claim 7, wherein

The image processing unit calculates the signature pattern, signature height, signature speed, and signature direction information of the signature input unit from the photographed image, and transmits the signature pattern information to the electronic payment system so as to be used as user identity inquiry information. .
The method of claim 7, wherein

The mapping information is

Position a calibration tool having a marker formed at a point corresponding to each vertex of the virtual signature space area to be spaced apart from the display device by a predetermined height,

Determine an invisible virtual signature space region from the captured images taken by two or more cameras,

Calculate relative positions of two or more cameras with respect to a physical display device or virtual signature space region using marker information obtained from images captured by the two or more cameras,

The conversion information is generated using the marker position information of the calibration tool and the marker position information extracted from the image information acquired from the two or more cameras, and the mapping relation between the virtual three-dimensional signature space region and the two-dimensional display apparatus is determined. A contactless signature pad, characterized in that obtained.