KR101358422B1 - Device for character input - Google Patents

Abstract

A processing information device comprises an input unit identification unit, an interface control unit, a search unit, and a character information display unit. The input unit identification unit enables a user to identify the kinds of a finger which performs input processing from the input data of the user inputted through a character input interface device. The interface control unit displays a graphic user interface on a display of the information processing device. The graphic user interface displays character information which is matched according to a combination state of the finger or the kinds of the finger through a point guide. The search unit searches the character information matched to the combination of the finger identified by the input unit identification unit. The character information display unit displays the character information searched through the search unit on the display. The point guide comprises a point which guides finger input and a connection line which guides each point. The length of the connection line is increased in comparison to point intervals. [Reference numerals] (100) Character input device; (300) Information processing device

Description

Character input device {DEVICE FOR CHARACTER INPUT}

The present invention relates to a character input device.

In keyboards for text input of computers, smart devices, ATMs, etc., a one-to-one keyboard with one button corresponding to a single letter or a text input keyboard using a 3x4 array of numeric keypads mainly composed of an electronic telephone is generally used. It is used. However, the one-to-one type keyboard has to use both hands and the keyboard buttons are configured as many as the number of characters, which takes up a lot of space, and the keyboard using the numeric keypad has a disadvantage of pressing a button several times for character input.

In addition, computers are increasingly diversified in size, shape, interface, and the like, such as tablet PCs, kiosks, and wearable computers, but keyboards are still using conventional formats. In particular, as the development of smart devices using touch screens and sensors has been actively conducted, various user experiences and conveniences have been provided, while the development of character input keyboards is very insufficient.

Therefore, it is necessary to develop a keyboard that complements the disadvantages of the conventional keyboard and provides the user with convenient usability by reflecting the characteristics of the touch screen and the sensor. In particular, there is a demand for development of a keyboard that can conveniently enter characters with one hand without taking much space in keyboard configuration.

In this regard, Korean Patent Laid-Open Publication No. 2011-0050356 (name of the invention: an apparatus and method for processing touch information of a touch screen device) includes input of a keyboard or a mouse corresponding to change information of a predefined coordinate or rotation information of an acceleration meter. A touch information processing apparatus and a method including a database storing input conversion information as information and receiving a plurality of touch information from a touch screen device and converting the same into input of an input device are proposed.

The present invention is to solve the above-described problems of the prior art, some embodiments of the present invention can identify the input combination of five fingers to input characters, an interface that can intuitively check the character input and character input process It is intended to provide a device for configuring.

As a technical means for achieving the above-described technical problem, the information processing apparatus according to an embodiment of the present invention is to identify the type of the finger, the user performed the input processing from the user's input data input through the character input interface device An interface control unit for displaying on the display of the information processing apparatus a graphic user interface indicating, via a point guide, character information matched according to an input means identification unit, a type of finger, or a combination state of the fingers; A search unit for searching for text information matching the combination of the fingers, and a text information display unit for displaying the text information searched through the search unit on the display, wherein the point guide includes a point for guiding a finger input and a combination of each point. Connecting line to guide the Including, but the length of the connecting line may increase in proportion to the point spacing.

In addition, the character input device according to another embodiment of the present invention is a touch screen, a camera coupled to a position adjacent to the edge of the touch screen, coupled to the camera adjacent to the direction of the camera is coupled to the position adjacent to the camera, A mirror for guiding a user's touch action, a touch point display unit for outputting a marker for highlighting a corresponding point at a point where a touch on the touch screen occurs when a touch action on the touch screen occurs, a touch action on the touch screen An input means identification unit for identifying a type of a finger performing a touch operation based on the captured image, a search unit for searching for character information corresponding to the finger identified through the input means identification unit, and an output for outputting the character information It may include wealth.

According to the problem solving means of the present invention described above, since a character matching a five-finger input by a single and two or more combinations can be input, all the predetermined characters can be input using one hand.

In addition, according to the above-described problem solving means of the present invention, by providing an intuitive user graphic interface that can check the user's hand and whether the recognition, the input character through the screen can increase the usability and convenience of the user.

1 is a block diagram showing a character input system through a character input device according to an embodiment of the present invention.
2 is a block diagram illustrating a configuration of an information processing apparatus according to an embodiment of the present invention.
3 is a block diagram illustrating a configuration of a character input apparatus according to another embodiment of the present invention.
4 is a block diagram illustrating a configuration of an input means identification unit in the character input apparatus according to another embodiment of the present invention.
5 is an example of a graphical user interface illustrating a combination of point guides in accordance with one embodiment of the present invention.
6 illustrates an example of highlighting a combination point input by a user in a graphical user interface according to an embodiment of the present invention.
7 is an example for explaining matching of a graphic user interface and text information according to an embodiment of the present invention.
8 is an example for describing a character input method using a character input apparatus according to another embodiment of the present invention.
FIG. 9 is an example for explaining a method of extracting a finger point using a 3D depth sensor in a text input device according to another embodiment of the present invention.
FIG. 10 is an example for explaining a method of extracting a finger point using a 3D depth sensor in an information processing apparatus according to an embodiment of the present invention.
11 is an example for describing an input method using a physical button in an information processing apparatus according to an embodiment of the present invention.
12 is an example for explaining the configuration of a character input system through a character input apparatus and an information processing apparatus according to an embodiment of the present invention.
FIG. 13 is a flowchart illustrating a text input method through an information processing device according to an embodiment of the present invention.
14 is a flowchart illustrating a text input method through the text input device according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between . Also, when an element is referred to as "comprising ", it means that it can include other elements as well, without departing from the other elements unless specifically stated otherwise.

1 is a block diagram showing a character input system through a character input device according to an embodiment of the present invention.

The character input system 400 according to an embodiment of the present invention includes an information processing apparatus 300 and a character input apparatus 100.

The information processing apparatus 300 may receive a text input through the text input apparatus 100. In this case, the information processing apparatus 300 may display the character information matched according to the type of the finger or the combination state of the fingers through a graphic user interface indicating through the point guide.

In addition, the text information output through the text input apparatus 100 may be transmitted to the information processing apparatus 300. In this case, the text information output from the text input apparatus 100 may be data generated by recognizing a user's finger or data matched with a combination of two or more points, or may include text data encrypted with text information.

Meanwhile, although FIG. 1 shows the information processing apparatus 300 and the character input apparatus 100 separately, the character input system 400 performs a table top and kiosk that simultaneously perform information processing and character input. Device), and the like.

2 is a block diagram illustrating a configuration of an information processing apparatus according to an embodiment of the present invention.

At this time, in an embodiment of the present invention, it will be described as an example that the information processing device 300 is a multimedia device (for example, a smart TV).

For reference, FIG. 2 illustrates the configuration of the information processing apparatus 300 through finger recognition according to an embodiment of the present invention. However, other processing units (not shown) may be further included according to the type of the apparatus.

In addition, the information processing apparatus 300 according to an embodiment of the present invention may be various types of devices, not limited to a multimedia device, and may further include different processing units according to the type and purpose of each device.

The information processing apparatus 300 according to an embodiment of the present invention may include an input means identification unit 310, a text information storage unit 320, a search unit 330, an interface control unit 340, and a text information display unit 350. Include.

The input means identification unit 310 may identify the type of the finger that the user has performed the input process from, the user's input data input through the text input interface device 360.

For example, when the text input interface device 360 is the 3D depth sensor 361, the endpoint of the finger may be extracted based on the depth information of the finger detected by the 3D depth sensor 361. For example, five points discontinuous within the close range from the 3D depth sensor 361 can be extracted as the end points of five fingers. The extracted five points may be distinguished by the end points of the respective fingers, and the finger performing the touch operation may be identified according to the change and the correspondence of the depth information. A more detailed description thereof will be described later with reference to FIG. 10.

In addition, when the character input interface device 360 is the physical button 362, the input finger of the physical button 362 may be mapped to the input of one finger of the user to identify the input finger. For example, when the physical button 362 corresponding to the thumb and the stop is input by the user, the input means identification unit 310 may identify that the thumb button and the stop button are input through the input data. A more detailed description thereof will be described later with reference to FIG. 11.

The text information storage unit 320 may store text information corresponding to the finger identified through the input means identification unit 310. That is, character information matched according to each point corresponding to the thumb, the index finger, the middle finger, the ring finger, and the hand and a combination of two or more points may be stored.

The searcher 330 may search for text information that matches a combination of fingers identified by the input means identification unit 310. That is, the character information corresponding to the combination of the fingers identified by the input means identification unit 310 may be retrieved from the character information storage unit 320.

The interface controller 340 may display, on the display of the information processing apparatus 300, a graphical user interface indicating character information matched according to the type of finger or the combination state of the fingers through a point guide. In addition, according to the user's input processing, the selected point and connection line and the text information matched thereto may be displayed in real time. In this case, the point guide includes a point for guiding a finger input and a connecting line for guiding a combination of each point, but the length of the connecting line may increase in proportion to the interval between points, and the character and arrangement matched according to the combination of the points may be used. It can be set by.

The text information display unit 350 may display text information searched through the search unit 330 on the display.

In addition, the information processing apparatus 300 may further configure the external character input interface device 360 to receive data for character input from the external character input intern interface device 360.

The text input interface device 360 may be a 3D depth sensor 361 that detects a user's finger touch operation or may be a physical button 362 composed of a plurality of distinct buttons corresponding to each finger. In addition, it may include a device that can distinguish the type of finger and input the input data.

3 is a block diagram illustrating a configuration of a character input apparatus according to another embodiment of the present invention.

At this time, in an embodiment of the present invention, it will be described as an example that the character input device 100 is a portable communication device (for example, a tablet PC).

For reference, FIG. 3 illustrates the configuration of the character input apparatus 100 through finger recognition according to an embodiment of the present invention. However, other processing units (not shown) may be further included according to the type of the apparatus.

In addition, the character input apparatus 100 according to an embodiment of the present invention may be various types of devices, not limited to a portable communication device, and may further include different processing units according to the type and purpose of each device. For example, when the character input device 100 is an ATM device used in a bank, a communication module connected to a bank computer, a computer processing unit that performs a bank computer operation for each account, and a user may use information to process a bank business. It may be configured to further include a plurality of processing units, such as a user interface module for receiving input or output the banking business processing results.

Character input device 100 according to an embodiment of the present invention is a touch screen 110, camera 120, mirror 130, 3D depth sensor 140, touch point display unit 150, input means identification unit ( 160, a search unit 170, an output unit 180, a text information storage unit 190, and an interface processor 200.

The touch screen 110 may be located in front of the text input apparatus 100, and a marker for highlighting a point generated by a user's touch may be generated and displayed. For example, when the user touches the middle finger and the ring finger at the same time, a marker for highlighting a point where the middle finger and the ring finger touches may be displayed on the touch screen 110.

The camera 120 may be coupled to a position adjacent to the edge of the touch screen 110 and capture a user's touch motion reflected by the mirror 130. The captured touch operation image may be used to extract the end point of the user's finger, and the end point of the finger may be used as information for identifying the finger. A process of identifying the input means through the captured image will be described later with reference to FIG. 7.

The mirror 130 may be coupled to intersect the photographing direction of the camera 120 at a position adjacent to the camera 120 to guide the camera 120 to capture a user's touch operation. That is, the touch motion of the user who is out of the photographing angle of the camera 120 may be deflected by using the mirror 130 to guide the touch motion to be included in the photographing angle of the camera 120. An example of the configuration of the mirror 130 and the captured image in the character input device 100 will be described later with reference to FIG. 8.

The 3D depth sensor 140 may be coupled adjacent to the point where the camera 120 is coupled, and may extract the end point of the finger based on the depth information detected by the sensor. For example, the point where the camera 120 is coupled to the position facing the user's touch movement in front of the point adjacent to the point where the camera 120 is coupled, and the five points discontinuous within the close distance from the 3D depth sensor 140, the end point of the five fingers. Can be extracted with An end point of the finger detected by the 3D depth sensor 140 may be used as information for identifying a finger, and a more detailed process will be described later with reference to FIG. 9.

The touch point display unit 150 may output a marker for highlighting a corresponding point at a point where a touch on the touch screen 110 occurs when a touch operation on the touch screen 110 occurs. That is, the marker for highlighting the touch point input by the touch screen 110 may be output from the touch point display unit 150 and displayed on the touch screen 110 again. For example, the point where the touch action is generated may be highlighted using a marker recognition method accepted by an image processing technology such as a color that is emphasized in contrast to a background color or other colors present on a display, or a specific shape or phrase. .

The input means identification unit 160 may identify the type of the finger that performed the touch operation based on the image of the touch operation on the touch screen 110. That is, it is possible to identify which of the thumb, the index finger, the middle finger, the ring finger, and the finger of the finger performing the touch operation in the touch motion photographed image. A more detailed description of the process of identifying a finger will be described later with reference to FIG. 4.

The search unit 170 may search for text information that matches a combination of fingers identified by the input means identification unit 160. In this case, the text information is data stored in the text information storage 190 and is text data matched according to the type of the user's finger or the combination state of the fingers. That is, the search unit 170 may search for text information corresponding to the finger identified from the text information storage unit 190.

The output unit 180 may output text information searched by the search unit 170. That is, the output character information may be transmitted to an external information processing apparatus 300 such as a TV, a computer, a screen, a tablet PC, and the like. In addition, the output unit 180 may transmit character encrypted data obtained by encrypting the character information to the external information processing apparatus 300.

The text information storage unit 190 may store text information corresponding to the finger identified through the input means identification unit 160. In this case, the character information is matched according to each point and a combination of two or more points corresponding to the thumb, the index finger, the middle finger, the ring finger, and the hand.

Also, the text information storage unit 190 may store matched text information according to a touch input gesture of touching each point and a combination of two or more points. That is, when the touch screen is touched, character information may be input by inputting character information matched according to a touch input gesture such as touch and drag, continuous touch, long touch, and the like through the touch screen. For example, when the user drags upward while touching a finger combination matching 'a' on the touch screen to input a lowercase alphabet 'a', the uppercase alphabet 'A' may be input.

The interface processor 200 may display a graphic user interface indicating character information matched according to the type of the finger or the combination state of the fingers through the point guide. In this case, the point guide includes a point for guiding a finger input and a connection line for guiding a combination of each point, but the length of the connection line may increase in proportion to the point interval. In addition, the point guide may be set by the user to match the character and arrangement according to the combination of the points.

4 is a block diagram illustrating a configuration of an input means identification unit in the character input apparatus according to another embodiment of the present invention.

The input means identification unit 160 according to an embodiment of the present invention includes an edge detector 161, a touch point identification unit 162, and a finger identification unit 163.

The edge detector 161 may extract the end point of the finger based on the touch motion image of the user captured by the camera 120. That is, five finger endpoints corresponding to the finger identification code may be extracted from the image photographed by the camera 120. For example, it can be extracted based on a generally known edge detection algorithm, and a detailed description of the algorithm is omitted.

In addition, the edge detector 161 may extract the end point of the finger based on the depth information detected by the 3D depth sensor 140.

The touch point identification unit 162 may identify the position of the point where the touch operation occurred through the marker displayed on the touch operation image. That is, when a user touches the touch screen 110, a marker indicating a touch point is generated by the touch point display unit 150 and displayed on the touch screen 110, and the touch point identification unit 162 displays the displayed marker. Through this, the location of the touch point can be identified.

The finger identification unit 163 matches the position of the end point of the finger with the position of the point where the touch action occurs to identify the finger that caused the touch action. That is, the finger identification code may be known through the end point of the finger detected by the edge detector 161. When the end point of the finger and the touch generation point coincide with each other, the finger may recognize that the finger performed the touch operation.

5 is an example of a graphical user interface illustrating a combination of point guides in accordance with one embodiment of the present invention.

At this time, in an embodiment of the present invention, the input means and the graphical user interface configuration will be described as an example based on the right hand.

The graphical user interface 900 is configured by combining each point based on each single point 911, 912, 913, 914, 915 corresponding to the thumb, index finger, middle finger, ring finger, and possession. That is, a graphic user interface includes an identification code generated by combining two or more points, i.e., two or more points, by simultaneously touching two or more fingers based on a single point 911, 912, 913, 914, and 915 corresponding to one finger. And (900). In this case, connection lines 920, 930, 940, and 950 may be included to guide the combination of two or more points, and the length of the connection line may be increased in proportion to the point interval.

For example, a graphical user interface showing each point and a combination of two or more points can be described as follows.

}이 표기된 포인트를 엄지 포인트(911)로 나타낼 수 있다. 또한, 검지 손가락 하나의 입력을 나타내는 인터페이스로 검지 식별 코드 {

}가 표기된 포인트를 검지 포인트(912)로 나타낼 수 있다. 더불어, 각각 중지, 약지, 소지에 해당하는 식별 코드 {

}, {

}, {

}를 표기한 포인트를 중지 포인트(913), 약지 포인트(914), 소지 포인트(915)로 나타낼 수 있다.First, as shown in FIG. 5, the graphical user interface 900 representing the points of five fingers, respectively, is as follows. An interface representing the input of a single thumb. Thumb identification code {

} May be represented as a thumb point 911. In addition, an interface representing the input of one index finger is provided.

} May be represented as a detection point 912. In addition, the identification code {

}, {

}, {

} May be represented as a stop point 913, a ring finger point 914, and a possession point 915.

Next, the combination of two or more points can be described as follows.

}로 표기될 수 있다.First, when the ring finger and the possession is input at the same time, the ring finger point 914 and the possession point 915 may be connected by one connecting line 920 to indicate that the two points are input at the same time. The identification code of the entered point is {

} May be indicated.

} 로 표기될 수 있다. 이때, 중지 포인트(913)과 소지 포인트(915)의 간격이 약지 포인트(914)와의 간격보다 길기 때문에 2간격 연결선(930)의 길이가 1간격 연결선(920)의 길이보다 길게 나타났음을 확인할 수 있다. 사용자는 연결선의 길이를 통해 직관적으로 포인트 사이의 간격을 파악할 수 있다.Next, in the case of simultaneously inputting the stop and the possession, the stop point 913 and the possession point 915 may be connected by a two-gap connection line 930. The identification code of the entered point is {

} May be indicated. At this time, since the interval between the stop point 913 and the possession point 915 is longer than the interval between the ring finger point 914, it can be seen that the length of the two interval connecting line 930 appeared longer than the length of the one interval connecting line 920. . The length of the connection line allows the user to intuitively determine the distance between the points.

,

}로 표기될 수 있다.Meanwhile, in the case of additionally inputting the ring finger together with the middle finger and the branch input, the ring finger point 914 may be additionally displayed on the two-interval connecting line 930 to which the stop point 913 and the point 915 are connected. . The identification code of the entered point is {

,

} May be indicated.

} 로 표기될 수 있다.Next, when the detection and the possession at the same time input, the detection point 912 and the possession point 915 may be connected by three intervals connecting line 940. The identification code of the entered point is {

} May be indicated.

,

}, {

,

}이다.On the other hand, in the case of additionally inputting the middle finger or ring finger together with the index finger and possession input, respectively, the stopping point 913 or ring finger point 914 on the three-interval connecting line 940 to which the detecting point 912 and the holding point 915 are connected. ) May be additionally represented. The identification codes of the entered points are each {

,

}, {

,

}to be.

,

}이며, 연결선 안에 두 포인트의 연결을 의미하는 {

}가 표기될 수 있다.In addition, when the middle finger and the ring finger further input simultaneously with the index finger and the possession input, the one-point connecting line 920 to which the stop point 913 and the ring finger point 914 shown on the three-interval connecting line 940 are connected is indicated. Can be represented. The identification code of the entered point is {

,

}, Which means the connection of two points within a connecting line {

} May be indicated.

}이며, 연결선 안에 두 포인트의 연결을 의미하는 {

}가 표기될 수 있다.Next, when the thumb and the base at the same time input, the thumb point 911 and the possession point 915 may be connected by the four interval connecting line 950. The identification code of the entered point is {

}, Which means the connection of two points within a connecting line {

} May be indicated.

,

}, {

,

}, {

,

}이다.On the other hand, when one or more of the index finger, middle finger or ring finger input simultaneously with the thumb and the finger input, respectively, the index point 912 on the four interval connecting line 950 connected to the thumb point 911 and the finger point 915, Each of the break point 913 or the ring finger point 914 may be additionally represented. The identification codes of the entered points are each {

,

}, {

,

}, {

,

}to be.

,

}, {

,

}이며, 각각의 연결선 안에 두 포인트의 연결을 의미하는 {

}, {

}가 각각 표기될 수 있다.Also, when the index finger and the middle finger or middle finger and the middle finger are additionally input together with the thumb and the finger input, the index finger 912 and the middle point 913 or the middle finger 913 and the middle finger which are marked on the four-interval connecting line 950 are also input. The points 914 may be connected to the one-interval connecting line 920, respectively. The identification codes of the entered points are each {

,

}, {

,

}, Which means connecting two points within each connecting line {

}, {

} May be written respectively.

,

}이며, 연결선 안에 두 포인트의 연결을 의미하는 {

}가 표기될 수 있다.In addition, when the index finger and the ring finger is additionally input simultaneously with the thumb and the finger input, the index point 912 and the stop point marked on the four interval connecting line 950 to which the thumb point 911 and the finger point 915 are connected 913 or the stopping point 913 and the ring finger point 914 may be indicated by marking the two interval connecting line 930 on the one-interval connecting line 920 respectively. The identification code of the entered point is {

,

}, Which means the connection of two points within a connecting line {

} May be indicated.

,

,

}이다.In addition, when five fingers are input at the same time, the detection point 912, the stop point 913, and the ring finger point 914 are displayed on the four interval connecting line 950 to which the thumb point 911 and the holding point 915 are connected. The detection point 912 and the stop point 912 and the stop point 913 or the stop point 913 and the ring finger point 914 on the four-interval connection line 950 are connected to the detection point 912 on the connection line connected to the one-interval connection line 920, respectively. The two-point connecting line 930 representing the connection of the ring finger point 914 may be indicated by marking the stop point 913 on the marked connecting line. The identification code of the entered point is {

,

,

}to be.

6 illustrates an example of highlighting a combination point input by a user in a graphical user interface according to an embodiment of the present invention.

In the graphic user interface 900, a point corresponding to a finger on which the user has performed input processing may be highlighted.

,

}임을 확인할 수 있다.For example, in the case of the combination point P921 inputted as a combination of four points, the inside of the connection line connecting the two points may be displayed in an accent color. In the case of FIG. 6A, the highlighted combination point P921 connects the stop point 913 and the ring finger point 914 on the four-space connecting line 950 connecting the thumb point 911 and the possession point 915. It can be confirmed that the interval 1 connecting line 920. That is, the fingers entered by the user are the thumb, middle finger, ring finger, and possession, and the identification code of the input point is {

,

} Can be confirmed.

,

}임을 확인할 수 있다.For another example, in the case of the combination point P916 input as a combination of three points, the points on the connection line connecting the two points may be displayed in the highlight color. In the case of FIG. 6B, it can be seen that the highlighted combination point P916 is the detection point 912 on the three interval connecting line 940 connecting the thumb point 911 and the ring finger point 914. That is, the fingers entered by the user are the thumb, the index finger and the ring finger, and the identification code of the input point is {

,

} Can be confirmed.

7 is an example for explaining matching of a graphic user interface and text information according to an embodiment of the present invention.

The text information storage units 320 and 190 store text information, and a matching character and arrangement according to a point and a combination of each point may be set by a user. In addition, the set character information may indicate a combination of a corresponding point and two or more points using the graphic user interface 900, and the user inputs a point matching the character information and a combination of two or more points to identify the identification code of the corresponding finger. Can be entered.

As shown in FIG. 7A, a matching alphabet may be set for each finger identification code. That is, the user may set an alphabet matching each point and a combination of two or more points.

Characters matching the finger identification code may be set by the user with alphabets, Hangul, Chinese characters, numbers, punctuation, hiragana, etc. that can be displayed on the screen by pressing the keyboard. Can be set.

}, {

}, {

}, {

}, {

}이며, 각각의 포인트에 매칭되는 문자는 알파벳 ‘A’, ‘E’, ‘I’, ‘O’, ‘U’로 설정되어 있음을 확인할 수 있다.For example, as shown in (a) of FIG. 7, the finger identification code corresponding to the thumb, index finger, middle finger, ring finger, and possession point is {

}, {

}, {

}, {

}, {

}, And the letter matching each point is set to the alphabet 'A', 'E', 'I', 'O', 'U'.

On the other hand, Figure 7 (b) is an example showing the alphabet matching the finger identification code to the graphical user interface 900. That is, an alphabet matching each point may be set and displayed at the points 911, 912, 913, 914, and 915, and two or more point combinations may be connected to the connection lines 920, 930, 940, and 950 to set the alphabet. And by displaying on the graphical user interface 900.

For example, as shown in FIG. 7B, the thumb point 911 is an alphabet 'A', the index point 912 is an alphabet 'E', and the stop point 913 is an alphabet 'I'. Thus, it can be confirmed that the ring finger point 914 is set to the letter 'O' and the holding point 915 is set to the letter 'U'. In addition, it may be confirmed that an alphabet is displayed at a connection line of two or more point combinations or a point on the connection line. In this case, in addition to the alphabet, additional function keys for inputting characters such as 'enter', 'space', and 'reserved' may be set.

8 is an example for describing a character input method using a character input apparatus according to another embodiment of the present invention.

As shown in FIG. 8A, the character input apparatus 100 according to another embodiment of the present invention may include a touch screen 110, a camera 120, and a mirror 130. First, a user's hand is positioned on the touch screen 110 of the text input device 100 for text input. In this case, since the light reflected by the hand deviates from the photographing angle of the camera 120, a mirror 130 may be additionally installed to adjust the photographing angle of the camera 120. That is, the camera 120 may be coupled to cross the camera 120 at a predetermined angle with the touch screen 110 while the photographing direction is set in a direction perpendicular to the touch screen 110. have. An image obtained through the hardware configuration of FIG. 8A is as shown in FIG. 8B.

As shown in FIG. 8B, the camera 120 may photograph the front of a user's hand touching the touch screen 110. The image acquired by capturing the camera 120 may be used as data for identifying the type of a finger performing a touch operation through the input means identification unit 160, and may be used as a data of the user through the display of the information processing device 300. Can also be used for input verification.

Next, as illustrated in (c) of FIG. 8, the finger that has performed the input may be identified through the markers 723 and 724 indicating the positions of the end points of the fingers and the points at which the touch action occurs in the captured touch motion image. Can be. That is, the thumb end point 711 corresponding to the thumb, the indexing end point 712 corresponding to the index finger, the stopping end point 713 corresponding to the middle finger, the ring finger end point 714 corresponding to the ring finger, and the holding finger The possession end point 715 can be extracted. This can be extracted based on commonly known edge detection algorithms, and a detailed description of the algorithm is omitted.

In addition, as illustrated in FIG. 8D, markers 723 and 724 highlighting a corresponding point at a point where a touch on the touch screen 110 is generated by the touch point display unit 150 when the finger performs a touch operation. ) May be output and displayed. In this case, it may be determined whether the output markers 723 and 724 and the end points of the touched finger are located within a preset distance to identify the finger that caused the touch operation.

For example, when the middle finger and the ring finger are simultaneously touched, the middle touch input marker 723 and the ring finger touch input marker 724 are displayed on the touch screen 110. That is, as shown in (d) of FIG. 8, since the fingers of the end points 713 and 714 that are closest to the markers 723 and 724 are the middle finger and the ring finger, respectively, it may be determined that the touch operation is performed by the corresponding finger.

In addition, the text input apparatus 100 having a touch screen may input text information matching the touch input gesture input through the touch screen. That is, the character information matched with the gesture input such as the drag direction and the number of instant touches may be input through the touch screen to input the matched character information. For example, when the user drags upward while touching a finger combination matching 'a' on the touch screen to input a lowercase alphabet 'a', the uppercase alphabet 'A' may be input.

FIG. 9 is an example for explaining a method of extracting a finger point using a 3D depth sensor in a text input device according to another embodiment of the present invention.

As shown in FIG. 9A, the character input apparatus 100 may further include a 3D depth sensor 140. The 3D depth sensor 140 may find five discontinuous points within a close distance from the sensor position based on the detected depth information and extract the five points as the end points of five fingers. The extracted five finger endpoints may be used as data for identifying the type of the finger that has performed the touch operation by the input means identification unit 160.

Next, as shown in FIG. 9B, the user's hand is positioned on the touch screen 110, and the user's hand is photographed in front to identify the type of finger in the input means identification unit 160. The data may be used as data, and may also be used for checking a user's input through the display of the information processing device 300.

Next, as shown in FIG. 9C, end points 711, 712, 713, 714, and 715 according to depth information of each of the five fingers may be extracted.

Finally, as illustrated in FIG. 9D, markers 723 and 724 for highlighting a finger performing a touch operation may be displayed on the touch screen 110. By determining whether the output markers 723 and 724 and the end points of the touched finger are located within a preset distance, the finger that has caused the touch operation may be identified.

FIG. 10 is an example for explaining a method of extracting a finger point using a 3D depth sensor in an information processing apparatus according to an embodiment of the present invention.

As illustrated in FIG. 10A, the finger processing point may be extracted using the 3D depth sensor 361 in the information processing apparatus 300. For example, the finger depth information may be obtained using the 3D depth sensor 361 of the character input interface device 360 connected to the information processing apparatus 300.

That is, as illustrated in FIG. 10B, five fingertips 711, 712, 713, 714, and 715 having different depth information may be extracted from the front of the hand located on the plane 111. . When the finger touches the plane 111, if the depth information from the end of the touched finger toward the plane 111 is not discontinuous or there is no sudden change, the finger detects that the depth information of the fingertip and the plane are similar and touches the corresponding finger. It can be determined that the operation has been performed.

For example, if the user touches the thumb and the ring finger, since the depth information 731 and 735 of the plane does not show a sharp difference from the thumb end point 711 and the ring finger end point 715, the user may determine that the thumb and the ring finger are touched. have. Therefore, even if the plane 111 is a general plane, not an electronic device capable of sensing a touch, the plane 111 may detect a finger that has performed a touch operation through the 3D depth sensor 361.

11 is an example for describing an input method using a physical button in an information processing apparatus according to an embodiment of the present invention.

The text input interface device 360 of the information processing device 300 may include a physical button 210. The physical button 210 may be composed of five distinct buttons for inputting an input value of each of five fingers. Each button input may distinguish five-digit individual inputs, and a character corresponding to the input combination may be configured by inputting one or two or more buttons in combination.

For example, it is a code keyboard method that can input characters by combining and inputting five buttons equal to the number of fingers rather than a general keyboard corresponding to one character per button.

12 is an example for explaining the configuration of a character input system through a character input apparatus and an information processing apparatus according to an embodiment of the present invention.

As illustrated in FIG. 12, a character to be input to the information processing apparatus 300 may be input through the character input apparatus 100. The text information transmitted from the text input apparatus 100 may be displayed by the information processing apparatus 300 on the text information display 351 and the graphic user interface 900.

In addition, the information processing device 300 may receive a finger input through the text input interface device 360 and display a text through a processing unit in the information processing device 300.

FIG. 13 is a flowchart illustrating a text input method through an information processing device according to an embodiment of the present invention.

In operation S310, finger data is input through the character input interface device. For example, when the finger data is input by the 3D depth sensor, five finger end points may be extracted according to the depth information of the finger, and the depth information generated by performing the input operation may be input.

In operation S320, the type of the finger on which the user performs an input process may be identified based on the data received in the operation.

For example, it is possible to identify the type of the finger that has performed the input process by determining which finger has performed the input operation according to the change and the correspondence of the depth information received by the 3D depth sensor.

In operation S330, the character information matched according to the type of the finger or the combination state of the finger may be represented as a graphic user interface representing the point guide. In this case, the selected point, its connection line, and text information matched thereto may be displayed in real time according to a user's input processing.

In operation S340, character information matching the combination of the fingers identified in the identifying of the input means may be retrieved.

For example, when the thumb and the index finger are simultaneously touched, the alphabet 'S' may be searched when the letter corresponding to the combination of the predetermined thumb point and the index point is the alphabet 'S'.

In step S350, the character information retrieved in the above step is displayed on the display.

14 is a flowchart illustrating a text input method through the text input device according to an embodiment of the present invention.

In step S110, the camera receives received photographing data of a user's touch operation.

In operation S120, the fingertip point of the user may be extracted based on the photographing data received in the operation. That is, five consecutive end points corresponding to the end of the user's finger may be extracted from the data photographed by the user's touch operation.

In operation S130, the position of the point where the touch operation occurs may be identified through a marker displayed on the point on the touch screen generated by the user's touch operation based on the photographing data received in the step.

In step S140, the finger may be identified based on the finger endpoint extracted in the step and the touch point position identified in the step. That is, a finger that has generated a touch may be identified by identifying a finger that has generated a marker indicating a touch point among five fingertips.

In operation S150, text information matching a point corresponding to the finger identified in the step and a combination of two or more points may be searched for.

For example, when the thumb and the index finger are simultaneously touched, the alphabet 'S' may be searched when the letter corresponding to the combination of the predetermined thumb point and the index point is the alphabet 'S'.

In step S160, the character information retrieved in the above step is output.

The invention can also be embodied in the form of a recording medium containing instructions executable by a computer, such as a program module executed by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

The foregoing description of the present invention is intended for illustration, and it will be understood by those skilled in the art that the present invention may be easily modified in other specific forms without changing the technical spirit or essential features of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is shown by the following claims rather than the above description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

100: character input device 110: touch screen
120: camera 130: mirror
140: 3D depth sensor 150: touch point display
160: input means identification unit 161: edge detection unit
162: touch point identification unit 163: finger identification unit
170: search unit 180: output unit
190: text information storage unit 200: interface processing unit
300: information processing apparatus 310: input means identification unit
320: text information storage unit 330: search unit
340: interface control unit 350: character information display unit
360: character input interface device 400: character input system
723: stop touch input marker 724: ring finger touch input marker
900: graphical user interface

Claims (19)

In the information processing apparatus,
An input means identification section for identifying a type of a finger on which the user has performed an input process, from the user's input data input through the character input interface device;
An interface control unit for displaying on the display of the information processing apparatus a graphic user interface indicating character information matched according to the type of a finger or the combination state of the fingers through a point guide;
A retrieval unit for retrieving character information matching the combination of fingers identified by the input means identification unit;
Including a character information display unit for displaying the character information retrieved through the search unit on the display,
The point guide includes a point for guiding a finger input and a connecting line for guiding a combination of each point, wherein the length of the connecting line increases in proportion to the point interval. The method of claim 1,
The point guide is an information processing apparatus that can be set by the user to match characters and arrangement in accordance with the combination of the points. The method of claim 1,
And the character input interface device is a 3D depth sensor for detecting a user's finger touch or a physical button composed of a plurality of distinct buttons corresponding to each finger. The method of claim 3, wherein
And the input means identification unit extracts an end point of the finger based on depth information of the finger sensed by the 3D depth sensor. The method of claim 3, wherein
And the input means identification unit identifies the input finger by mapping the input data of one physical button to the input of one finger of the user. The method of claim 1,
And the interface controller is configured to display, in real time, a selected point and connection line according to a user's input process and text information matched thereto. The method of claim 1,
The information processing apparatus further includes a text information storage section for storing text information corresponding to the finger identified through the input means identification section,
And the retrieval unit retrieves character information corresponding to the identified finger combination from the character information storage unit. The method of claim 7, wherein
And the character information storage unit stores character information that is matched according to each point corresponding to a thumb, index finger, middle finger, ring finger, and possession and a combination of two or more points. In the character input device,
touch screen,
A camera coupled to a position adjacent to an edge of the touch screen,
A mirror coupled to a location adjacent to the camera to intersect the shooting direction of the camera to guide the camera to capture a user's touch motion;
A touch point display unit for outputting a marker for highlighting a corresponding point at a point where a touch on the touch screen occurs when a touch operation with respect to the touch screen occurs;
An input means identification unit for identifying a type of a finger performing a touch operation based on an image photographing a touch operation on the touch screen;
A search unit for searching for character information matching the combination of fingers identified by the input means identification unit;
Character input device comprising an output unit for outputting the character information. The method of claim 9,
And the output unit transmits the text information to an external information processing apparatus. The method of claim 9,
And the output unit transmits character encrypted data obtained by encrypting the character information to an external information processing apparatus. The method of claim 9,
The character input apparatus further includes a character information storage unit for storing character information corresponding to the finger identified through the input means identification unit,
And the search unit retrieves text information corresponding to the identified finger from the text information storage unit. 13. The method of claim 12,
The character information storage unit stores character information that is matched according to a combination of two or more points and each point corresponding to the thumb, the index finger, the middle finger, the ring finger, and the hand. The method of claim 13,
The text information storage unit stores text information that is matched according to a touch input gesture of touching each point and a combination of two or more points. The method of claim 9,
The character input apparatus further includes an interface processor configured to display a graphic user interface indicating character information matched according to a type of finger or a combination state of the fingers through a point guide. The method of claim 15,
The point guide includes a point for guiding a finger input and a connection line for guiding a combination of each point, wherein the length of the connection line increases in proportion to the point interval. The method of claim 15,
The point guide is a character input device that can be set by the user the matching characters and arrangement in accordance with the combination of the points. The method of claim 9,
The input means identification unit
An edge detector extracting an end point of the finger based on a user's touch operation image photographed through the camera;
A touch point identification unit for identifying a position of a point where a touch action occurs through a marker displayed on the touch action image of the user;
And a finger identification unit configured to identify a finger generating a touch action by matching a position of an end point of the finger with a point where a touch action occurs. The method of claim 9,
Further comprising a 3D depth sensor coupled adjacent to the point where the camera is coupled,
The input means identification unit
An edge detector extracting an end point of the finger based on depth information detected by the 3D depth sensor;
A touch point identification unit for identifying a position of a point where a touch action occurs through a marker displayed on a user's touch action image photographed through the camera;
And a finger identification unit configured to identify a finger generating a touch action by matching a position of an end point of the finger with a point where a touch action occurs.

Images