KR102076539B1 - Portable terminal using touch pen and hndwriting input method therefor - Google Patents

Abstract

The present invention relates to a handwriting input method of a portable terminal using a pen for touch and a device supporting the handwriting input method.
To this end, when a handwriting image is input using a pen for a touch on an input field provided by an application that the mobile terminal is running, the handheld terminal converts the handwriting image input using the touch pen into text, and the converted text Is inserted into and displayed in an input field provided by the application.

Description

A portable terminal using a pen for touch and a handwriting input method using the same {PORTABLE TERMINAL USING TOUCH PEN AND HNDWRITING INPUT METHOD THEREFOR}

The present invention relates to a portable terminal using a pen for touch and a handwriting input method using the same, and more particularly, to a method for performing handwriting input on an application running in a portable terminal.

Recently, as the number of portable terminals increases, users' demands for an intuitive input / output method are increasing. However, the size of the portable terminal is limited for ease of portability, and as a result, the size of the display unit for displaying information has a relatively small feature compared to a general TV or a monitor. For this reason, the mobile terminal deviates from a conventional UI method, such as a keyboard, a keypad, and a mouse, for a variety of user input, and uses a finger or a pen for touch to directly touch the screen or use voice. It is evolving in an intuitive UI way to input information. In particular, the user can make a more detailed touch with the pen for touch, and the conventional pen for touch simply stays at a level of touching a specific area output on the display unit. Accordingly, various demands for inputting data as well as touches in a mobile terminal using a pen for touch continue.

The present invention provides a method and apparatus for inputting information using a handwriting-based user interface in a portable terminal.

In particular, the present invention provides a method and apparatus for inputting information into the application using a handwriting-based user interface in a situation where an application is being driven in a portable terminal.

In particular, the present invention provides a method and apparatus for inputting information into the input field using a handwriting-based user interface in a situation where an input field provided by an application is displayed on a portable terminal.

In addition, the present invention provides a user interface differently for inputting information into an input field according to whether a touch is performed using a finger or a touch pen when a input field provided by an application is displayed on a mobile terminal. It provides an apparatus and method.

According to an exemplary embodiment of the present invention, a handwriting input method of a portable terminal using a touch pen includes executing an application to display at least one input field provided by the application, directly from above on the at least one input field And receiving a handwritten image using the touch pen, converting the inputted handwritten image into text, and inserting and displaying the converted text in the input field.

In addition, the step of receiving a handwriting image using the touch pen includes overlaying the input field to provide a handwriting input layer and receiving a handwriting image through the provided handwriting input layer, and the input handwriting The converting the image to text may include converting the handwriting image input through the handwriting input layer into text.

In addition, when there are a plurality of input fields, the step of providing the handwriting input layer may include a plurality of handwriting input layers corresponding to each of the plurality of input fields.

A step of providing and converting the handwritten image to text may include converting the handwritten image received from each of the plurality of handwriting input layers into text.

In addition, when text is already inserted in some of the plurality of input fields, the providing of the handwriting input layer may include providing a handwriting input layer only in input fields excluding some of the plurality of input fields. It can contain.

In addition, the step of converting the handwritten image into text may include converting all of the handwritten images received from each of the plurality of handwriting input layers into text when a switch button is selected.

In addition, the step of providing the handwriting input layer includes providing a delete button for deleting the handwriting image input through the handwriting input layer, and when the delete button is selected, at least a part of the input handwriting image It may include the step of deleting.

In addition, displaying the input field provided by the application may include displaying a plurality of times provided by the application, and when one of the plurality of times is selected, an input field for writing content related to the selected time It may include the step of displaying.

In addition, in the step of converting the handwritten image into text, the handwritten image input from the first handwriting input layer among the plurality of handwriting input layers is first converted to a first type among a first language, a second language, numbers, and special symbols. And converting the handwritten image input from the second handwriting input layer among the plurality of handwriting input layers to a second type different from the first type among the first language, the second language, numbers, and special symbols. You can.

In addition, the step of converting the handwritten image into text includes receiving coordinate values of points touched on the handwriting input layer, storing coordinate values of the points as a stroke, and generating a stroke array list using the stroke And converting the handwritten image into text using a pre-stored handwriting library and the stroke array list.

In addition, the handwriting input method according to the present invention includes receiving a user's touch using a finger on the input field, providing a keypad for receiving input using the finger, receiving text through the keypad, and The method may further include inserting and displaying the selected text in the input field.

According to an exemplary embodiment of the present invention, a handwriting input method of a portable terminal using a touch pen includes executing an application to display at least one input field provided by the application, and a user on the at least one input field Receiving a touch of the user, determining whether the user's touch is using the finger or a touch using a pen, and if the touch is using a finger, a keypad for receiving input using the finger Provides, and displays the text input through the keypad by inserting it into the input field, and when the touch is a touch using a touch pen, the handwriting input layer for receiving input using the touch pen is provided in the input field Provided by overlapping, and through the handwriting input layer And converting the input handwritten image into text and inserting it into the input field to display it.

According to an exemplary embodiment of the present invention, a portable terminal using a touch pen displays an application including at least one input field and receives a handwriting image input using the touch pen on the at least one input field. Screen, and

And a control unit that converts the handwritten image input through the touch panel unit into text and inserts the converted text into the input field to display.

In addition, the controller may provide a handwriting input layer for receiving the handwriting image by overlapping the input field and convert the handwriting image input through the handwriting input layer into text.

In addition, when there are a plurality of input fields, the controller may provide a plurality of handwriting input layers corresponding to each of the plurality of input fields and convert a handwriting image received from each of the plurality of handwriting input layers into text.

In addition, when text is already inserted in some of the plurality of input fields, the controller may provide a handwriting input layer only in an input field excluding some of the plurality of input fields.

A mobile terminal, characterized by.

In addition, when the switch button is selected, the control unit may convert all of the handwritten images received from each of the plurality of handwriting input layers into text.

In addition, the controller may further provide a delete button for deleting the handwriting image input through the handwriting input layer, and when the delete button is selected, may delete at least a part of the received handwriting.

Further, the touch screen may display a plurality of times provided by the application, and when one of the plurality of times is selected, may display an input field for writing content related to the selected time.

In addition, when the handwriting image received is converted into text, the control unit converts the handwriting image input from the first handwriting input layer among the plurality of handwriting input layers into a first type among a first language, a second language, numbers, and special symbols. Switch to priority, and

The handwriting image input from the second handwriting input layer among the plurality of handwriting input layers may be firstly switched to a second type different from the first type among the first language, the second language, numbers, and special symbols.

In addition, when the controller converts the input handwritten image into text,

After receiving the coordinate values of the points touched on the handwriting input layer, storing the coordinate values of the points as a stroke, and using the stroke to generate a stroke array list, using the pre-stored handwriting library and the stroke array list By doing so, the handwritten image can be converted into text.

In addition, the control unit may receive a user's touch using a finger on the input field, provide a keypad for receiving the input using the finger, and insert the selected text through the keypad into the input field for display. .

According to another exemplary embodiment, a mobile terminal using a touch pen displays an application including at least one input field, a touch screen receiving a user's touch on the at least one input field, and the user's touch screen. It is determined whether the touch is a touch using the finger or a touch using a pen, and when the touch is a finger touch, a keypad for receiving input using the finger is provided, and text received through the keypad is received. It is inserted into the input field and displayed. When the touch is a touch using a pen for touch, a handwriting input layer for receiving input using the touch pen is superimposed on the input field, and through the handwriting input layer Convert the input handwriting image to text to remind It may comprise a control unit which displays the insertion force in the field.

In the present invention, data can be quickly input into an application running on a mobile terminal using a pen for touch. In particular, by inputting a handwritten image using a touch pen on the input field of the application, a fast and intuitive interface can be provided to the user.

In addition, effects obtained or predicted by embodiments of the present invention will be disclosed directly or implicitly in the detailed description of the embodiments of the present invention. That is, various effects predicted according to embodiments of the present invention will be disclosed within a detailed description to be described later.

1 is a view showing a handwriting input system according to an embodiment of the present invention.
2 is a view showing the configuration of a mobile terminal according to an embodiment of the present invention.
3 is a view showing in more detail the configuration of the control unit of the present invention.
4 is a flowchart illustrating a handwriting input method of a portable terminal according to an embodiment of the present invention.
5 is an example of a scenario in which text is displayed in an input field of an application in a mobile terminal.
6 is another example of a scenario in which text is displayed in an input field of an application in a mobile terminal.
7A and 7B are examples of scenarios in which text is displayed in an input field of a hospital chart application in a mobile terminal.
7C is an example of images that can be displayed in an image area of a hospital chart application in a mobile terminal.
8A and 8B are examples of scenarios in which text is displayed in an input field of a schedule management application in a mobile terminal.
9A and 9B are examples showing methods for displaying text in an input field in a mobile terminal.
10 is an example of a scenario in which text is displayed in an input field of a web browser application in a mobile terminal.
11 is a flowchart illustrating a handwriting input method of a portable terminal according to another embodiment of the present invention.
12 is a flowchart illustrating a handwriting input method of a portable terminal according to another embodiment of the present invention.

Hereinafter, in the detailed description of the present invention, a representative embodiment for achieving the above-described technical problem will be presented. In addition, for convenience of description of the present invention, the names of defined objects may be used identically. However, the names used for the convenience of description do not limit the rights according to the present invention, and it is of course possible to apply the same or easy changes to a system having a similar technical background.

For the same reason, in the accompanying drawings, some components are exaggerated, omitted, or schematically illustrated, and the size of each component does not entirely reflect the actual size. Therefore, the present invention is not limited by the relative size or spacing drawn in the accompanying drawings.

1 is a view showing a handwriting input system 10 according to an embodiment of the present invention.

Referring to FIG. 1, the handwriting input system 10 may include a portable terminal 100 and a touch pen 200. In the handwriting input system 10, a user may input a handwritten image on the screen of the portable terminal 100 while the touch pen 200 is held. The handwriting input system 10 shows an example of a configuration according to an embodiment of the present invention, but a configuration for other functions may be additionally provided.

2 is a view showing the configuration of a mobile terminal 100 according to an embodiment of the present invention.

Referring to FIG. 2, the portable terminal 100 of the present invention comprises a communication unit 110, an input unit 120, an audio processing unit 130, a touch screen 140, a storage unit 150, and a control unit 160. It can contain.

The touch screen 140 may be provided with a display panel 141 that performs a display function for output information output from the portable terminal 100 and an input panel 142 that performs various input functions by a user. . At this time, the input panel may be implemented as at least one panel capable of detecting various inputs such as a single or multi-touch input, drag input, handwriting input, and drawing input by a user using various objects such as fingers and pens. For example, the input panel 142 may be implemented using one panel capable of both finger input detection and pen input detection, and the touch panel 145 capable of finger input detection and pen recognition capable of pen input detection Like the panel 143, it may be implemented using two panels. Hereinafter, in the embodiment of the present invention, an example in which the input panel 142 includes two panels, such as a touch panel 145 capable of detecting finger input and a pen recognition panel 143 capable of detecting pen input, is implemented. It will be explained.

The display panel 141 may be a panel such as an LCD or AMOLED. The display panel 141 may display various screens according to various operating states of the portable terminal 100, application execution, and services. In particular, according to an embodiment of the present invention, the display panel 141 may display an application including an input field.

The touch panel 145 may detect a user's touch input. The touch panel 145 may have, for example, a form of a touch film, a touch sheet, or a touch pad. The touch panel 145 detects a touch input and outputs a touch event value corresponding to the sensed touch signal. At this time, information corresponding to the sensed touch signal may be displayed on the display panel 141. The touch panel 145 may receive an operation signal by a user's touch input by various input means. For example, the touch panel 145 may sense a touch input including a user's human body (eg, a finger) or a physical tool. According to an embodiment of the present invention, the touch panel 145 may be configured as a capacitive touch panel. When configured as a capacitive touch panel, the touch panel 145 is coated with a thin metal conductive material (for example, ITO (Indium Tin Oxide) film, etc.) on both sides of the glass so that current flows through the glass surface. And coated with a dielectric that can store electric charges. When a surface is touched by an object on the touch panel 145, a certain amount of charge moves to the contact position by static electricity, and the touch panel 145 recognizes the amount of change in current according to the movement of the charge to detect the touched position. Become and track touch events. Here, the touch event generated in the touch panel 145 may be mainly generated by a human finger, but may also be generated by another object capable of applying a change in capacitance, for example, an object made of a conductive material capable of applying a change in capacitance. Can be.

The fan recognition panel 143 detects the proximity input or touch input of the pen according to the operation of the user's touch pen 200 (eg, a stylus pen, a digitizer pen), and the detected pen proximity Outputs an event or pen touch event. The pen recognition panel 143 may be implemented by an EMR method, and may sense a touch or proximity input according to a change in the intensity of the electromagnetic field due to proximity or touch of the pen. Specifically, the pen recognition panel 143 has a plurality of loop coils disposed in a predetermined first direction and a second direction crossing the first direction, respectively, and has an electromagnetic induction coil sensor (not shown) and an electromagnetic induction coil sensor having a grid structure. It may be configured to include an electronic signal processing unit (not shown) for sequentially providing an AC signal having a predetermined frequency to each loop coil. When a pen incorporating a resonance circuit in the vicinity of the loop coil of the pen recognition panel 143 is present, a magnetic field transmitted from the loop coil generates a current based on mutual electromagnetic induction in the resonance circuit in the pen. Based on this current, an induction magnetic field is generated from the coils constituting the resonance circuit in the pen, and the pen recognition panel 143 detects the induction magnetic field in the loop coil in the signal reception state, so that the pen approaches or Touch position is detected. Through the pen recognition panel 143, if an object capable of generating an electric current based on electromagnetic induction is possible, it is possible to detect the approach and touch, and an embodiment of the present invention will be described as being used only for pen proximity or pen touch recognition. The pen recognition panel 143 may be disposed at a certain position of the terminal and may have an active state according to occurrence of a specific event or by default. In addition, the pen recognition panel 143 may be provided with a certain area under the display panel 141, for example, an area capable of covering the display area of the display panel 141.

The communication unit 110 is a configuration that can be included when the portable terminal 100 supports a communication function. In particular, when the mobile terminal 100 supports a mobile communication function, the communication unit 110 may be configured as a mobile communication module. The communication unit 110 may perform a specific function of the portable terminal 100 requiring a communication function, for example, a chat function, a message transmission / reception function, a call function, and the like.

The input unit 120 may be configured with a side key or a separately provided touch pad. In addition, the input unit 120 may include a button key for performing turn-on or turn-off of the mobile terminal 100, a home key for returning to a basic screen supported by the mobile terminal 100, and the like. .

The audio processing unit 130 may include at least one of a speaker for outputting an audio signal from the portable terminal 100 and a microphone for collecting audio signals. In addition, the audio processor 130 may control the vibration module to control the vibration magnitude of the vibration module. For example, the audio processing unit 130 may vary the vibration level according to the gesture input operation. That is, when the gesture recognition information is different, the audio processing unit 130 may control the vibration module to have a vibration level corresponding to each gesture recognition information.

The storage unit 150 is configured to store various programs and data necessary for operating the portable terminal 100. For example, the storage unit 150 may store an operating system required for operating the portable terminal 100, and may store function programs for supporting a screen output on the display panel 141 described above.

The control unit 160 includes various components for receiving and displaying a handwritten image using a touch pen according to an embodiment of the present invention. Based on these components, the signal processing and data processing and function operation for handwriting input are performed. Can be controlled. For example, the control unit 16 controls to display a handwriting input layer by overlapping with an input field of an application being displayed, and convert a handwritten image by the user into text on the provided handwriting input layer to be inserted into the input field for display. You can. To this end, the control unit 160 may include a configuration as shown in FIG. 3.

3 is a view showing in more detail the configuration of the control unit 160 of the present invention.

Referring to FIG. 3, the control unit 160 of the present invention may include an application operation unit 161, a writing area providing unit 162, and a writing recognition unit 163.

The application operating unit 161 may execute a specific application when a user commands (eg, selects an icon corresponding to a specific application on the desktop) or powers on. In addition, the executed application may be operated to be displayed through the display panel 141. At this time, the running application may provide at least one input field on the display panel 141.

The writing area providing unit 162 may provide a handwriting input layer for recognizing a handwritten image using a touch pen. For example, the handwriting area providing unit 162 may overlap the input field provided by the application being displayed to provide a handwriting input layer. In addition, when there are a plurality of input fields included in the application, the handwriting area providing unit 162 may provide a plurality of handwriting input layers corresponding to each of the plurality of input fields. The user may perform handwriting on the handwriting input layer using a touch pen.

At this time, the implementation method for providing a handwriting input layer overlapping with an input field in the Android build system may be as follows.

   First, the layout for determining the position of the input field and the handwriting input layer on the screen can be programmed as follows using XML code.

<FrameLayout

android: id = "@ + id / nli_title_layout"

… … …

>

… … …

                 <TextView

                android: id = "@ + id / title"

                        style = "@ style / TextAppearance.EditEvent_Value“

… … …

android: minHeight = "48dip"

                        androidingleLine = "true" />

                  … … …

                  <com.samsung.nli.util.view.NliSCanvasView

                    android: id = "@ + id / nli_title_nliSCanvasView"

                    … … …

android: focusable = "true" />

Referring to the above program, the developer can define the layout of the input field by calling the TextView method, and if the input field is title, for example, in the method, the property related to the position or size associated with it can be defined. In addition, the developer can define properties related to the position or size of the handwriting input layer for receiving a handwriting image by calling the nli_title_nliSCanvasView method. At this time, since the nli_title_nliSCanvasView method shares the same parent as the TextView method, properties related to the position and size of the handwritten input layer may be the same.

Meanwhile, the NliScanvasView method may include an nli_title_eraseImageButton method associated with a delete button for deleting a part of the handwriting input in the handwriting input layer.

Next, it can be programmed as follows so that the layout defined by XML code in Java code can be used.

mTitleTextView = (TextView) view.findViewById (R.id.title);

titleLayout = (FrameLayout) view.findViewById (R.id.nli_title_layout);

titleSCanvasView = (NliSCanvasView) view.findViewById (R.id.nli_title_nliSCanvasView);

titleEraseImageButton = (ImageButton) view.findViewById (R.id.nli_title_eraseImageButton);

titleSCanvasView.setContent (mActivity, titleEraseImageButton, mTitleTextView);

Referring to the above program, the developer can call values associated with each TextView, FrameLayout, NliSCanvasView, and ImageButton properties written in XML code. And, it can be arranged by combining the called values using the titleSCanvasView.setContent method.

The handwriting recognition unit 163 uses the handwriting image by input means such as a pen for touch on the handwriting input layer as input information. In addition, the handwriting recognition unit 163 recognizes the input handwriting image and converts the input handwriting image into text as a result of the recognition. To this end, the handwriting recognition unit 163 includes a cognitive management module 164 that processes overall control for outputting a result recognized from the input information, a remote cognitive request module 165, and a local cognition module 166 for recognizing input information. ). The remote recognition request module 165 may transmit the handwritten image input to the pen recognition panel 143 to a server (not shown) in order to recognize the handwritten image. In addition, the remote recognition module 165 may receive text that is a result of recognizing the handwritten image from the server. Meanwhile, the local recognition module 165 may include a handwriting recognition block 167 for recognizing information input based on the handwritten image. Specifically, the handwriting recognition block 167 may receive coordinate values of points touched by the pen recognition panel 143, store coordinate values of the touched points as a stroke, and generate a stroke array using the stroke. have. In addition, the handwriting recognition block 167 may recognize a handwritten image using a pre-stored handwriting library and the generated stroke array list.

Meanwhile, the handwriting recognition unit 163 may perform handwriting recognition in consideration of the type of text. When the handwriting area providing unit 162 provides a plurality of handwriting input layers, the handwriting image input from one handwriting input layer of the plurality of handwriting input layers is first in a first language, a second language, numbers, or special symbols. It is recognized as a kind of priority, and as a result, the handwritten image can be converted into text. In this case, the handwriting recognition unit 163 may receive information for recognizing the first type from the application operation unit 161. For example, if the type of handwriting input in the input field provided by the application is already determined, the application operation unit 161 may advance information on the type of handwriting to be input in the input field to the handwriting recognition unit 163 in advance. Can provide. The handwriting recognition unit 163 acquires the type of handwriting to be input in the input field, and when handwriting is input in the handwriting input layer corresponding to the input field, the handwriting image is used as the handwriting type obtained from the application operation unit 161. Priority can be recognized.

The text that is the result of recognizing the handwritten image by the handwriting recognition unit 163 may be provided to the application operation unit 161. As another example, when a plurality of handwriting is input to each of the plurality of handwriting input layers, the handwriting recognition unit 163 generates a plurality of texts as a result of recognizing each of the plurality of handwriting images, and the application operation unit 161 Can be provided as

The application operating unit 161 may insert text received from the handwriting recognition unit 163 into an input field of a running application. In addition, the application operation unit 161 may control the input field in which text is inserted to be displayed on the display panel 141. As another example, when a plurality of texts are received from the handwriting recognition unit 163, the application operating unit 161 may store a plurality of texts in an input field corresponding to each of the plurality of texts. In addition, the application operating unit 161 may control a plurality of input fields in which text is inserted to be displayed on the display panel 141.

4 is a flowchart illustrating a handwriting input method of the portable terminal 100 according to an embodiment of the present invention.

Referring to FIG. 4, the portable terminal 100 may execute a specific application when a user requests or powers up (S401).

When the mobile terminal 100 displays an application on the display panel 141, at least one input field provided by the application may be displayed on the display panel 141 (S403). In other words, the mobile terminal 100 may provide a screen including at least one input field provided by the application on the display panel 141.

Next, the portable terminal 100 may receive a handwritten image using a touch pen on the input field (S405). For example, the portable terminal 100 may provide a handwriting input layer superimposed on an input field in order to receive a handwritten image. The handwriting input layer is automatically provided when an application including an input field is displayed as a virtual layer on an application for recognizing a handwriting image, or a predefined user gesture (eg, swipe with a pen for touch on one side of the terminal) (Swipe)). Alternatively, it may be provided when the touch pen is detached from the portable terminal 100. Such a virtual layer can be provided, for example, as transparent or translucent.

The portable terminal 100 may receive a handwritten image using a touch pen through the provided handwriting input layer. At this time, the handwriting input layer need not be limited to the size of the input field being displayed. The handwriting input layer may have a larger area than the input field including the input field. That is, receiving a user's handwriting using a touch pen directly on the input field means that even if all of the handwritten images that the user writes for the input field do not overlap with the input field, that is, a part of the handwritten image is out of the input field. Even if it deviates, it means that a user's handwritten image is inputted substantially on the input field.

As another example, the input field may usually consist of one column. In this case, the handwriting input layer may be provided to correspond to the one column. That is, even when the user writes in two columns in the handwriting input layer, the mobile terminal 100 converts each of the two handwritten images written in the handwriting input layer into text, and combines the texts corresponding to each converted column into a single column behavior. Can be created and inserted into the input field.

As another example, when there are a plurality of input fields included in the application, the mobile terminal 100 may provide a plurality of handwriting input layers corresponding to each of the plurality of input fields. At this time, some of the plurality of handwriting input layers may overlap each other. In addition, when the user's handwriting occurs in the overlapped area, the portable terminal 100 may determine which of the plurality of handwriting input layers the handwriting is in consideration of the user's handwriting input order or the location where the handwriting started. For example, the portable terminal 100 may determine the handwriting image in the overlapped area as the handwriting of the handwriting input layer corresponding to the location where the handwriting originally started.

Next, the portable terminal 100 may convert the handwritten image input through the handwriting input layer into text (S407). In addition, when a plurality of handwritings are respectively input through a plurality of handwriting input layers, the portable terminal 100 may convert the plurality of handwriting images into text respectively.

Next, the portable terminal 100 may display the converted text by inserting it into the input field (S409). In addition, when each of the plurality of texts corresponding to a plurality of fields is switched, the portable terminal 100 may insert and display each of the converted plurality of texts in each of the plurality of input fields.

5 is an example of a scenario in which text is displayed in an input field of an application in the mobile terminal 100.

In 510 of FIG. 5, the mobile terminal 100 may display a contact registration application including a plurality of input fields 511,512,513,514.

In 530 of FIG. 5, the mobile terminal 100 may provide a plurality of handwriting input layers 531,532,533,534 corresponding to each of the plurality of input fields 511,512,513,514. Then, the user's handwriting input may be received through the provided handwriting input layers 531,532,533,534, respectively. In 530 of FIG. 5, the mobile terminal receives a handwritten image called “Kim Hwa-kyung” through the handwriting input layer 531, receives a handwritten image of “010-1234-1234” through the handwriting input layer 532, and writes A "nalza98" handwriting image may be input through the input layer 533 and a "2/18 birthday" handwritten image may be input through the handwriting input layer 534.

In 550 of FIG. 5, the mobile terminal 100 may recognize a plurality of inputted handwritten images and convert them into a plurality of tests corresponding to each of the plurality of handwritten images. Then, the obtained texts may be inserted into and displayed in a plurality of input fields 511, 512, 513, and 514, respectively. In 550 of FIG. 5, the mobile terminal 100 inserts and displays the text “Kim Hwa-kyung”, which is the result of the handwriting image conversion received through the handwriting input layer 531, in the input field 511, and in the input field 512 The text "010-1234-1234", which is the result of the conversion of the handwritten image received through the handwriting input layer 532, is inserted and displayed. In the input field 513, the handwriting image received through the handwriting input layer 533 is displayed. The text "nalza98", which is the result of the conversion, is inserted and displayed. In the input field 514, the text "2/18 birthday", which is the result of the conversion of the handwritten image received through the handwriting input layer 534, can be inserted and displayed. have.

6 is another example of a scenario in which text is displayed in an input field of an application in the mobile terminal 100.

In 610 of FIG. 6, the mobile terminal 100 may display an application including a plurality of input fields 611,612,613,614. At this time, some of the input fields 611,612,613,614 may have text already inserted and displayed. This means that a user does not need to separately input handwriting on some of the input fields 612 and 613 because the contents to be written in some of the input fields 612 and 613 have already been determined.

In 630 of FIG. 6, the mobile terminal 100 may provide respective handwriting input layers 631 and 632 corresponding to the remaining input fields 611 and 614 except for the input fields 612 and 613 in which text is already inserted. In other words, a plurality of handwriting input layers 631 and 632 may be provided only to some of the input fields 611 and 614 of the plurality of input fields 611,612,613,614.

In addition, the user's handwriting input may be received through the handwriting input layers 631 and 632. In 650 of FIG. 6, the mobile terminal 100 may insert and display texts resulting from the conversion of the input handwriting image into the remaining fields 611 and 614, respectively.

7A and 7B are examples of scenarios in which the mobile terminal 100 displays text in an input field of a hospital chart application.

7A is an example of recording patient information in a hospital chart using the input method according to the present invention.

In 710 of FIG. 7A, the mobile terminal 100 may display a hospital chart application including a plurality of input fields. The application may include a completion button 715, a storage button 716, and a plurality of input fields 711,712,713,714. The completion button 715 is a switching button for converting a handwritten image input by a user into a hospital chart into a test using a touch pen. At this time, the conversion button can be called by various names such as a completion button, a recognition button, a conversion button, etc. if a function for converting a handwritten image into text is supported.

  The storage button 716 is a button for storing a handwritten image input by a user (eg, a doctor) into a hospital chart. The plurality of input fields 711,712,713,714 are areas for recording patient or hospital information. At this time, information of a patient or a hospital may already be displayed in some of the input fields 714 of the plurality of input fields.

In 720 of FIG. 7A, the mobile terminal 100 may provide a plurality of handwriting input layers 721, 722, and 723 corresponding to each of the input fields 711, 712, and 713. At this time, a handwriting input layer may not be provided in the input field 714 where information is already displayed. However, if the user wants to modify the text in the input field 714, a handwriting input layer may be provided on the input field 714 where information is already displayed. Next, the portable terminal 100 may receive the user's handwriting input through the handwriting input layers 721, 722, and 723, respectively. In 720 of FIG. 7A, the mobile terminal 100 may receive a handwriting input corresponding to a patient's name, a patient's address, and a preliminary item. Then, when the user selects the completion button 715, the portable terminal may recognize each handwritten image input through the handwriting input layers 721, 722, and 723 and convert it to text.

In 730 of FIG. 7A, the mobile terminal 100 recognizes each handwritten image received as patient information, and uses a plurality of texts corresponding to each of the plurality of handwritten images, such as "Kim Hwa-kyung", "Sadang-dong, Dongjak-gu, Seoul", "√ ". Then, the obtained texts may be inserted into and displayed in a plurality of input fields 711,712,713, respectively.

7B is another example of recording patient information in a hospital chart using the input method according to the present invention.

In 740 of FIG. 7B, the mobile terminal 100 may display a hospital chart registration application including a plurality of input fields. The application may include a completion button 745, a storage button 746, and a plurality of input fields 741,742,743,744. The completion button 745 is a button for recognizing a handwritten image input by a user on a hospital chart using a touch pen and converting it into text. The storage button 746 is a button for storing a handwritten image and a patient image 748 input by the user in the hospital chart. At this time, the handwriting image or the patient image 748 input by the user may be automatically stored at regular intervals without selecting a separate storage button 746. The plurality of input fields 741,742,743,744 are areas for recording patient or hospital information. At this time, information of a patient or a hospital may already be displayed in some of the input fields 742 of the plurality of input fields.

In 750 of FIG. 7B, the mobile terminal 100 may provide a plurality of handwriting input layers 751,753,754 corresponding to each of the plurality of input fields 741,742,743,744. At this time, a handwriting input layer may not be provided in the input field 742 where information is already displayed. Next, the portable terminal 100 may receive a user's handwriting image through the handwriting input layers 751,753,754, respectively. In 750 of FIG. 7B, the portable terminal 100 may receive handwriting input corresponding to the patient's name and the patient's gender, respectively. Next, when the user selects the completion button 745, the mobile terminal may convert each handwritten image input through the handwriting input layers 751,753,754 into text. At this time, the text conversion may be omitted in the area 753 not written by the user.

In 760 of FIG. 7B, the mobile terminal 100 recognizes each handwritten image received as patient information and obtains “Kim Hwa-kyung” and “√” as a plurality of texts corresponding to each of the plurality of handwritten images. . Also, the obtained texts may be inserted into and displayed in a plurality of input fields 741 and 744, respectively.

The image change area 748 is an area where a user can insert or replace an image of a patient on a hospital chart. The user can select the image button 747 to load or replace the image regarding the patient. The image may be an image stored in the mobile terminal 100 or an external storage device (not shown) of the mobile terminal 100 or may be a picture that the user is currently using the mobile terminal 100. 7C shows an example of images 771,772,773 that can be displayed in the image area 748 of the hospital chart. Meanwhile, the stored or captured image may be managed as a group consisting of a plurality of images, such as a photo album, along with other images. In this case, when the user performs a specific gesture (eg, swipe) on the image area 748, the image in the image area 748 may be replaced with another image in the same group. For example, the images 771, 772 and 773 in FIG. 7C are managed as a group and can be replaced by a specific gesture of the user. As described above, when the present invention is used in a hospital, a doctor can quickly describe a patient's condition on a hospital chart provided by the mobile terminal 100 by using a touch pen during patient interviews, and easily manage it.

8A and 8B are examples of scenarios in which the mobile terminal 100 displays text in an input field of a schedule management application.

In 810 of FIG. 8A, the mobile terminal 100 may display a plurality of times provided by the application. For example, a plurality of times may be calendars provided by the application. The user can select a specific time at which he / she wants to write the contents of the schedule among the plurality of times. For example, a user may select a specific time by marking 811 one of the calendar dates. Alternatively, the user may select a plurality of dates by dragging (812) a plurality of dates from the calendar with a touch pen. Alternatively, the user may input a date to be selected (813) to an area of the screen.

In 820 of FIG. 8A, when the mobile terminal 100 receives a time, the mobile terminal 100 may provide a feedback 821 requesting content of a schedule related to the selected time. The user can write the name 822 of the schedule related to the selected time in any area of the screen.

In 830 of FIG. 8A, when the mobile terminal 100 receives a schedule name, the mobile terminal 100 may provide a pop-up window 831 for requesting detailed schedule input. At this time, when the user selects an input field 832 for inputting a detailed schedule, the mobile terminal 100 may further provide a screen for entering a detailed schedule.

In 840 of FIG. 8B, the mobile terminal 100 may display a screen including a plurality of input fields 841,842,843,844 as a screen for requesting a detailed schedule. At this time, among the plurality of input fields 841,842,843,844, the name 822 of the schedule that has already been input in 820 of FIG. 8A may be recognized and displayed in the input field 841 that describes the name of the schedule. In 840 of FIG. 8B, the recognition button 846 is a button for starting recognition of a user-written image in each input field to convert to text and insert the converted text into the input field, and the save button 845 is text Is a button for storing the state of the inserted input field, and the cancel button 847 is a screen for returning the state in the input field where the text has already been inserted to a state in which the previous text has not been inserted.

In 850 of FIG. 8B, the mobile terminal 100 may provide respective handwriting input layers 851, 852 and 853 corresponding to the remaining input fields 842, 843, 844 except for the input field 841 in which text has already been inserted. The user may input a schedule-related place, a schedule-related attendee, and a schedule-related other content by using a touch pen on the provided handwriting input layers 851, 852 and 853. At this time, if the user wants to re-enter the handwritten image, the user may select the delete button 854 to delete the handwritten content on the handwriting input layer 851. When the delete button 854 is selected, all of the handwriting written in the handwriting input layer 851 may be deleted, or a recently written stroke during handwriting in the handwriting input layer 851 may be deleted. Meanwhile, when a plurality of handwriting input layers are provided, a delete button may be provided corresponding to each of the plurality of handwriting input layers in order to delete each of the handwriting images on the plurality of handwriting input layers. Alternatively, a plurality of handwriting input layers may be provided as one button for deleting all handwriting.

In 860 of FIG. 8B, the mobile terminal 100 may receive a handwriting image regarding a schedule-related place, a schedule-related attendee, and a schedule-related other content in each of the handwriting input layers 851, 852,853, and convert it into text. In this case, the portable terminal 100 may start handwriting recognition when there is no user's handwriting input for a predetermined period of time or convert the inputted handwriting image into text at regular intervals. Alternatively, the mobile terminal 100 may include a recognition button 861 to convert all of a plurality of handwriting input to each of the plurality of handwriting input layers 851, 852 and 853 into text when the user selects the recognition button 861. have. In addition, the mobile terminal 100 may insert and display texts obtained through recognition into the input fields 842, 843, 844.

In 870 of FIG. 8B, when the user's input using the touch pen for adding a detailed schedule is completed, the portable terminal 100 may display at least a part of the received schedule at a time selected by the user. For example, when a plurality of times provided by the application is a calendar, the portable terminal 100 may display a name and location of the schedule 871 in a column related to a specific date included in the calendar.

In another embodiment of FIGS. 8A and 8B, when the mobile terminal 100 displays a plurality of times provided by the application as shown in 810 of FIG. 8A, when the user selects one of the plurality of times, FIG. 8A Steps 820 and 830 may be omitted and step 840 of FIG. 8B may be directly performed. That is, the mobile terminal 100 may display input fields 841, 842, 843, 844 with contents related to the selected time when one of the plurality of times is selected by the user. In addition, the portable terminal 100 may provide a handwriting input layer corresponding to the remaining fields including the schedule name input field 841, respectively. The portable terminal 100 may receive a handwritten image from each of the provided handwriting input layers, recognize it as text, and insert it into an input field to display it.

The present invention can be actively used for documents conforming to a certain format having an input field. In particular, when the present invention is applied to a government office document, convenience and speed can be provided to a user. For example, in a conventional government office, the complainant writes information necessary for the complaint with a pen in the government office document. Then, the officials of the government office re-enter the contents of the complaints made on the computer in the government office while looking at the contents of the documents prepared by the complaints. However, when applying the present invention, the civil petitioner can easily write the information necessary for the petition through the government office document application provided by the portable terminal 100 at the government office or at home. In other words, the civil petitioner can input necessary information for the petition by using a pen for touch while maintaining usability such as writing with a pen on a government document. In this case, it is possible to save the waste of paper associated with a conventional government office document, and also to reduce the burden of having to re-enter the contents written by the civil servant by the government official. In particular, if the user authentication is confirmed in the portable terminal 100 prior to displaying the government office document application, some of the basic personal information of the user may be displayed as text in some of the input fields of the government office document. In this case, the user can save the trouble of having to fill out the same content every time, such as recognition.

9A and 9B are examples of methods for displaying text in an input field in the mobile terminal 100.

9A is an example of using a touch pen as a user's touch, and FIG. 9B is an example of using a user's finger as a user's touch.

In 910 of FIG. 9A, the mobile terminal 100 may display an application including the input field 911 on the touch screen 140.

At this time, when the user touches the input field 911 using the touch pen or when the user withdraws the touch pen from the mobile terminal 100, as shown in 930 of FIG. 9A, the mobile terminal 100 is for touch A handwriting input layer 931 for receiving handwriting input by a pen may be provided on the input field 911.

As another example, in 950 of FIG. 9B, the mobile terminal 100 may display an application including an input field 951 on the touch screen 140.

At this time, when the user touches with the finger on the input field 951, as shown in 970 of FIG. 9B, the mobile terminal 100 provides a keypad 971 for receiving input using a finger to an area of the screen. You can.

10 is an example of a scenario in which text is displayed in an input field of a web browser application in the mobile terminal 100.

In 1010 of FIG. 10, the mobile terminal 100 may display a web browser application including an input field 1011 for inputting a web address.

In 1030 of FIG. 10, the mobile terminal 100 may provide a handwriting input layer 1031 on the input field 1011. At this time, the handwriting input layer 1031 may be larger than the size of the input field 1011. In this case, the user may perform a touch drag gesture on the handwriting input layer 1031 or adjust the size of the handwriting input layer 1031 using a separate menu. If the user has an amblyopia or is an elderly person with poor vision, such a function may improve accessibility for the user to input a handwritten image.

Next, the portable terminal 100 may receive the user's handwriting input through the provided handwriting input layer 1031, respectively. In 1030 of FIG. 10, the mobile terminal may receive a handwritten image “wwww.google.com” by the user through the handwriting input layer 1031.

In 1050 of FIG. 10, the portable terminal 100 recognizes the input handwriting image and acquires the handwriting image as a test. Then, the converted texts may be inserted into the input fields 1011 and displayed. In 1050 of FIG. 5, the mobile terminal 100 inserts and displays the text “www.google.com” in the input field 1011, which is a recognition result of the handwriting received through the handwriting input layer 1031, and displays the text You can access the web page related to.

11 is a flowchart illustrating a handwriting input method of the portable terminal 100 according to another embodiment of the present invention.

Referring to FIG. 11, the mobile terminal 100 may display an application on the touch screen 140 by executing a specific application (S1101). In this case, a plurality of input fields provided by the application may be displayed on the touch screen 140 (S1102). In addition, the portable terminal 100 may provide a plurality of handwriting input layers corresponding to each of the plurality of input fields in order to receive a user's handwriting input (S1103).

Next, the portable terminal 100 may receive a handwriting image by a pen for touch from each of the plurality of handwriting input layers and convert it into text (S1104). In this case, the portable terminal 100 may convert the handwritten image input according to the handwriting input layer to text by giving priority to the first type among the first language, the second language, numbers, and special symbols. For example, the type of text to be inserted into the input field corresponding to the first handwriting input layer is determined by the first language (eg, Korean), and the type of text to be inserted into the input field corresponding to the second handwriting input layer is When the second language (eg, English) is determined, the mobile terminal 100 determines whether the handwriting input layer is the first handwriting input layer (S1105), and transmits the handwriting image through the first handwriting input layer to the first language. It can be switched to text preferentially (S1106). On the other hand, when the handwriting input rager is not the first handwriting input layer, the portable terminal 100 determines whether the handwriting input layer is the second handwriting input layer (S1108), through the second handwriting input layer The handwritten image may be preferentially converted to text in a second language (S1109).

Next, the portable terminal 100 may insert and display texts recognized in correspondence with each of the plurality of handwriting input layers in the corresponding plurality of input fields (S1107).

12 is a flowchart illustrating a handwriting input method of the portable terminal 100 according to another embodiment of the present invention.

Referring to FIG. 12, the mobile terminal 100 may display an application on the touch screen 140 by executing a specific application (S1201). In this case, an input field provided by the application may be displayed on the touch screen 140 (S1202).

Next, the portable terminal 100 may detect a user's touch on the input field (S1203). Then, the portable terminal 100 may determine whether the user's touch is a finger touch (S1204). When the user's touch is a touch using a finger, the portable terminal 100 may provide a keypad for receiving input using a finger to an area on the screen (S1205). Then, the text selected by the user may be acquired through the keypad (S1206). Next, the portable terminal 100 may insert and display the acquired texts in an input field provided by the application (S1210).

   If the user's touch is not a touch using a finger, the portable terminal 100 may determine whether the user's touch is a touch using a touch pen (S1207). The determination process may be performed simultaneously with step S1204 described above, or step S1204 may be sequentially performed after step S1207. When the user's touch is a touch using a touch pen, the portable terminal 100 may provide a handwriting input layer for receiving input using a touch pen by overlapping the field (S1208). Then, the handwriting image received through the handwriting input layer may be recognized and converted into text (S1209). Next, the portable terminal 100 may insert and display the converted text in an input field provided by the application (S1210).

It will be appreciated that the above-described embodiments of the present invention can be realized in the form of hardware, software, or a combination of hardware and software. Any such software may be volatile or non-volatile storage, such as a storage device such as ROM, or, for example, RAM, memory chip, device or integrated circuit, whether or not it is erasable or rewritable. It may be stored in memory or a storage medium readable by a machine (eg, a computer) while being optically or magnetically recordable, such as, for example, a CD, DVD, magnetic disk, or magnetic tape.

A portable terminal using a touch pen of the present invention and a handwriting input method using the same may be implemented by a computer or a portable terminal including a control unit and a memory, and the memory is a program including instructions for implementing embodiments of the present invention. Or you will see that it is an example of a machine-readable storage medium suitable for storing programs. Accordingly, the present invention includes a program comprising code for implementing the apparatus or method described in any claim of the present specification and a storage medium readable by a machine (computer, etc.) storing the program. In addition, such a program can be transferred electronically through any medium, such as a communication signal transmitted over a wired or wireless connection, and the present invention suitably includes this equivalent. In addition, the portable terminal using the touch pen and the handwriting input method using the touch pen may receive and store the program from a program providing device connected by wire or wirelessly. In addition, the user may selectively use the operation of limiting the operation according to the embodiment of the present invention to the user terminal or by interworking with a server through a network by adjusting the setting of the user portable terminal.

Claims (22)

In the handwriting input method of the mobile terminal,
The mobile terminal executing an application and displaying at least one input field provided by the application on a touch screen of the mobile terminal;
Displaying at least one handwriting input layer for receiving handwriting input by the touch pen on the touch screen as the touch pen is withdrawn from the portable terminal;
Receiving a handwriting image using the touch pen from the at least one handwriting input layer displayed on the touch screen;
Converting the input handwriting image into text; And
And inserting the converted text into the at least one input field and displaying the text on the touch screen.
According to claim 1,
The at least one handwriting input layer is provided by overlapping the at least one input field.
According to claim 2,
When the at least one input field is plural,
Providing the handwriting input layer,
And providing a plurality of handwriting input layers corresponding to each of the plurality of input fields,
The step of converting the handwritten image into text,
And converting a plurality of handwritten images received from each of the plurality of handwriting input layers into text, respectively.
According to claim 3,
When text is already inserted in some of the plurality of input fields,
Providing the handwriting input layer,
And providing a handwriting input layer only in an input field excluding some of the plurality of input fields. According to claim 3,
The step of converting the handwritten image into text,
And when a switch button is selected, converting a plurality of handwritten images received from each of the plurality of handwriting input layers into text at the same time. According to claim 2,
And providing a delete button for deleting the handwritten image input through the at least one handwriting input layer,
And when the delete button is selected, deleting at least a part of the inputted handwriting image. According to claim 1,
The displaying of the at least one input field may include:
Displaying a plurality of times provided by the application;
Selecting at least one of the plurality of times; And
And displaying at least one input field associated with the selected at least one time. According to claim 3,
The step of converting the handwritten image into text,
A first handwriting image input from the first handwriting input layer among the plurality of handwriting input layers is converted into text of a first type among a first language, a second language, numbers, and special symbols,
And converting the second handwriting image input from the second handwriting input layer among the plurality of handwriting input layers into text of a second type different from the first type among the first language, the second language, numbers, and special symbols. Handwriting input method of a mobile terminal. According to claim 1,
The step of converting the input handwritten image to text,
Receiving coordinate values of points touched by EK as the handwriting image is input to the first handwriting input layer among the at least one handwriting input layer;
Storing the coordinate values of the points as a stroke;
Generating a stroke array list using the stroke; And
And converting the input handwriting image into text using a pre-stored handwriting library and the stroke array list. According to claim 1,
Receiving a user's touch by a body in the at least one input field;
Displaying a keypad based on a user's touch input by the body
Receiving text through key input of the keypad, and
And inserting and displaying the inputted text in the input field. delete In a portable terminal using a touch pen,
touch screen; And
At least one processor,
The at least one processor executes an application, displays at least one input field provided by the application on the touch screen, and writes by the touch pen on the touch screen as the touch pen is withdrawn from the portable terminal. Displaying at least one handwriting input layer for receiving input, receiving a handwriting image using the touch pen from at least one handwriting input layer displayed on the touch screen, converting the input handwriting image to text, A mobile terminal configured to insert the converted text into the at least one input field and display it on the touch screen. The method of claim 12,
The at least one processor is configured to control at least one handwriting input layer to be displayed overlapping the at least one input field. The method of claim 13,
When the at least one input field is plural,
The at least one processor displays a plurality of handwriting input layers corresponding to each of the plurality of input fields, and converts a plurality of handwriting images received from each of the plurality of handwriting input layers into text, respectively. terminal. The method of claim 14,
The at least one processor may display a handwriting input layer in at least one input field excluding the first field among the plurality of input fields when text is already inserted in the first field among the plurality of input fields. A portable terminal characterized by. The method of claim 14,
The at least one processor,
When a convert button is selected, a portable terminal characterized in that a plurality of handwritten images received from each of the plurality of handwriting input layers are simultaneously converted to text. The method of claim 13,
The at least one processor further provides a delete button for deleting the input handwriting image, and when the delete button is selected, deletes at least a part of the input handwriting. The method of claim 12,
The at least one processor displays a plurality of times provided by the application on the touch screen, and when at least one of the plurality of times is selected, displays at least one input field associated with the selected at least one time. A portable terminal characterized by displaying. The method of claim 14,
The at least one processor converts the first handwriting image input from the first handwriting input layer among the plurality of handwriting input layers into text of a first language, a second language, numbers, and special symbols, and text of the first type, A portable terminal for converting a second handwriting image input from a second handwriting input layer among the plurality of handwriting input layers into text of a second type different from a first type among a first language, a second language, numbers, and special symbols. The method of claim 12,
The at least one processor receives coordinate values of points touched on the touch screen as the handwriting image is input to the at least one handwriting input layer, stores the coordinate values of the points as a stroke, and stores the stroke. A portable terminal for generating a stroke array list using a pre-stored handwriting library and converting the inputted handwritten image into text using the stroke array list. The method of claim 12,
The at least one processor,
Receive a user's touch by the body in the at least one input field, display a keypad on the touch screen based on the user's touch input by the body, and input the at least one text received through the keypad A mobile terminal inserted and displayed in a field. delete

Images