KR20210110471A - Electronic device and operating method for performing an operation based on a handwritten input - Google Patents

Abstract

Provided is a method for performing operations based on handwritten input in an electronic device. The electronic device receives handwritten input of a user on a lock screen of the electronic device, obtains input information corresponding to the handwritten input, determines whether to perform user authentication based on the input information, performs user authentication for the handwritten input based on the determination, and performs an operation corresponding to the input information based on the result of user authentication.

Description

Electronic device and operating method for performing an operation based on handwriting input and operating method thereof

The present disclosure relates to an electronic device that performs an operation based on a handwriting input and a method of operating the same.

A user may perform various operations using various types of electronic devices. For example, when the electronic device is in the locked state, after the electronic device is unlocked by user authentication, an application is performed through an interface (eg, home screen, application list screen) of the unlocked electronic device. This can be done. Accordingly, an operation desired by the user may be performed through an application executed after the lock state is released.

However, until the user executes the application related to the desired operation, various processes such as user authentication for unlocking the lock state and execution of the application through the interface are required. may take

Accordingly, it is necessary to provide a method by which an operation desired by a user can be performed more quickly and easily according to a user input.

SUMMARY OF THE INVENTION An object of the present disclosure is to solve the above-described problem, and to provide an electronic device that performs an operation based on a handwriting input and a method of operating the same.

Another object of the present invention is to provide a computer-readable recording medium in which a program for executing the method in a computer is recorded. The technical problem to be solved is not limited to the technical problems as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, a first aspect of the present disclosure provides a method of performing an operation based on a handwriting input in an electronic device, wherein, on a lock screen of the electronic device, a user's handwriting input is provided. receiving; obtaining input information corresponding to the handwriting input; determining whether to perform user authentication based on the input information; performing user authentication for the handwriting input based on the determination; and performing an operation corresponding to the input information based on a result of the user authentication.

Also, a second aspect of the present disclosure provides an electronic device performing an operation based on a handwriting input, comprising: a user input unit configured to receive a user's handwriting input on a lock screen of the electronic device; obtaining input information corresponding to the handwriting input, determining whether to perform user authentication based on the input information, performing user authentication on the handwriting input based on the determination, and performing the user authentication at least one processor configured to perform an operation corresponding to the input information based on the performed result; and an output unit outputting a result of the operation being performed.

In addition, a third aspect of the present disclosure may provide a recording medium in which a program for performing the method of the first aspect is stored.

According to an embodiment, it is possible to quickly perform an operation only by inputting a handwriting without running a separate application for performing the operation.

1 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.
2 is a flowchart illustrating a method of performing an operation based on a handwriting input according to an exemplary embodiment.
3 is a block diagram illustrating an internal configuration of an electronic device according to an exemplary embodiment.
4 is a block diagram illustrating an internal configuration of an electronic device according to an exemplary embodiment.
5 is a diagram illustrating an example of determining at least one name corresponding to a handwriting input according to an embodiment.
6 is a diagram illustrating an example of determining at least one name corresponding to a handwriting input according to an embodiment.
7 is a diagram illustrating a method of classifying at least one stroke as a picture or text according to an exemplary embodiment.
8 is a diagram illustrating a method of recognizing a text from a stroke classified as a text, according to an exemplary embodiment.
9 is an exemplary diagram illustrating an example of extracting at least one name from text according to an embodiment.
10 is a diagram illustrating an example of performing an operation based on a handwriting input according to an exemplary embodiment.
11 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.
12 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.
13 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.
14 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.
15 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.
16 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.
17 is a diagram illustrating an example of performing an operation of transmitting a message based on a handwriting input received on a lock screen according to an embodiment.
18 is a diagram illustrating an example of performing an operation of executing an application based on a handwriting input received on a lock screen according to an embodiment.
19 is a diagram illustrating an example of performing an operation by displaying at least one icon based on a handwriting input received on a lock screen according to an embodiment.
20 is a diagram illustrating an example of allocating a tag to a received handwriting input on a lock screen according to an embodiment.
21 is a diagram illustrating an example of a handwriting input to which a tag is assigned according to an embodiment.
22 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.
23 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.
24 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that other components may be further included, rather than excluding other components, unless otherwise stated.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

Referring to FIG. 1 , the electronic device 1000 may receive a writing input 110 by a writing tool 120 on a lock screen. In response to receiving the user's handwriting input 110 , the electronic device 1000 according to an embodiment may perform an operation corresponding to the handwriting input 110 .

For example, the user may input the handwriting input 110 corresponding to "123+12=" to the electronic device 1000 on the lock screen. The electronic device 1000 may determine that user authentication is not required because the operation of calculating the equation of "123+12=" does not use the user's personal information. Accordingly, the electronic device 1000 may display “123+12=133” on the lock screen as a result of the operation of calculating the equation without user authentication.

The electronic device 1000 according to an embodiment is a device capable of receiving the handwriting input 110 and performing an operation, and may be implemented in various forms. For example, the electronic device 1000 described herein may include a digital camera, a smart phone, a laptop computer, a tablet PC, an electronic book terminal, a digital broadcasting terminal, and a personal digital assistant (PDA). , a Portable Multimedia Player (PMP), a navigation system, an MP3 player, a vehicle, and the like, but is not limited thereto. The electronic device 1000 described herein may be a wearable device that can be worn by a user. Wearable devices include accessory type devices (e.g., watches, rings, wristbands, ankle bands, necklaces, eyeglasses, contact lenses), head-mounted-devices (HMDs), textile or clothing-integrated devices (e.g., electronic clothing), a body attachable device (eg, a skin pad), or a bioimplantable device (eg, an implantable circuit). Hereinafter, for convenience of description, a case in which the electronic device 1000 is a smart phone will be described as an example.

The electronic device 1000 according to an embodiment is not limited to the example of receiving the writing input 110 by the writing tool 120 on the lock screen, and receives the writing input 110 through various means and methods. can do. For example, the electronic device 1000 may receive the handwriting input 110 based on the user's touch input.

In the electronic device 1000 according to an embodiment, when user authentication for an operation corresponding to the handwriting input 110 input in the lock screen state is unnecessary, the user authentication is performed without user authentication for unlocking the electronic device 1000 . can perform the intended operation.

Also, the electronic device 1000 according to an embodiment may automatically determine and execute an application related to an operation corresponding to the handwriting input 110 . Accordingly, after the user releases the lock state of the electronic device 1000 , the user does not need to select and execute an application for performing an operation, and only perform the user's intended operation in the electronic device 1000 by using the handwriting input 110 . This can be done in a locked state.

2 is a flowchart illustrating a method of performing an operation based on a handwriting input according to an exemplary embodiment.

Referring to FIG. 2 , in operation 210 , when the electronic device 1000 is in the locked state, the electronic device 1000 may receive a user's handwriting input through the lock screen.

When the electronic device 1000 according to an embodiment is in the locked state, the displayed lock screen may include a screen in which the display screen of the electronic device 1000 is turned off. Also, the lock screen according to an embodiment may include a screen in which only limited functions and information are provided while the display screen of the electronic device 1000 is turned on. The lock screen according to an embodiment is not limited to the above-described example, and before user authentication is performed, for security of the electronic device 1000, functions and information provision may be restricted, and various types of screens may be used. have.

The electronic device 1000 according to an embodiment may not perform an operation according to the touch input even if a user's touch input is received when in the locked state. For example, the electronic device 1000 may acquire a user's touch input received through various methods such as a user's finger or a touch tool.

However, according to an embodiment, when the writing input 110 by the writing tool 120 is received, even though the electronic device 1000 is in the locked state, the operation according to the writing input 110 is performed in steps 220 to 250 below. So it can be done. For example, the electronic device 1000 may be in a state in which a writing input 110 by the writing tool 120 can be received (eg, the writing tool 120 is separated from the electronic device 1000 , or the writing tool When the power of 120 is turned on or it is determined that the writing tool 120 is connected to the electronic device 1000 through short-distance communication), the writing input 110 according to an embodiment is displayed on the lock screen. ) can be received through

Accordingly, according to an embodiment, the handwriting input 110 may be received by the electronic device 1000 without authentication for unlocking the electronic device 1000 , and an operation according to the handwriting input 110 may be performed, In a locked state, it can be performed.

In operation 220 , the electronic device 1000 may obtain input information corresponding to the handwriting input received in operation 210 . The input information according to an embodiment may include at least one text corresponding to the handwriting input.

When the handwriting input according to an embodiment includes at least one character, the input information may include text corresponding to the at least one character of the handwriting input. For example, as a result of recognizing at least one character from the handwriting input, at least one text corresponding to the recognized at least one character may be obtained as input information. For example, when the handwriting input includes the character “Cup”, at least one text corresponding to “Cup” may be obtained as input information.

Also, when the handwriting input according to an embodiment includes at least one picture, the input information may include text corresponding to the at least one picture of the handwriting input. For example, as a result of recognizing a picture of a handwriting input, at least one text corresponding to a word corresponding to the picture may be obtained as input information. For example, when a picture of a cat is recognized in the handwriting input, at least one text corresponding to “cat” may be acquired as input information.

According to an embodiment, the operation of recognizing a character or a picture from a handwriting input may be performed based on a pre-trained artificial intelligence model. For example, based on features extracted from the handwriting input, a pre-trained artificial intelligence model (ex. Convolutional Neural Network (CNN), Deep Neural Network), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN), or Deep Q-Networks) may be used. have.

The operation of recognizing a character or a picture from a handwriting input may be performed according to various methods, without being limited to the above-described example.

In operation 230 , the electronic device 1000 may determine whether to perform user authentication based on the input information obtained in operation 220 .

In an embodiment, whether to perform user authentication may be determined according to a user's intention determined based on an entity corresponding to at least one text included in the input information.

A name according to an embodiment may be determined based on a result of classification according to a meaning indicated by at least one text. For example, the name may be determined as a name, date, number, symbol, mathematical expression, time, or the like.

The name according to an embodiment may include a name of a lower level and a name of a higher level. The name of the lower level according to an embodiment may include a name that can be directly determined from at least one text. For example, the name of the lower level may include numbers, letters, symbols, and the like. The name of the higher level according to an embodiment may include a name that can be determined from the name of the lower level. For example, the name of the upper level may include a title, date, name, equation, time, phone number, and the like.

For example, when the name of the upper level is a date, the date that is the name of the upper level from at least one text based on a template for a pre-stored date including a method in which the date can be expressed as a combination of numbers and symbols can be extracted.

In addition, the user's intention according to an embodiment may indicate an action desired by the user, which may be inferred from at least one name. For example, when the name of the input information is a title and a time, it may be determined that the user's intention is “schedule storage”.

Not limited to the above example, the user's intention according to an embodiment is based on various context information about the user, such as the user's preference, the user's surrounding environment, and an operation that can be performed by the electronic device 1000 as well as the name. Thus, it can be determined.

According to an embodiment, it may be determined whether user authentication is performed according to a user intention corresponding to input information. For example, when an operation determined according to a user intention corresponding to input information is performed or an application to which the operation is to be performed is executed, it may be determined whether user authentication is required.

Whether user authentication is required according to an embodiment may be determined according to whether an operation is performed using sensitive information of a user or an application requiring user authentication. For example, in the case of an operation performed using the user's personal information, it may be determined that the operation requires user authentication. In addition, in the case of an operation performed through an application requiring user authentication (eg, a banking app, a payment app), it may be determined as an operation requiring user authentication before the application is executed.

It is not limited to the above-described example, and whether user authentication is required may be determined according to various methods based on various information. Also, according to an embodiment, an operation or application requiring user authentication may be preset by the user.

In operation 240 , the electronic device 1000 may perform user authentication based on the result determined in operation 230 .

User authentication according to an embodiment may include user authentication that may be performed to release the lock state of the electronic device 1000 . For example, user authentication may include various authentication methods capable of authenticating a user, such as pattern authentication, fingerprint authentication, iris authentication, and face authentication.

According to an embodiment, user authentication may be performed according to a preset authentication method for each operation, or user authentication may be performed according to a security level determined for each operation.

According to an embodiment, user authentication with different security levels may be performed according to an operation to be performed. For example, in the case of an operation requiring a relatively high security level, high security level biometric authentication (eg, fingerprint authentication, iris authentication, face authentication) may be performed. In addition, in the case of an operation requiring a relatively low security level, password authentication or pattern authentication having a low security level may be performed.

In operation 250 , the electronic device 1000 may perform an operation corresponding to the input information based on the result of user authentication performed in operation 240 .

According to an embodiment, an operation corresponding to the input information may be determined according to a user intention corresponding to the input information. For example, a user intention corresponding to the input information may be determined, and an operation corresponding to the user intention may be performed.

Also, an operation corresponding to the input information according to an embodiment may be performed by executing an application for performing the operation.

According to an embodiment, an application for performing an operation may be automatically executed based on a user's intention. For example, among a plurality of applications installed in the electronic device 1000 , an application capable of performing an operation corresponding to a user's intention is executed, whereby the operation may be performed.

According to an embodiment, the operation corresponding to the input information may be performed by the electronic device 1000 according to various methods, without being limited to the above-described example.

3 is a block diagram illustrating an internal configuration of the electronic device 1000 according to an exemplary embodiment.

4 is a block diagram illustrating an internal configuration of the electronic device 1000 according to an embodiment.

Referring to FIG. 3 , the electronic device 1000 may include a processor 1300 , a user input unit 1100 , and an output unit 1200 . However, not all of the components shown in FIG. 3 are essential components of the electronic device 1000 . The electronic device 1000 may be implemented by more components than those illustrated in FIG. 3 , or the electronic device 1000 may be implemented by fewer components than those illustrated in FIG. 3 .

For example, as shown in FIG. 4 , the electronic device 1000 includes a sensing unit (in addition to the processor 1300 , the user input unit 1100 and the output unit 1200 ). 1400 ), a communication unit 1500 , an A/V input unit 1600 , and a memory 1700 may be further included.

The user input unit 1100 means a means for a user to input data for controlling the electronic device 1000 . For example, the user input unit 1100 includes a key pad, a dome switch, and a touch pad (contact capacitive method, pressure resistance film method, infrared sensing method, surface ultrasonic conduction method, integral type). There may be a tension measurement method, a piezo effect method, etc.), a jog wheel, a jog switch, and the like, but is not limited thereto.

According to an embodiment, the user input unit 1100 may receive the user's handwriting input 110 . For example, the user input unit 1100 may be a touch pad, a touch tool, or a writing tool capable of receiving the user's handwriting input 110 .

The output unit 1200 may output an audio signal, a video signal, or a vibration signal, and the output unit 1200 may include a display unit 1210 , a sound output unit 1220 , and a vibration motor 1230 . have. The output unit 1200 according to an embodiment may output a result of performing an operation based on input information corresponding to the handwriting input 110 .

The display unit 1210 displays and outputs information processed by the electronic device 1000 . According to an embodiment, the display 1210 may display the handwriting input 110 received from the electronic device 1000 and input information (eg, text) corresponding to the handwriting input 110 . Also, the display unit 1210 according to an embodiment may display a result of performing an operation based on input information corresponding to the handwriting input 110 .

On the other hand, when the display unit 1210 and the touch pad form a layer structure to form a touch screen, the display unit 1210 may be used as an input device in addition to an output device. The display unit 1210 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display ( 3D display) and electrophoretic display (electrophoretic display) may include at least one. Also, depending on the implementation form of the electronic device 1000 , the electronic device 1000 may include two or more display units 1210 .

The sound output unit 1220 outputs audio data received from the communication unit 1500 or stored in the memory 1700 . According to an embodiment, the sound output unit 1220 may output audio data related to a result of performing an operation based on the handwriting input 110 .

The vibration motor 1230 may output a vibration signal. Also, the vibration motor 1230 may output a vibration signal when a touch is input to the touch screen. According to an embodiment, the vibration motor 1230 may output a vibration signal related to a result of performing an operation based on the handwriting input 110 .

The processor 1300 generally controls the overall operation of the electronic device 1000 . For example, the processor 1300 executes programs stored in the memory 1700 , and thus the user input unit 1100 , the output unit 1200 , the sensing unit 1400 , the communication unit 1500 , and the A/V input unit 1600 . ) can be controlled in general.

The electronic device 1000 may include at least one processor 1300 . For example, the electronic device 1000 may include various types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), and a neural processing unit (NPU).

The processor 1300 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. The command may be provided to the processor 1300 from the memory 1700 or may be received through the communication unit 1500 and provided to the processor 1300 . For example, the processor 1300 may be configured to execute instructions according to program codes stored in a recording device such as a memory.

The processor 1300 according to an embodiment may obtain input information corresponding to the received user's handwriting input 110 from the lock screen of the electronic device 1000 . Also, the processor 1300 may determine whether to perform user authentication in order to perform an operation corresponding to the handwriting input 110 based on the input information, and perform user authentication based on the determination. .

In addition, the processor 1300 according to an embodiment may perform user authentication based on a user intention corresponding to the input information. For example, the processor 1300 may determine at least one name corresponding to at least one text included in the input information, and determine a user's intention corresponding to the input information based on the at least one name. . Also, the processor 1300 may determine whether to perform user authentication based on the determined intention of the user.

Also, the processor 1300 according to an embodiment may perform an operation corresponding to the input information based on a result of user authentication.

The sensing unit 1400 may detect a state of the electronic device 1000 or a state around the electronic device 1000 , and transmit the sensed information to the processor 1300 .

The sensing unit 1400 includes a geomagnetic sensor 1410 , an acceleration sensor 1420 , a temperature/humidity sensor 1430 , an infrared sensor 1440 , a gyroscope sensor 1450 , and a position sensor. (eg, GPS) 1460 , a barometric pressure sensor 1470 , a proximity sensor 1480 , and at least one of an illuminance sensor 1490 , but is not limited thereto.

The sensing unit 1400 according to an embodiment may detect various types of sensor information as user context information that may be used to determine a user's intention corresponding to the handwriting input 110 .

The communication unit 1500 may include one or more components that allow the electronic device 1000 to communicate with the server 2000 or an external device (not shown). For example, the communication unit 1500 may include a short-range communication unit 1510 , a mobile communication unit 1520 , and a broadcast receiving unit 1530 .

Short-range wireless communication unit 1510, Bluetooth communication unit, BLE (Bluetooth Low Energy) communication unit, short-range wireless communication unit (Near Field Communication unit), WLAN (Wi-Fi) communication unit, Zigbee (Zigbee) communication unit, infrared ( It may include an IrDA, infrared Data Association) communication unit, a Wi-Fi Direct (WFD) communication unit, an ultra wideband (UWB) communication unit, an Ant+ communication unit, and the like, but is not limited thereto.

The mobile communication unit 1520 transmits/receives a radio signal to and from at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to transmission/reception of a voice call signal, a video call signal, or a text/multimedia message.

The broadcast receiver 1530 receives a broadcast signal and/or broadcast-related information from the outside through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. According to an embodiment, the electronic device 1000 may not include the broadcast receiver 1530 .

According to an embodiment, the communication unit 1500 may transmit/receive data to and from an external device or server. For example, the communication unit 1500 may receive data necessary to perform an operation corresponding to the handwriting input 110 according to an embodiment, from an external device or a server.

The A/V (Audio/Video) input unit 1600 is for inputting an audio signal or a video signal, and may include a camera 1610 , a microphone 1620 , and the like. The camera 1610 may obtain an image frame such as a still image or a moving image through an image sensor in a video call mode or a shooting mode. The image captured through the image sensor may be processed through the processor 1300 or a separate image processing unit (not shown). The microphone 1620 receives an external sound signal and processes it as electrical voice data.

The audio/video (A/V) input unit 1600 according to an embodiment may generate image data or audio data as data that may be used to perform an operation corresponding to the handwriting input 110 .

The memory 1700 may store a program for processing and control of the processor 1300 , and may also store data input to or output from the electronic device 1000 .

The memory 1700 according to an embodiment may store various types of data required to perform an operation corresponding to the handwriting input 110 .

The memory 1700 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (eg, SD or XD memory), and a RAM. (RAM, Random Access Memory) SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk , may include at least one type of storage medium among optical disks.

Programs stored in the memory 1700 may be classified into a plurality of modules according to their functions, for example, may be classified into a UI module 1710 , a touch screen module 1720 , a notification module 1730 , and the like. .

The UI module 1710 may provide a specialized UI, GUI, or the like that interworks with the electronic device 1000 for each application. The touch screen module 1720 may detect a touch gesture on the user's touch screen and transmit information about the touch gesture to the processor 1300 . The touch screen module 1720 according to some embodiments may recognize and analyze a touch code. The touch screen module 1720 may be configured as separate hardware including a controller.

Various sensors may be provided inside or near the touch screen to detect a touch or a proximity touch of the touch screen. A tactile sensor is an example of a sensor for detecting a touch of a touch screen. A tactile sensor refers to a sensor that senses a touch of a specific object to the extent or higher than that felt by a human. The tactile sensor may sense various information such as the roughness of the contact surface, the hardness of the contact object, and the temperature of the contact point.

The user's touch gesture may include a tap, touch & hold, double tap, drag, pan, flick, drag and drop, swipe, and the like.

The notification module 1730 may generate a signal for notifying the occurrence of an event in the electronic device 1000 .

5 is a diagram illustrating an example of determining at least one name corresponding to a handwriting input according to an embodiment.

Referring to FIG. 5 , when the handwriting input 110 is received, the electronic device 1000 may analyze the received handwriting input 110 in operation 510 .

The handwriting input 110 according to an embodiment may be a set of strokes input in a predetermined time order.

A “stroke” may mean a single trajectory of drawing while maintaining a touch from a point in time when an input tool touches the electronic device 1000 . For example, in ‘3x + 6y = 5’, if the user draws each of 3, x, 6, and y at once while attracting a touch, each of 3, x, 6, and y may constitute one stroke. In the case of ‘+’, since the user draws ‘-’ first and then ‘ㅣ’, ‘-’ and ‘ㅣ’ can constitute one stroke, respectively. According to an embodiment, one stroke may constitute one character or symbol or a combination of a plurality of strokes may constitute one character or symbol.

The electronic device 1000 according to an embodiment may classify the strokes included in the handwriting input 110 into the text 511 or the picture 512 . For example, the strokes may be classified based on the appearance characteristics of the strokes. Strokes classified as text 511 may include strokes that can be recognized as at least one character. Also, the strokes classified as the picture 512 may include strokes that can be recognized as at least one picture.

In operation 520 , the electronic device 1000 may recognize at least one text corresponding to strokes classified as the text 511 or the picture 512 . According to an embodiment, at least one text 521 corresponding to strokes classified as text 511 may be recognized. Also, at least one text corresponding to the strokes classified as the figure 512 may be recognized.

The electronic device 1000 according to an embodiment may obtain at least one text 521 from the strokes classified as the text 511 by comparing characteristics of various texts with characteristics of the strokes. Also, the electronic device 1000 according to an embodiment may determine the figure 512 corresponding to the strokes by comparing the features of various figures with the features of the strokes from the strokes classified as the figure 512 . The electronic device 1000 may obtain the word representing the determined picture 512 as at least one text for strokes classified as the picture 512 .

Without being limited to the above-described example, the electronic device 1000 may acquire at least one text from strokes according to various methods.

However, as in the illustrated example, when the text corresponding to the strokes classified as the figure 512 cannot be recognized 522, in determining the name according to an embodiment, the strokes classified as the figure 512 may not be used.

In 530 , the electronic device 1000 may extract a name corresponding to the text 521 recognized in 520 . The name according to an embodiment may be determined based on a preset template corresponding to each name.

A template for a lower-level name according to an embodiment may include features that can be directly determined from text, such as symbols, numbers, and characters.

In addition, the template for the higher-level name according to an embodiment may include information about a display format of at least one lower-level name, such as a symbol, number, or word. For example, a template for "hour" would be "(2 consecutive digits 24 or less):(2 consecutive 60 digits or less)", "(1-2 consecutive 24 or less digits)" It may include information about at least one expression method for expressing "hour" such as "number) hour (a number of 1 to 2 consecutive digits 60 or less) minutes". Also, the "template for "title" may include "(words within 10 characters)" as a way in which the "title" can be expressed.

Not limited to the above-described example, the templates of lower-level and higher-level names may include information on various types of expression methods for texts for each name.

According to an embodiment, a lower-level name for “Meeting” may be determined as a word, and a higher-level name may be determined as a “title” 531 based on a pre-stored name template. In addition, the lower level name for "16:00" may be determined by numbers and symbols, and the upper level name may be determined as "time 532" based on a pre-stored name template.

In 540 , the electronic device 1000 may determine a user intention based on the lower level and higher level names determined in 530 , and perform an operation corresponding to the user intention.

The user intention according to an embodiment may be determined based on at least one of a lower level name and a higher level name. For example, a user's intention for "title + time" may be determined by adding "schedule" information.

According to an embodiment, since the "operation of adding schedule information" does not correspond to an operation using sensitive information of a user, an operation of adding schedule information through a schedule management application without performing user authentication ( 542 ) This can be done.

Accordingly, according to an embodiment, according to the user's intention determined to "add schedule information", the electronic device 1000 displays the "schedule information" through the schedule management application, the text 521 and the picture 512 recognized at 520 . can be added based on For example, the title is "Meeting", and as time information, at 4 pm on the day (eg, the day the handwriting input is received), a new "schedule information", including a picture 512 as a related memo, is added to the schedule management application. An operation 542 added to may be performed.

6 is a diagram illustrating an example of determining at least one name corresponding to a handwriting input according to an embodiment.

Referring to FIG. 6 , when the handwriting input 110 is received, the electronic device 1000 may analyze the received handwriting input 110 at step 610 .

The electronic device 1000 according to an embodiment may classify strokes included in the handwriting input 110 as a picture 611 or a text 612 . For example, the strokes may be classified based on the appearance characteristics of the strokes.

In operation 620 , the electronic device 1000 may recognize at least one text 621 and 622 corresponding to strokes classified as a picture 611 or text 612 . According to an embodiment, at least one text 621 corresponding to the strokes classified as the figure 611 may be recognized. Also, at least one text 622 corresponding to strokes classified as text 612 may be recognized.

In 630 , the electronic device 1000 may extract a name 631 corresponding to the text 621 recognized in 620 . The name according to an embodiment may be determined based on a preset template corresponding to each name.

According to an embodiment, a lower-level name among the names for “Cup” may be determined as “word” based on a pre-stored name template. Also, a higher-level name for “Cup” may be determined as “Title” 631 . In addition, the lower level and upper level names for "17:00" may be determined by a number, a symbol, and "time 632", respectively.

In 640 , the electronic device 1000 may determine a user intention based on the lower-level and upper-level names determined in 630 , and perform an operation corresponding to the user intention.

The user intention according to an embodiment may be determined based on at least one of a lower level name and a higher level name. For example, a user's intention for "title + time" may be determined by adding "schedule" information.

According to an embodiment, since the "operation of adding schedule information" does not correspond to an operation using sensitive information of a user, adding schedule information through a schedule management application without performing user authentication ( 542 ) This can be done.

Accordingly, according to an embodiment, according to the user's intention determined to "add schedule information", the electronic device 1000 provides "schedule information" based on the texts 621 and 622 recognized in 620 through the schedule management application. can be added For example, an operation of adding new "schedule information" in the schedule management application, including the title "Cup" and 5:00 pm on the day (eg, the day the handwriting input is received) as time information, may be performed. have.

7 is a diagram illustrating a method of classifying at least one stroke as a picture or text according to an exemplary embodiment.

Referring to FIG. 7 , the electronic device 1000 may classify strokes 710 included in the handwriting input 110 into one of a picture 770 and a text 780 . According to an embodiment, the handwriting input analysis 510 and 610 of FIGS. 5 and 6 may be performed according to the stroke classification method illustrated in FIG. 7 .

In operation 720 , the electronic device 1000 may extract a feature of each stroke from a stroke sequence in which the strokes 710 are sequentially arranged. A stroke sequence according to an embodiment may be generated by dividing strokes according to a region in which a stroke is displayed and then sequentially arranging the strokes divided by regions.

For example, in 710, strokes representing the "sun"-shaped picture and strokes representing each letter of "H", "a", ... , "!" , a stroke sequence may be generated. As an example, the stroke sequence may be generated by arranging each stroke in order based on a time point at which it is received as a handwriting input.

In the control block 730 , the electronic device 1000 may classify the strokes included in the stroke sequence into at least one group based on the characteristics of each stroke included in the extracted stroke sequence, in step 720 . For example, the strokes may be classified into at least one group according to whether the similarity between the strokes is within a reference value or exists in an adjacent position between the strokes based on the characteristics of the strokes.

For example, as for strokes included in the stroke sequence of 710 , at 730 , strokes representing a “sun”-shaped figure are classified into a first group, and strokes representing “H” are classified into a first group, “a”, . .. , strokes of “!” may be classified into a second group to a twelfth group.

The stroke buffer of 740 may temporarily store the stroke sequence of 710 and information about each group classified by the control block of 730 .

In operation 750 , the electronic device 1000 may extract features of strokes belonging to each group based on the stroke sequence stored in the stroke buffer and information about each group.

In operation 760 , the electronic device 1000 may classify the strokes into a picture 770 and a text 780 based on the features extracted for each group. According to an embodiment, as it is determined that the feature extracted from the first group is a feature close to a stroke expressed in a figure, the strokes of the first group may be classified as a figure 770 . In addition, as each feature extracted from the second group to the twelfth group is determined to be a feature close to a stroke expressed in text, the strokes of the second group to the twelfth group may be classified as text 780 . have.

8 is a diagram illustrating a method of recognizing a text from a stroke 810 classified as a text 780, according to an exemplary embodiment.

Referring to FIG. 8 , the electronic device 1000 may recognize a text 870 from a stroke 810 classified as a text 780 in FIG. 7 . According to an embodiment, the text recognition 520 and 620 of FIGS. 5 and 6 may be performed according to the method of recognizing the text shown in FIG. 8 .

In 820 , the electronic device 1000 may divide the at least one stroke 810 into at least one line. For example, “e” and “t” of “Meeting” 810 may be displayed as one stroke, but a plurality of lines (eg, “e”) may correspond to lines that may be composed of text. lines corresponding to ", and lines corresponding to "-" and "|" of t).

In operation 830 , the electronic device 1000 may perform preprocessing on the divided lines by adjusting, rotating, or moving the sizes of the plurality of lines divided in 820 . The preprocessing of 830 according to an embodiment may be performed so that features of lines can be clearly extracted in 840 below.

In 840 , the electronic device 1000 extracts features from the plurality of preprocessed lines in 840 , and in 850 , recognizes at least one text based on the extracted features.

In operation 860 , the electronic device 1000 may finally obtain the text 870 corresponding to the stroke 810 by correcting an error included in the recognized result and performing post-processing. Post-processing according to an embodiment may include an operation that may be performed to minimize an error that may occur in obtaining the text 870 corresponding to the stroke 810 .

Without being limited to the above-described example, the electronic device 1000 may recognize the text 870 from the stroke 810 of the handwriting input 110 through various methods.

9 is an exemplary diagram illustrating an example of extracting at least one name from the text 910 according to an embodiment.

Referring to FIG. 9 , the electronic device 1000 may extract at least one name from texts 870 and 910 recognized in the handwriting input 810 in FIG. 8 . According to an embodiment, the name extraction 530 and 630 of FIGS. 5 and 6 may be performed according to the method of extracting the name shown in FIG. 9 .

In operation 920 , the electronic device 1000 may classify the text 910 as a minimum unit of a word that can be used independently. For example, the text 910 may be divided into “Meeting”, “1”, “6”, “:”, “0”, and “0”.

In operation 930 , the electronic device 1000 may extract at least one name based on a template for a predefined name included in the template set 940 according to the result of the classification of the text 910 in operation 920 . have.

The template set 940 according to an embodiment may include information on how each name is expressed. For example, the template set 940 may include, for each name, a number template, a symbol template, a name template, a date template, a mathematical expression template, and various other templates.

Also, the template set 940 according to an embodiment may further include a template for a name of a higher level that may be obtained based on the name of the lower level as well as the name of the lower level. The template for the name of the higher level according to an embodiment may include information on a method of expressing the name of the higher level based on the name of the lower level.

As an example, the name template includes information about various formats in which names can be expressed, such as pre-stored names (ex. Mark, Donald) or "Mr/Mrs. may include. Also, the date template is "(numbers from 1 to 12)month (numbers from 1 to 31)day", "(numbers up to four digits).(numbers from 1 to 12).(numbers from 1 to 31" number of)", etc., may include information about various formats in which dates can be expressed. Without being limited to the above example, the template set 940 may include various templates for various types of names.

10 is a diagram illustrating an example of performing an operation based on a handwriting input according to an exemplary embodiment.

Referring to FIG. 10 , at 1001 , the electronic device 1000 may receive a handwriting input 1001-1 on a lock screen. According to an embodiment, the electronic device 1000 detects that the writing tool 1001 - 2 is activated or that the writing tool 1001 - 2 and the electronic device 1000 are connected to each other (eg, Bluetooth connection). A handwriting input 1001-1 may be received through the tool 1001-2.

In steps 1002 and 1003 , the electronic device 1000 may analyze the handwriting input 1001-1 to obtain at least one text corresponding to the handwriting input 1001-1. According to an embodiment, a text corresponding to the handwriting input 1001-1, "123+12=" may be obtained.

In operation 1004 , the electronic device 1000 may extract the name of the lower level from the text based on the pre-stored template for the name of the lower level. The electronic device 1000 according to an embodiment may extract “number”, which is a name for “123” and “12”, and an “operator,” which is a name for “+” and “=" from among texts. can

In operation 1005 , the electronic device 1000 may extract the name of the higher level from the name of the lower level based on a template for the name of the higher level stored in advance. The electronic device 1000 according to an embodiment includes a method in which “Equation” is expressed and As a match, it is possible to extract "Equation" as a higher-level name.

In 1006 , the electronic device 1000 determines the user intention as “calculation of a mathematical expression” based on the name of the higher level, and performs an operation according to the user intention, thereby including the result obtained by the mathematical expression in operation 1007 . You can create an answer. Since the "calculation of mathematical expression" operation according to an embodiment is an operation that does not use the user's sensitive personal information, it may be performed without user authentication.

In the operation according to “calculation of equation” according to an embodiment, after the operation of calculating the equation is performed in the background of the electronic device 1000 , the result 1008 - 1 is displayed in 1008 of the electronic device 1000 . It may be displayed on the lock screen.

11 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

Referring to FIG. 11 , at 1101 , the electronic device 1000 may receive a handwriting input 1101-1 on the lock screen.

In steps 1102 and 1103 , the electronic device 1000 may obtain at least one text corresponding to the handwriting input 1101-1 by analyzing the handwriting input 1101-1. According to an embodiment, a text corresponding to the handwriting input 1101-1, “John Doe 66-333-44-55” may be obtained.

In operation 1104 , the electronic device 1000 may extract the name of the lower level from the text based on the pre-stored template for the name of the lower level. The electronic device 1000 according to an exemplary embodiment may include "words", which are names for "John" and "Doe", and "66", "333", "44", and "55", which are names for "Doe" in the text. You can extract "symbol", which is a name for "operator" and "-".

In step 1105 , the electronic device 1000 may extract the name of the higher level from the name of the lower level based on a template for the name of the higher level stored in advance. The electronic device 1000 according to an embodiment regards “(word)(word),” which is an expression of lower-level names extracted in step 1104, as being consistent with the expression method of “name”, and As the name, "name" can be extracted. Further, with respect to "(number) (sign -) (number) (sign -) (number) (sign -) (number)", "telephone number" can be extracted as a higher-level name.

In operation 1106 , the electronic device 1000 may determine the user intention to “add contact” based on the higher-level name, and may perform an operation of adding contact information as an operation according to the user intention. The operation of "adding a contact" according to an embodiment may be performed without user authentication because sensitive personal information of the user is not used.

The operation for “adding a contact” according to an embodiment may be performed by executing a contact management application for adding a contact in the electronic device 1000 . For example, in a contact management application, as the name (ex. John Doe) and phone number (ex. 66-333-44-55) are added as contact information, the "add contact" operation is performed as shown in 1107. Results may be displayed.

As an operation for adding a contact is performed without user authentication being performed, the electronic device 1000 according to an embodiment maintains the locked state and may display a result of the operation for adding a contact as 1107 .

On the other hand, when the operation according to the user's intention according to an embodiment is "contact inquiry", since the user's contact information corresponds to the user's sensitive personal information, user authentication may be additionally performed. Accordingly, after performing user authentication, the electronic device 1000 according to an embodiment may perform a “contact inquiry” operation through the contact management application.

12 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

Referring to FIG. 12 , in 1201 , the electronic device 1000 may receive a handwriting input 1201-1 on the lock screen.

In 1202 and 1203 , the electronic device 1000 may analyze the handwriting input 1201-1 to obtain at least one text corresponding to the handwriting input 1201-1. According to an embodiment, "14:00", which is a text corresponding to the handwriting input 1201-1, may be obtained.

In step 1204 , the electronic device 1000 may extract the name of the lower level from the text, based on the pre-stored template for the name of the lower level. The electronic device 1000 according to an embodiment may extract “number”, which is a name for “14” and “00”, and a “symbol”, which is a name for “:” from the text.

In step 1205 , the electronic device 1000 may extract the name of the higher level from the name of the lower level based on the pre-stored template for the name of the higher level. In the electronic device 1000 according to an embodiment, with respect to “(number) (symbol :) (number)”, which is an expression of lower-level names extracted in 1204 , the method in which “time of day” is expressed , and as a higher-level name, “time of day” can be extracted.

In operation 1206 , the electronic device 1000 may determine the user intention as “adding an alarm” based on the name of the higher level, and may perform an operation of adding alarm information as an operation according to the user intention. Since the operation of "adding an alarm" according to an embodiment does not use sensitive personal information of a user, it may be performed without user authentication.

The operation for “adding an alarm” according to an embodiment may be performed by executing an alarm application for adding an alarm in the electronic device 1000 . For example, in an alarm application, as alarm time information (eg, 14:00) is added as alarm information, a result of performing an operation for adding an alarm as shown in 1207 may be displayed.

As an operation for adding an alarm is performed without user authentication being performed, the electronic device 1000 according to an embodiment maintains the locked state and may display a result of the operation of adding an alarm as 1207 .

On the other hand, when the operation according to the user's intention according to an embodiment is "alarm information inquiry", since the user's alarm information corresponds to the user's sensitive personal information, user authentication may be additionally performed. Accordingly, after performing user authentication, the electronic device 1000 according to an embodiment may perform an "alarm information inquiry" operation through the alarm application.

13 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

Referring to FIG. 13 , in 1301 , the electronic device 1000 may receive a handwriting input 1301-1 on the lock screen.

In steps 1302 and 1303 , the electronic device 1000 may analyze the handwriting input 1301-1 to obtain at least one text corresponding to the handwriting input 1301-1. According to an embodiment, texts corresponding to the handwriting input 1301-1, “Meeting”, “7.2.2019”, and “15:00” may be obtained.

In operation 1304 , the electronic device 1000 may extract the name of the lower level from the text based on the pre-stored template for the name of the lower level. The electronic device 1000 according to an embodiment may include "words", "7", "2", "2019", "15", and "00", which are names for "Meeting" in the text. "Number" and ".", "Symbol", which is a name for ":", can be extracted.

In step 1305 , the electronic device 1000 may extract the name of the higher level from the name of the lower level, based on a template for the name of the higher level stored in advance. The electronic device 1000 according to an embodiment may include “(word)” and “(numbers from 1 to 12) (symbol .) (numbers from 1 to 31) that are expressions of lower-level names extracted in 1304 . (Symbol .)(4-digit number)" and "(Number)(Symbol :)(Number)" are considered to be consistent with the way "Title", "Date", and "Time" are expressed, respectively, and As the name of the level, "Title", "Date", and "Time" can be extracted.

In operation 1306 , the electronic device 1000 may determine the user intention as “add schedule” based on the name of the higher level, and perform an operation of adding schedule information as an operation according to the user intention. The "add schedule" operation according to an embodiment may be performed without user authentication because sensitive personal information of the user is not used.

The operation for “adding a schedule” according to an embodiment may be performed by executing a schedule management application for adding a schedule in the electronic device 1000 . For example, in a schedule management application, as the title (ex. Meeting), date and time information (ex. July 2, 2019, 14:00) of the schedule information is added as schedule information, 1307 and Similarly, a result of adding schedule information may be displayed.

The electronic device 1000 according to an embodiment maintains the locked state as the operation for adding schedule information is performed without user authentication being performed, and displays the result of the operation of adding schedule information as 1307 . can be displayed

On the other hand, when the operation according to the user's intention according to an embodiment is "schedule information inquiry", since the user's schedule information corresponds to the user's sensitive personal information, user authentication may be additionally performed. Accordingly, after performing user authentication, the electronic device 1000 according to an embodiment may perform a “schedule information inquiry” operation through the schedule management application.

14 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

Referring to FIG. 14 , in 1401 , the electronic device 1000 may receive a handwriting input 1401-1 on the lock screen.

In steps 1402 and 1403 , the electronic device 1000 may analyze the handwriting input 1401-1 to obtain at least one text corresponding to the handwriting input 1401-1. According to an embodiment, "02:00:00", which is a text corresponding to the handwriting input 1401-1, may be obtained.

In step 1404 , the electronic device 1000 may extract the name of the lower level from the text, based on the pre-stored template for the name of the lower level. The electronic device 1000 according to an embodiment may extract "number", which is a name for "02", "00", and "00", and a "symbol", which is a name for ":", from among texts.

In step 1405 , the electronic device 1000 may extract the name of the higher level from the name of the lower level based on the pre-stored template for the name of the higher level. In the electronic device 1000 according to an embodiment, “time” is expressed with respect to “(number) (sign:: (number) (sign::) (number))”, which is an expression of lower-level names extracted in 1404 . As a higher-level name, "time" can be extracted.

In step 1406 , the electronic device 1000 may determine the user intention as “timer execution” based on the name of the higher level, and execute the timer as an operation according to the user intention. Since the "timer execution" operation according to an embodiment does not use sensitive personal information of a user, it may be performed without user authentication.

The operation for “timer execution” according to an embodiment may be performed by executing an application for timer execution in the electronic device 1000 . For example, in the timer application, as timer measurement time information (eg, 2 hours 0 minutes 0 seconds) is automatically set, a result of performing an operation for timer execution as shown in 1407 may be displayed.

The electronic device 1000 according to an embodiment maintains the locked state as the operation for adding schedule information is performed without user authentication being performed, and displays the result of the operation for adding schedule information as shown in 1407. can be displayed

15 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

Referring to FIG. 15 , in 1501 , the electronic device 1000 may receive a handwriting input 1501-1 on the lock screen.

In steps 1502 and 1503 , the electronic device 1000 may analyze the handwriting input 1501-1 to obtain at least one text corresponding to the handwriting input 1501-1. According to an embodiment, a text corresponding to the handwriting input 1501-1, “+1 water” may be obtained.

In 1504 , the electronic device 1000 may extract the name of the lower level from the text based on the pre-stored template for the name of the lower level. The electronic device 1000 according to an embodiment may extract a name for “+”, “operator”, “number” for “1”, and “word” as a name for “water” from among texts. can

In step 1505 , the electronic device 1000 may extract the name of the higher level from the name of the lower level based on a template for the name of the higher level stored in advance. The electronic device 1000 according to an embodiment, for “(operator +) (constant)” and “(word representing food/drink)”, which are expressions of lower-level names extracted in 1504 , respectively, “amount” ", and "food/drink" are considered to be consistent with the way they are expressed, and "quantity" and "food/drink" can be extracted as higher-level names.

In 1506 , the electronic device 1000 may determine the user intention as “updating health information” based on the name of the higher level, and may perform an operation of adding certain information as an operation according to the user intention. The operation of "updating health information" according to an embodiment may be performed without user authentication because sensitive personal information of the user is not used.

The operation for “updating health information” according to an embodiment may be performed by executing a health management application for updating health information in the electronic device 1000 . For example, in a health management application, as health information (eg, adding a glass of water) is updated as health information, a result of adding schedule information may be displayed as shown in 1507 .

The electronic device 1000 according to an embodiment maintains the locked state as the operation for updating health information is performed without user authentication being performed, and displays the result of the operation of updating the health information as shown in 1507 . can be displayed

On the other hand, when the operation according to the user's intention according to an embodiment is "health information inquiry", the user's schedule information corresponds to the user's sensitive personal information, so user authentication may be additionally performed. Accordingly, after performing user authentication, the electronic device 1000 according to an embodiment may perform a “health information inquiry” operation through the health management application.

16 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

Referring to FIG. 16 , in 1601 , the electronic device 1000 may receive a handwriting input 1601-1 on a lock screen.

In steps 1602 and 1603 , the electronic device 1000 may analyze the handwriting input 1601-1 to obtain at least one text corresponding to the handwriting input 1601-1. According to an embodiment, as a result of analyzing the handwriting input 1601-1, strokes of the handwriting input 1601-1 may be classified into pictures and text. In addition, based on the result of the classification of strokes, "Basketball", which is text corresponding to strokes for a picture, and "22.02" and "19:30", which are texts corresponding to strokes for text, may be obtained. .

In step 1604 , the electronic device 1000 may extract the name of the lower level from the text based on the pre-stored template for the name of the lower level. The electronic device 1000 according to an embodiment may include “words”, “22”, “02”, “19”, and “30”, which are names for “Basketball,” among texts, “numbers”, and "Symbol", which is a name for "." and ":", can be extracted.

In step 1605 , the electronic device 1000 may extract the name of the higher level from the name of the lower level, based on the pre-stored template for the name of the higher level. The electronic device 1000 according to an embodiment may include “(word)” and “(numbers from 1 to 31) (symbol .) (numbers from 1 to 12) that are expressions of lower-level names extracted in 1604 . For ", and "(numbers from 1 to 23) (symbol :) (numbers from 1 to 59)" , "Title", "Date" and "Time" can be extracted as higher-level names.

In step 1606 , the electronic device 1000 may determine the user intention as “add schedule” based on the name of the higher level, and perform an operation of adding schedule information as an operation according to the user intention. The "add schedule" operation according to an embodiment may be performed without user authentication because sensitive personal information of the user is not used.

The operation for “adding a schedule” according to an embodiment may be performed by executing a schedule management application for adding a schedule in the electronic device 1000 . For example, in a schedule management application, as the title (ex. Basketball), date and time information (ex. February 22, 19:30) of schedule information is added as schedule information, schedule information such as 1607 A result of the operation of adding information may be displayed.

The electronic device 1000 according to an embodiment maintains the locked state as the operation for adding schedule information is performed without user authentication being performed, and displays the result of the operation of adding schedule information as 1607 . can be displayed

On the other hand, when the operation according to the user's intention according to an embodiment is "schedule information inquiry", since the user's schedule information corresponds to the user's sensitive personal information, user authentication may be additionally performed. Accordingly, after performing user authentication, the electronic device 1000 according to an embodiment may perform a “schedule information inquiry” operation through the schedule management application.

17 is a diagram illustrating an example of performing an operation of transmitting a message based on a handwriting input received on a lock screen according to an embodiment.

Referring to FIG. 17 , the electronic device 1000 may receive a handwriting input 1703 on a screen displayed in a locked state of 1701 .

The electronic device 1000 according to an embodiment may obtain at least one text corresponding to the handwriting input 1703 and determine the user's intention based on the at least one text.

The user's intention according to an embodiment is not limited to at least one text, and may be further determined based on the user's context information. According to an embodiment, the user's context information may include various information about the user that may be obtained from the electronic device 1000 . For example, when the handwriting input 1703 is received, information displayed on the electronic device 1000, which is information currently provided to the user, may be used as the user's context information to determine the user's intention. have.

Referring to 1701 , the electronic device 1000 may receive a handwriting input 1703 while the message 1702 is displayed on the electronic device 1000 . The electronic device 1000 according to an embodiment may determine the user's intention as a response message to the message 1702 based on the message 1702 .

The operation of transmitting the response message according to an embodiment may be performed without user authentication because only the message 1702 that can be displayed in the locked state is used without using sensitive personal information of the user.

In operation 1704 , the electronic device 1000 may transmit the text 1705 obtained from the handwriting input 1703 as a response message to the message 1702 based on the user's intention. The electronic device 1000 according to an embodiment may maintain the locked state as the operation of transmitting the response message is performed without user authentication being performed, and may display the result of transmitting the response message as 1704 .

18 is a diagram illustrating an example of performing an operation of executing an application based on a handwriting input received on a lock screen according to an embodiment.

Referring to FIG. 18 , the electronic device 1000 may receive a handwriting input 1802 on a screen displayed in a locked state of 1801 .

The electronic device 1000 according to an embodiment may obtain at least one text corresponding to the handwriting input 1802 and determine the user's intention based on the at least one text.

The electronic device 1000 according to an embodiment may determine the user's intention by determining at least one name corresponding to the handwriting input 1802 . According to an embodiment, lower-level names such as “number” and “symbol” may be obtained from “32.2799” and “127.0126”, which are texts corresponding to the handwriting input 1802 . In addition, the upper-level names, “latitude” and “longitude,” may be obtained based on the analysis of the method in which the lower-level names of “numbers” and “symbols” are expressed.

"Latitude" according to an embodiment may be expressed as a number up to 5 decimal places between -90 and +90. In addition, "hardness" may be expressed as a number from 0 to 180 to 5 decimal places. According to an embodiment, the text of the handwriting input 1802 expressed as a combination of “numbers” and “symbols” is based on a template including information on how the above-described “latitude” and “longitude” are expressed. , "latitude" and "longitude" may be obtained by determining whether or not they correspond to the expressed method, thereby obtaining a higher-level name.

The electronic device 1000 according to an embodiment may determine the user intention as “location search” based on the higher-level name. The user's intention according to an embodiment may be determined based on various context information about the user, such as whether a "location search" is possible or a map application is available in the electronic device 1000 .

Accordingly, by executing the map application according to the user's intention, the electronic device 1000 may display a search result of a location corresponding to a latitude of "32.2799" and a longitude of "127.0126" as shown in 1803.

The operation of searching for a location through the map application according to an embodiment may be performed without user authentication because sensitive personal information of the user is not used.

19 is a diagram illustrating an example of performing an operation by displaying at least one icon based on a handwriting input received on a lock screen according to an embodiment.

Referring to FIG. 19 , the electronic device 1000 may receive a handwriting input 1902 on a screen displayed in a locked state of 1901 .

The electronic device 1000 according to an embodiment may obtain at least one text corresponding to the handwriting input 1902 and determine the user's intention based on the at least one text.

The electronic device 1000 according to an embodiment may determine the user's intention by determining at least one name corresponding to the handwriting input 1902 . According to an embodiment, lower-level names such as “number” and “symbol” may be obtained from “010-2334-1234”, which is text corresponding to the handwriting input 1902 . In addition, based on the analysis of the method in which the lower-level names of “numbers” and “symbols” are expressed, the higher-level names of “phone number” may be obtained.

"Phone number" according to an embodiment may be expressed as a combination of a number and a symbol "-". According to an embodiment, the text of the handwriting input 1902 expressed as a combination of "number" and the symbol "-" is " By determining whether or not the method corresponds to the expression "phone number", a higher-level name may be obtained.

The electronic device 1000 according to an embodiment may determine the user intention to “make a call” and “add a contact” based on the higher-level name. When a plurality of user intentions are determined, the electronic device 1000 according to an embodiment may display a plurality of icons corresponding to the operation so that the operation can be performed according to the user's selection.

According to an embodiment, as shown in the example shown in 1903 , the electronic device 1000 has two icons 1904 corresponding to “an operation of making a call” and “an operation of adding contact information”, respectively, based on a user intention. ) can be displayed.

The electronic device 1000 according to an embodiment may perform an operation corresponding to the icon selected according to a user input from among the two icons 1904 . As shown in the example shown in 1903 , when the user selects an icon corresponding to the “action of making a call,” the electronic device 1000 displays a phone number corresponding to the handwriting input 1902 as shown in the example shown in 1905 . (1906) may display an interface for conducting a call.

According to an embodiment, the operation of making a call with the handwritten-input phone number may be performed after user authentication as it is determined that the operation corresponds to a security-required operation. According to an embodiment, a call may be performed after user authentication is performed so that an unauthorized person cannot make a call by impersonating the user.

The electronic device 1000 according to an embodiment is not limited to the above-described example, and according to a user setting, a call operation may be performed without user authentication.

20 is a diagram illustrating an example of allocating a tag to a received handwriting input on a lock screen according to an embodiment.

Referring to FIG. 20 , the electronic device 1000 may receive a plurality of handwriting inputs 2002 and 2003 as in the example shown in 2001 .

The electronic device 1000 according to an embodiment may obtain at least one text corresponding to the handwriting inputs 2002 and 2003 and determine a name corresponding to the at least one text.

According to an embodiment, lower-level names such as “number” and “symbol” may be obtained from “javer.com” and “jdd.com”, which are texts corresponding to the handwriting inputs 2002 and 2003 . In addition, based on the analysis of the method in which the lower-level names of “numbers” and “symbols” are expressed, the higher-level names of “web address” may be obtained.

A “web address” according to an embodiment may be expressed as (number or alphabet).com, (number or alphabet).net, and the like. According to an embodiment, based on a template including information on a method of expressing the above-described "web address", whether the text of the handwriting input 2002 or 2003 corresponds to a method of expressing the "web address" By determining , a higher-level name may be obtained.

The electronic device 1000 according to an embodiment may determine the user's intention to "access a web page" based on a higher-level name. As in the example shown in 2004, the electronic device 1000 according to an embodiment provides a tag corresponding to "access to a web page" in the handwriting inputs 2002 and 2003 based on the user's intention of "accessing a web page". Each "Link" can be assigned and displayed.

According to an embodiment, when a tag for a web page address to be accessed is selected by the user, the electronic device 1000 may perform an operation of accessing a web page corresponding to the selected tag. The electronic device 1000 according to an embodiment may display the result of accessing the web page by executing a browser application for accessing the web page, as in the example shown in 2007 .

The operation of accessing the web page through the browser application according to an embodiment may be performed without user authentication because sensitive personal information of the user is not used.

21 is a diagram illustrating an example of a handwriting input to which a tag is assigned according to an embodiment.

According to an embodiment, the handwriting inputs 2005 and 2006 to which tags are assigned as shown in FIG. 20 may be stored as memos. Referring to 2101 of FIG. 21 , the electronic device 1000 may display memos including at least one handwriting input through a memo application.

The electronic device 1000 according to an embodiment includes a handwriting input to which a tag corresponding to the user's input is assigned, based on the user's input 2102-1 for a search, as shown in the example shown in 2102 . Notes can be displayed. According to an embodiment, a memo including a tag corresponding to the user's input 2102-1 may be displayed. For example, when "mail" is input as the user's input 2102-1 for a search, a memo including a handwriting input to which a tag including "mail" is assigned may be displayed.

Referring to 2103 , the electronic device 1000 may display a memo selected according to the user's input among memos including tags corresponding to the user's input 2102-1.

22 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

Referring to FIG. 22 , in 2201 , the electronic device 1000 may receive a handwriting input 2201-1 on a lock screen. The electronic device 1000 according to an embodiment may obtain at least one text corresponding to the handwriting input 2201-1 and perform an operation based on the at least one text.

The electronic device 1000 according to an embodiment may determine that at least one name corresponding to the handwriting input 2201-1 is a “phone number”. As in the example shown in 2202 , as a result of the operation corresponding to the handwriting input 2201-1, an interface screen for performing a call may be displayed.

However, according to an embodiment, while the result of the operation corresponding to the handwriting input 2201-1 is displayed on the electronic device 1000 as shown in 2202, the handwriting input may be additionally received. In the electronic device 1000 according to an embodiment, as the handwriting input 2203-1 corresponding to “=" is further received, as in the example shown in 2203 , at least one corresponding to the writing input 2201-1 is received. One name can be determined again as "Equation".

Based on the re-determined name, the electronic device 1000 according to an embodiment performs an equation calculation as a result of an operation corresponding to the handwriting inputs 2201-1 and 2203-1, as shown in the example shown in 2204 . A screen indicating the result 2204 - 1 may be displayed.

The operation of performing a call according to an embodiment may be performed after user authentication as it is determined that the operation corresponds to an operation requiring security. For example, in 2202 , when a button for conducting a call is selected while a call interface is displayed, user authentication may be performed first, and then a call operation may be performed. The electronic device 1000 according to an embodiment is not limited to the above-described example, and according to a user setting, a call operation may be performed without user authentication.

In addition, the operation of calculating the equation according to an embodiment may be performed without user authentication because sensitive personal information of the user is not used.

23 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

Referring to 2310 of FIG. 23 , a user's handwriting input may be received through the screen of the electronic device 1000 . The electronic device 1000 according to an embodiment is not limited to a smart phone as shown in the illustrated example, and may include various types of home appliances.

Referring to 2320 , the electronic device 1000 may perform an operation by recognizing the text of the received handwriting input. According to an embodiment, as texts corresponding to the handwriting input, “Butter”, “Steak”, “Potato”, “Cheese”, and “Eggs” may be obtained. Also, based on the text, the name of “food ingredient” may be obtained.

The electronic device 1000 according to an embodiment may determine the user's intention based on the name of "food ingredient" and the user's context information. According to an embodiment, as the user's context information, the electronic device 1000 may determine the user's intention to "create a shopping list" based on the fact that the electronic device 1000 is a refrigerator capable of storing food ingredients. .

In operation 2320, the electronic device 1000 executes an application for creating a shopping list based on the user's intention or provides an interface to create a shopping list based on text corresponding to the handwriting input. You can display the results of your actions.

The operation of creating a shopping list according to an embodiment may be performed without user authentication because sensitive personal information of the user is not used.

24 is a diagram illustrating an example of performing an operation based on a handwriting input according to an embodiment.

Referring to 2410 of FIG. 24 , a user's handwriting input may be received through the screen of the electronic device 1000 .

Referring to 2420 , the electronic device 1000 may perform an operation by recognizing the text of the received handwriting input. According to an embodiment, as texts corresponding to the handwriting input, “Soup” and “15min” may be obtained. Also, based on the text, the name of “time” may be obtained.

The electronic device 1000 according to an embodiment may determine the user's intention based on the name of "time" and the user's context information. According to an embodiment, as the user's context information, based on the fact that the electronic device 1000 is a refrigerator located in a kitchen capable of cooking, the electronic device 1000 sets the user's intention as "a timer for measuring cooking time". settings" can be determined.

In 2420, the electronic device 1000 may display the result of performing the operation of setting the timer based on the text corresponding to the handwriting input by executing the application for setting the timer based on the user's intention. have. According to an embodiment, an operation of setting a timer for 15 minutes for cooking soup may be performed based on the handwriting input.

The operation of setting the timer according to an embodiment may be performed without user authentication because sensitive personal information of the user is not used.

According to an embodiment, it is possible to quickly perform an operation only by inputting a handwriting without running a separate application for performing the operation.

The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' is a tangible device and only means that it does not contain a signal (eg, electromagnetic wave). It does not distinguish the case where it is stored as For example, the 'non-transitory storage medium' may include a buffer in which data is temporarily stored.

According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a device-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play Store™) or on two user devices (eg, It can be distributed (eg downloaded or uploaded) directly, online between smartphones (eg: smartphones). In the case of online distribution, at least a portion of the computer program product (eg, a downloadable app) is stored at least in a machine-readable storage medium, such as a memory of a manufacturer's server, a server of an application store, or a relay server. It may be temporarily stored or temporarily created.

Also, in this specification, “unit” may be a hardware component such as a processor or circuit, and/or a software component executed by a hardware component such as a processor.

The description of the present invention described above is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

Claims (15)

A method of performing an operation based on a handwriting input in an electronic device, the method comprising:
receiving a user's handwriting input on the lock screen of the electronic device;
obtaining input information corresponding to the handwriting input;
determining whether to perform user authentication based on the input information;
performing user authentication for the handwriting input based on the determination; and
and performing an operation corresponding to the input information based on a result of performing the user authentication. The method of claim 1, wherein determining whether to perform the user authentication comprises:
determining at least one entity corresponding to at least one text included in the input information;
determining a user's intention based on the at least one name; and
and determining whether to perform the user authentication based on the determined intention of the user. The method of claim 2, wherein the user's intention is,
when the handwriting input is received, it is determined based on information displayed on the lock screen. 3. The method of claim 2,
At least one icon selectable by the user is displayed on the lock screen based on the user's intention, and an operation corresponding to the icon selected by the user from among the at least one icon is performed. The method of claim 4 , wherein it is determined whether the user authentication is required for the operation corresponding to the selected icon, and the operation is performed based on a result of the user authentication according to the determination result. . The method according to claim 2, wherein a tag representing the determined name is assigned to the at least one text. The method of claim 2 , wherein it is determined whether to perform the user authentication based on whether the determined operation corresponding to the user's intention is an operation requiring the user authentication. An electronic device that performs an operation based on a handwriting input, the electronic device comprising:
a user input unit configured to receive a user's handwriting input on the lock screen of the electronic device;
Obtaining input information corresponding to the handwriting input, determining whether to perform user authentication based on the input information, performing user authentication on the handwriting input based on the determination, and performing the user authentication at least one processor configured to perform an operation corresponding to the input information based on the performed result; and
and an output unit outputting a result of the operation. 9. The method of claim 8, wherein the at least one processor comprises:
determining at least one name respectively corresponding to at least one text included in the input information,
Based on the at least one name, determine the user's intention,
and determining whether to perform the user authentication based on the determined intention of the user. The method of claim 9, wherein the user's intention is
when the handwriting input is received, the electronic device is determined based on information displayed on the lock screen. The operation of claim 9 , wherein at least one icon selectable by the user is displayed on the lock screen based on the user's intention, and corresponding to the icon selected by the user from among the at least one icon This is done, the electronic device. The electronic device of claim 11 , wherein it is determined whether the user authentication for the operation corresponding to the selected icon is required, and according to the determination result, the operation is performed based on a result of the user authentication. Device. The electronic device of claim 9 , wherein a tag indicating the determined name is assigned to the at least one text. The electronic device of claim 9 , wherein it is determined whether to perform the user authentication based on whether the determined operation corresponding to the user's intention is an operation requiring the user authentication. A computer-readable recording medium in which a program for implementing the method of any one of claims 1 to 7 is recorded.

Images