KR102407577B1 - User device and method for processing input message - Google Patents

Abstract

The user terminal processing the input word transmits the input word to a main screen and a display unit for displaying a plurality of apps preset to be exposed on the main screen, an input unit for receiving input words from the user, and the plurality of apps, An input word transmitting unit receiving a return value for the input word from at least one app capable of processing the input word among the plurality of apps and changing the at least one app to a standby state based on the return value state change unit; and an app execution unit configured to execute an app corresponding to the additional input word when an additional input word related to any one of the apps in the standby state is received through the input unit.

Description

User terminal and method for processing input word {USER DEVICE AND METHOD FOR PROCESSING INPUT MESSAGE}

The present invention relates to a user terminal and method for processing an input word.

An intelligent personal assistant is a software agent that processes tasks requested by users and provides specialized services to users. The intelligent personal assistant collects and provides customized information to the user based on artificial intelligence (AI) engine and voice recognition, and performs various functions such as scheduling, sending e-mail, and restaurant reservation according to the user's voice command. It has the advantage of improving convenience.

These intelligent personal assistants are mainly provided as customized personal services on smartphones, and typically include Apple's Siri, Google's Now, and Samsung's Bixby. In this regard, Korean Patent Laid-Open No. 2016-0071111, which is a prior art, discloses a method of providing a personal assistant service in an electronic device.

Recently, intelligent personal assistants have been able to identify users' intentions and provide appropriate services. However, the intelligent personal assistant has a disadvantage in that it cannot recognize and reflect the user's intention in real time. To this end, the intelligent personal assistant must have information about every app in order to clearly understand the user's intentions. However, it has disadvantages in that the extension of ^3rd -party apps is limited, and there are few opportunities to run ^3rd -party apps without the command of an intelligent personal assistant.

An object of the present invention is to provide a user terminal and method in which all apps respond in real time to the user's continuous voice input, and the user has the option of running the app in response to each app's response. An object of the present invention is to provide a user terminal and method for providing an app execution environment that enables and executes a plurality of apps at the same time. An object of the present invention is to provide a user terminal and method for transmitting the user's commands and keywords to all apps and highlighting the apps that respond to the user's commands and keywords. However, the technical problems to be achieved by the present embodiment are not limited to the technical problems described above, and other technical problems may exist.

As a means for achieving the above-described technical problem, an embodiment of the present invention transmits the received input word to a display unit for displaying a plurality of apps, an input unit for receiving input words from a user, and the plurality of apps, An input word transmitting unit receiving a return value for the input word from at least one app capable of processing the input word among the plurality of apps and changing the at least one app to a standby state based on the return value state change unit; and an app execution unit configured to execute an app corresponding to the additional input word when receiving an additional input word related to any one of the apps in the standby state through the input unit.

Another embodiment of the present invention may include displaying a plurality of apps, receiving input words from a user, transmitting the input words to the plurality of apps, and processing the input words among the plurality of apps receiving a return value for the input word from at least one app that can do it; changing the at least one app to a standby state based on the return value; When receiving the input word, it is possible to provide an input word processing method comprising the step of running an app corresponding to the additional input word.

The above-described problem solving means are merely exemplary, and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and detailed description.

According to any one of the above-described problem solving means of the present invention, a user terminal and method in which all apps respond in real time to the user's continuous voice input, and the user has the option to execute the app in response to the response of each app can provide It is possible to provide a user terminal and a method for providing an app execution environment for simultaneously activating and executing a plurality of apps. It is possible to provide a user terminal and method for delivering a user's commands and keywords to all apps, and highlighting and displaying apps that respond to the user's commands and keywords.

1 is a block diagram of an input word processing system according to an embodiment of the present invention.
2 is a block diagram of a user terminal according to an embodiment of the present invention.
3 is an exemplary diagram illustrating a main screen displayed on a user terminal according to an embodiment of the present invention.
4A and 4B are exemplary views for explaining a process of transferring an input word received from a user terminal to a plurality of apps according to an embodiment of the present invention.
5A to 5D are exemplary views for explaining a process of processing an input word received from a user terminal through a first app according to an embodiment of the present invention.
6A and 6B are exemplary views illustrating a process of processing an input word received from a user terminal through a second app according to an embodiment of the present invention.
7A and 7B are exemplary views illustrating a process of processing an input word received from a user terminal through a third app according to an embodiment of the present invention.
8 is an exemplary view for explaining a process of transmitting an input word input from a user terminal to a plurality of apps and receiving a return value for the input word according to an embodiment of the present invention.
9A and 9B are exemplary views for explaining a process of providing a service through an app activated according to an input word input from a user terminal according to an embodiment of the present invention.
10 is a flowchart of a method of processing an input word according to whether a reserved word is input in a user terminal according to an embodiment of the present invention.
11 is a flowchart of a method of processing an input word in a user terminal according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated, and one or more other features However, it is to be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded in advance.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware.

Some of the operations or functions described as being performed by the terminal or device in this specification may be instead performed by a server connected to the terminal or device. Similarly, some of the operations or functions described as being performed by the server may also be performed in a terminal or device connected to the server.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an input word processing system according to an embodiment of the present invention. Referring to FIG. 1 , the input word processing system 1 may include a user terminal 110 , a display device 120 , and a plurality of service providing servers 130 . The user terminal 110 , the display device 120 , and the plurality of service providing servers 130 exemplarily illustrate components that can be controlled by the input word processing system 1 .

Each component of the input word processing system 1 of FIG. 1 is generally connected through a network. For example, as shown in FIG. 1 , the user terminal 110 may be connected to a plurality of service providing servers 130 at the same time or at a time interval.

A network refers to a connection structure in which information can be exchanged between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet (WWW: World). Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television networks, and the like. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

The user terminal 110 may display the main screen and a plurality of preset apps to be exposed on the main screen.

The user terminal 110 may receive an input word from a user and transmit the input word input to a plurality of apps. The input word may be composed of a natural language, and may include text, voice, motion, etc. that may be input from a user. Specifically, the user terminal 110 may assign an index to the input word input to classify the word unit by word, and transmit the input word divided by word unit to the plurality of apps in the order in which the input word is received. For example, when any one of the plurality of apps is in the running state, the user terminal 110 may transmit the input word to the running app. In addition, when the system is activated by an activation command, an input word may be transmitted to a plurality of apps.

The user terminal 110 may determine whether the input word corresponds to a reserved word to be systematically prioritized over processing in the app, and if the input word corresponds to the reserved word, it may process the reserved word. When the input word corresponds to a reserved word, the user terminal 110 does not transmit the input word to the plurality of apps. At this time, when the user terminal 110 ignores the reserved word, the user terminal 110 may transmit the input word to the plurality of apps without determining whether the input word corresponds to the reserved word. Alternatively, when any one of the plurality of apps is in the running state, the user terminal 110 may transmit the input word to the running app.

When the reserved word includes the app call word and the input word input from the user 115 corresponds to the app call word, the user terminal 110 changes the currently running app to the standby state, and the app that is the target of the app call word. can be changed to the running state.

The user terminal 110 may receive a return value for the input word from at least one app capable of processing the input word among a plurality of apps. The return value may include, for example, a result value that can be obtained when each app processes an input word input, and information about automatic execution. For example, when the return value includes information on auto-run, and the input word includes a keyword capable of auto-run for any one of a plurality of apps after a preset time, the user terminal 110 corresponds to the auto-run You can receive information about automatic execution from the app.

The user terminal 110 may change at least one app to a standby state based on the return value.

For example, when the return value includes a result value obtained when each app processes an input word input, the user terminal 110 may display the result value to be overlaid on at least one app. As another example, when the return value includes information on auto-run, the user terminal 110 may display the information on auto-run to be overlaid on an app corresponding to auto-run. At this time, when an additional input word is not input within a preset time, an app corresponding to automatic execution may be executed.

The user terminal 110 may receive an additional input word related to any one of the apps in the standby state, and may execute an app corresponding to the additional input word. In this case, the input word may be processed by the app corresponding to the additional input word.

The user terminal 110 may receive an additional input word related to any one of the apps in the standby state, change the app corresponding to the additional input word from the standby state to the running state, and execute the running app from the main screen You can switch to the screen. For example, the user terminal 110 may perform rendering processing while being switched from the main screen to the running screen of the app in the running state. In this case, the result screen of the input word may be processed by the app in the running state, and after switching to the app in the running state, rendering processing on the result screen of the input word may be performed.

When the user terminal 110 receives an input word for screen initialization from the user 115 , the user terminal 110 may transmit the input word to the running app. In this case, the running app may perform screen initialization based on the input word.

The user terminal 110 may be a terminal providing an artificial intelligence (AI) assistant function having a microphone (MIC) and a speaker (Speaker).

The display device 120 may output a main screen in conjunction with the user terminal 110 . Also, the display device 120 may output a plurality of preset apps together through the main screen.

The plurality of service providing servers 130 are external servers that provide services for each app, and may derive a return value for an input word corresponding to each app and deliver it to the user terminal 110 . For example, when the return value for the input word can be derived by each app without interworking with the plurality of service providing servers 130 , the user terminal 110 may each Based on the return value received from the app, the corresponding app can be activated. For another example, when the user terminal 110 requires interworking with a plurality of service providing servers 130 because the return value for the input word cannot be derived by each app, the return value from each app instead of the return value A component (eg, request URL, content type, callback function, etc.) may be received, and a return value may be derived by interworking with the plurality of service providing servers 130 based on the component for the received return value. .

When receiving an input word corresponding to the user's voice signal from the user terminal 110, the voice recognition server 140 interprets/infers/derives a command to execute a specific service or process from the voice signal, and then 110) can be transmitted. The user terminal 110 may process the input word in association with a plurality of apps based on the received command. The function of the voice recognition server 140 may be implemented as a module that can be installed in the user terminal 110 .

2 is a block diagram of a user terminal according to an embodiment of the present invention. Referring to FIG. 2 , the user terminal 110 includes a display unit 210 , an input unit 220 , a reserved word determination unit 230 , a reserved word execution unit 240 , an input word transmission unit 250 , and a state change unit 260 . , a conversion unit 270 and an execution unit 280 may be included.

The display unit 210 may display the main screen and a plurality of apps preset to be exposed on the main screen. It will be described in detail with reference to FIG. 3 in relation to the main screen.

3 is an exemplary diagram illustrating a main screen displayed on a user terminal according to an embodiment of the present invention. Referring to FIG. 3 , the display unit 210 displays the main screen and is exposed to the main screen, for example, a search app 310 , a music app 320 , a TV app 330 , a shopping app 340 , A stock app 350, a delivery app, a map app, a phone app, a coupon app, a memo app, etc. may be displayed.

Returning to FIG. 2 again, the input unit 220 may receive an input word from the user 115 . For example, the input unit 220 may receive an input word such as "today's KT stock price" through a microphone (MIC) from the user.

The input unit 220 may receive an input word for screen initialization from the user 115 . For example, the input unit 220 may receive an input word for screen initialization such as “back”, “end”, “first”, “home” from the user 115 .

The input unit 220 may receive an additional input word related to any one of the waiting apps from the user 115 . In this case, the input word may be processed by the app corresponding to the additional input word. For example, the input unit 220 may receive additional input words such as “play”, “play”, and “play” from the user 115 while the TV app is activated.

The reserved word determination unit 230 may determine whether the input word corresponds to a reserved word to be systematically processed prior to processing in the app. Reserved words may be divided into system reserved words and app reserved words. The system reserved words may include reserved words for activating the system, such as "My friend" and "Giga Genie", and "back", "exit", "home", etc., which are commands for jointly controlling a plurality of apps. The app reserved word may include an app call word, which is a command for calling an actual app, and an app command, which is a command independently executed by each app. For example, the app command may include a command such as “play” or “play” which is a command independently performed in the TV app.

When the input word corresponds to the reserved word, the reserved word execution unit 240 may process the reserved word. When the input word corresponds to the app call word, the reserved word execution unit 240 may change the currently running app to the standby state and change the app, the target of the app call word, to the running state. For example, when the TV app is currently running, the input word "Genie" is input from the user in the input unit 220, and the reserved word determination unit 230 determines that "Genie" is the app call word, the reserved word execution unit ( 240) may change the TV app to the standby state and the music app corresponding to the app call word "Genie" to the running state.

The input word transmitting unit 250 may transmit the input word input to a plurality of apps. The input word transfer unit 250 may transmit the input word to a plurality of apps when the system is activated by an activation command.

To this end, the input word transmission unit 250 may detect a natural language with respect to the input word in real time and correct the continuous input word. For example, when the input word input by the user is a sentence of 'because', the input word transmitting unit 250 detects and corrects it in real time as 'Hae->Haeba->Haebake->Haebake-Hae' in a unit unit. can be distinguished as To this end, the voice input interface of the present invention enables continuous voice input regardless of the user's total input time or interval time from the active state to the inactive state (for example, to correct the input word in word units). can be maintained

The input word transmission unit 250 may classify the input word by word unit by giving an index to the input word, and transmit the input word divided into word units in the order of input to a plurality of apps. In this case, a timer in which the update is reflected in correction or word division may be allocated. For example, if the corresponding index is updated to 'Haeba' within a certain period of time after the first input of the word 'sea' in the sentence 'exploit', the input word transmission unit 250 does not directly deliver it to a plurality of apps. , 'Haebake' and 'Haebakhae' may be continuously updated within a certain period of time after resetting the timer. At this time, it is possible to prevent the natural language from being transmitted directly to the app until the words are completely corrected and separated. When it is determined that the word is completely divided, the input word transmitting unit 250 may transmit the corresponding string to a plurality of apps.

The process of transmitting the input word input to the plurality of apps will be described in detail with reference to FIGS. 4A and 4B .

4A and 4B are exemplary views for explaining a process of transferring an input word received from a user terminal to a plurality of apps according to an embodiment of the present invention.

4A is an exemplary diagram illustrating a process of transmitting an input word received from a conventional user terminal to an app. Referring to FIG. 4A , a conventional user terminal may deliver an input word received from a user to a specific app, and may draw a conclusion through natural language detection, interpretation, and inference processes for the corresponding app.

For example, when the user terminal receives an input word 400, "What is the Hyundai Motor stock price today?" It can be delivered through the stock app.

4B is an exemplary diagram illustrating a process of transmitting an input word received from a user to a plurality of apps by the input word transmitting unit according to an embodiment of the present invention. Referring to FIG. 4B , the input word transfer unit 250 extracts only voice-related data from among the input words input to the input unit 220 from the user 115 and returns it as a character string, and uses the returned character string to change front and rear syllables and a dictionary. (dictionary) can be substituted with the most appropriate string.

For example, when the input unit 220 receives the input word 400 “what is the price of Hyundai Motor today?” from the user 115 , the input word transmitting unit 250 provides a natural language of “today/Hyundai Motor/Stock price”. The input word may be transmitted to the plurality of apps 420 so that only detection can be performed and the following processes of interpretation can be individually performed in all apps.

In order to implement the above-mentioned contents, the subject for transmitting/processing the input word in the present invention may include a central control unit (not shown) and each app. For example, 1) the central control unit (not shown or the main system) may transmit the input word received to each app and receive a return value for the input word from the app capable of processing the input word. For example, the return value may mean a result value (eg, the number of search content, etc.) that can be derived when the input word is processed, rather than directly processed by each app. In other words, since the process in which the input word/return value is transmitted by the central control unit is performed before direct processing of the input word is performed by each app, this transmission/return process can be viewed as a preprocessing process for the input word. .

2) After that, the central control unit displays the returned value received for the purpose of activating each app so that the user can recognize the return value in the corresponding app, and the user processes the input word by referring to the return value displayed in each app You can utter additional input words to select any of the available apps. The central control unit may perform a screen conversion process to process the input word through the app corresponding to the received additional input word, and may process the input word through the corresponding app after the screen conversion process is completed.

3) That is, the processing of input word delivery to each app, return value reception for input word, return value display, and screen change is performed in the central processing unit, and return value derivation and input word processing for input word are processed by each app. can be performed in Since the process of deriving the return value for the input word performed by each app is a process of deriving an example value that can be derived when the input word is processed, it is simpler than the input word processing process of directly processing and rendering the input word. It can be viewed as a process.

Returning to FIG. 2 , when the input word corresponds to a reserved word, the input word transmitting unit 250 may not transmit the input word to the plurality of apps. For example, when the user terminal 110 ignores the reserved word, the input word transmission unit 250 may transmit the input word to the plurality of apps without determining whether the input word corresponds to the reserved word. In this case, when any one of the plurality of apps is in the running state, the input word may be transmitted to the running app.

When any one of the plurality of apps is in the running state, the input word transmitter 250 may deliver the input word to the app in the running state.

The input word transmitting unit 250 may receive a return value for the input word from at least one app capable of processing the input word among a plurality of apps. The return value may include, for example, a result value that can be obtained when each app processes an input word input, information about automatic execution, and the like.

The input word transfer unit 250 may prevent transmission of unauthorized high-capacity data in each app by limiting the response time for the return value according to each app. The input word transfer unit 250 may designate a call order of each app differently by receiving a return value specified for the order in which it is received from each app. When the input word includes a keyword capable of being automatically executed for any one of a plurality of apps after a preset time, the input word transmitting unit 250 may receive information on automatic execution from an app corresponding to the automatic execution.

The state change unit 260 may change at least one app to a standby state based on the return value. For example, the standby state means that each app responds to being able to process an input word and is waiting before being executed, and the state change unit 260 converts the inactive app into a standby state based on the return value. can be changed

For example, the state change unit 260 may display a return value to be overlaid on at least one app with a result value obtained when each app processes an input word inputted thereto.

As another example, when the return value includes information on auto-run, the state change unit 260 may display the information on auto-run to be overlaid on an app corresponding to auto-run. At this time, when an additional input word is not input to the input unit 220 within a preset time, an app corresponding to automatic execution may be executed.

Also, the state change unit 260 may differentially process the display level for each app for each app based on the return value. For example, the state change unit 260 may set a difference for the emphasis effect, such as gradually changing the degree of the standby state as the input word or the return value for the input word is accumulated for some apps.

The switching unit 270 may change the app corresponding to the additional input word from the standby state to the running state, and may switch from the main screen to the running screen of the running app. In this case, the input word may be processed by the app in the running state. For example, the switching unit 270 performs rendering processing while being switched from the main screen to the running screen of the running app, and when the switching unit 270 switches from the main screen to the running app, the running state Rendering of the result screen of the input word may be processed by the in-app.

The transition unit 270 may differentially impose a screen transition effect processed for each app so that the screen transition effect for a specific app is visually highlighted.

When an input word for screen initialization is input through the input unit 220 , the conversion unit 270 may transmit the input word to the running app. In this case, the running app may perform screen initialization based on the input word.

When receiving an additional input word related to any one of the apps in the standby state through the input unit 220 , the app execution unit 280 may execute an app corresponding to the additional input word. The input word can be processed by the running app. In other words, the central control unit of the present invention can process the rendering of the conversion screen that occurs when the app is executed, and each app can process the rendering of the result screen corresponding to the input word after each app is executed. have.

5A to 5D are exemplary views for explaining a process of processing an input word received from a user terminal through a first app according to an embodiment of the present invention. When the user terminal 110 does not receive an input word from the user, all apps may be non-visualized. Alternatively, even if you visualize all apps, you can mark them as disabled.

5A is an exemplary diagram illustrating an app displayed in a standby state based on a return value for an input word in a user terminal according to an embodiment of the present invention. Referring to FIG. 5A , the user terminal 110 may receive an input word “infinite challenge” from the user 115 .

The user terminal 110 may transmit the input word “infinite challenge” to a plurality of apps, and receive and display return values for the input word from the plurality of apps. In this case, the return value for the input word may be derived from each app.

For example, the user terminal 110 may receive a return value of '241 cases' through the 'music app' 510 in relation to the input word "infinite challenge", and the 'TV app' 520 You can receive the return value of '4 cases' through In this case, the user terminal 110 may change the 'music app' and the 'TV app' to the standby state, and overlay and display the result value for the input word included in the received return value on the app.

5B and 5C are exemplary diagrams for explaining a process of changing an app corresponding to an additional input word from a standby state to an execution state in a user terminal according to an embodiment of the present invention.

Referring to FIG. 5B , when the user terminal 110 receives "TV" as an additional input word, the user terminal 110 changes the 'TV app' from the standby state to the running state 530, and executes it from the main screen. It can switch to the app in the state 530 . In this case, the rendering process during the transition from the main screen to the app in the running state 530 may be processed by the switching unit 270 .

Referring to FIG. 5C , when the user terminal 110 is switched from the standby app to the running screen 540 of the running app, the configuration of the running screen 540 after the switch is processed by the running app. can be In this case, the rendering process on the result screen of the input word may be performed by the app in the running state.

5D is an exemplary diagram illustrating a process of processing an additional input word by an app running in a user terminal according to an embodiment of the present invention. Referring to FIG. 5D , the user terminal 110 receives an additional input word “play” from the user 115 after the 'TV App', which was in the standby state, is switched to the running state, the received additional input It can be delivered to the 'TV app' that has been converted to the running state. In this case, the subject of VoD may be a 'TV app'.

Thereafter, the user terminal 110 may output the “Infinite Challenge” VoD 550 through the display device 120 .

If "back" or "end" is inputted from the user 115 instead of "play", the user terminal 110 may initialize the screen. In this case, the conversion unit 270 may request screen initialization to each app, perform a screen change, and each app may initialize an actual screen and an object, respectively.

6A and 6B are exemplary views illustrating a process of processing an input word received from a user terminal through a second app according to an embodiment of the present invention. When the user terminal 110 does not receive an input word from the user, all apps may be non-visualized. Alternatively, even if you visualize all apps, you can mark them as disabled.

6A is an exemplary diagram illustrating an app displayed in a standby state based on a return value for an input word in a user terminal according to an embodiment of the present invention. Referring to FIG. 6A , when the user terminal 110 receives the input word "KT" from the user 115, the user terminal 110 detects a natural language in real time and displays specific information on the screen based on the haejang string. It can be additionally displayed, or an existing object can be marked separately.

Thereafter, when an additional input word of "tell me about the stock today" 600 is inputted from the user 115, the user terminal 110 detects a natural language in real time and indexes 'KT/today/stock/tell/tell me'. It can be delivered to multiple apps in order. This process may be performed by the switching unit 270 of the user terminal 110 .

The user terminal 110 may change the 'Daewoo Securities App' 610 to a standby state based on the return value for the input word. When the user terminal 110 receives "Daewoo Securities" as an additional input word from the user, the user terminal 110 changes the app corresponding to the additional input word from the standby state to the running state, and the app in the running state from the main screen You can switch to the execution screen of Rendering processing during the transition from the main screen to the execution screen of the running app may be processed by the switching unit 270 instead of the corresponding app.

6B is an exemplary diagram for explaining a process of changing an app corresponding to an additional input word from a standby state to an execution state in the user terminal according to an embodiment of the present invention. Referring to FIG. 6B , when the 'Daewoo Securities App' 610 is changed to the running state, the 'Daewoo Securities App' 610 may query and display the 'KT' stock price 620 . In this case, the rendering process may be performed by the 'Daewoo Securities App' 610 .

If "back" or "end" is received from the user 115 , the user terminal 110 may perform screen initialization. In this case, the conversion unit 270 may request screen initialization to each app, perform a screen change, and each app may initialize an actual screen and an object, respectively.

7A and 7B are exemplary views illustrating a process of processing an input word received from a user terminal through a third app according to an embodiment of the present invention. When the user terminal 110 does not receive an input word from the user, all apps may be non-visualized. Alternatively, even if you visualize all apps, you can mark them as disabled.

7A is an exemplary diagram illustrating an app displayed in a standby state based on a return value for an input word in a user terminal according to an embodiment of the present invention. Referring to FIG. 7A , when the user terminal 110 receives an input word "YTN" 700 from the user 115, the user terminal 110 detects a natural language in real time and displays it on the screen based on the decoded string. Specific information may be additionally displayed, or an existing object may be marked separately. In this case, the performing subject may be the switching unit 270 of the user terminal 110 .

'TV app' (710) determines that the input word 'YTN' (700) contains a keyword that can be automatically executed, and receives information about automatic execution from the app corresponding to automatic execution, and provides information on automatic execution. Information can be displayed to be overlaid on the app corresponding to autorun. For example, the switching unit 270 of the user terminal 110 visually displays on the screen a guide that automatic execution is made after a predetermined time, and a command (eg, "Turn ", "play", "play", etc.) may receive information from the 'TV app' 710 and additionally display the corresponding command on the screen.

Thereafter, when no input is received from the user 115, the user terminal 110 may display the 'YTN' channel screen 720 by performing a channel change immediately when the timer set within a predetermined time expires. .

Alternatively, when receiving an input word of “turn on” from the user 115 , the user terminal 110 may ignore the timer and immediately change the channel.

Alternatively, when a separate input word or a reserved word is input from the user 115 , the user terminal 110 may remove the timer and the immediate execution command, and may also remove related indications.

8 is an exemplary diagram for explaining a process of transmitting an input word input from a user terminal to a plurality of apps and receiving a return value for the input word according to an embodiment of the present invention. Referring to FIG. 8 , the user terminal 110 may recognize an input word received from the user as a general search word regardless of whether or not there is a reserved word in the app and transmit it to a plurality of apps.

For example, when the user inputs "Moon Jae-in" 800 as an input word, the user terminal 110 transmits "Moon Jae-in" 800 to a plurality of apps, and returns a return value thereof from each app. can receive For example, the user terminal 110 may receive return values from the plurality of search apps 810 to 840 .

The user terminal 110 may check the search result for the input word through the search app. For example, the user terminal 110 may check the search result through 'Naver' corresponding to the second app, and when receiving the input word "up" from the user 115, the search result is displayed through 'Google'. You can check it, and if you input the word "down", you can check the search results through 'YouTube'.

When the user terminal 110 receives the reserved word "again" from the user 115, the user terminal 110 may transmit the input word input within a predetermined time back to the plurality of apps as a general character string.

9A and 9B are exemplary views showing an app converted to an execution state in a user terminal according to an execution form provided by the app according to an embodiment of the present invention.

9A is an exemplary diagram illustrating an execution form provided by a TV app in an execution state according to an embodiment of the present invention. Referring to FIG. 9A , the user terminal 110 may receive an input word “infinite challenge” 900 from the user 115 and display VoD information 910 for 535 infinite challenges. In this case, when "play" 920 is input from the user 115, the user terminal 110 may reproduce the VoD of the corresponding session.

9B is an exemplary diagram illustrating an execution form provided by a music app in an execution state according to an embodiment of the present invention. Referring to FIG. 9B , the user terminal 110 may receive an input word “Do-Hyun Yoon” 950 from the user 115 and display information 960 of an album recently released by “Do-Hyun Yoon” 950 . . At this time, when "play" 970 is input from the user 115, the user terminal 110 may play the corresponding album.

10 is a flowchart of a method of processing an input word according to whether a reserved word is input in a user terminal according to an embodiment of the present invention. Referring to FIG. 10 , the user terminal 110 may determine whether the system is activated ( S1000 ). Activation of the system is a state in which it is checked whether an input word is ready to be transmitted to each app based on the user's 115 voice input. For example, when the user 115 is only watching TV, the system is deactivated state can be considered.

For example, when the system is not activated (S1001), the user terminal 110 may determine whether an activation command is input (S1002). When an activation command is not input (S1003), the user terminal 110 ignores the input word (S1004), and when an activation command is input (S1005), the system may be activated (activated state) (S1006) ). The active command may be a command that transmits an input word to each app or drives the system to a state in which a specific app can be called.

When the system is activated (S1007), the user terminal 110 may determine whether the reserved word is ignored (S1008). The state of ignoring reserved words may mean a state in which the user terminal 110 ignores all reserved words and regards all input words input by the user 115 as search words.

The input word processing system proposed in the present invention may process all voices input from the user 115 in real-time in units of words or deliver them to a plurality of apps. However, when the input word input in real time is a reserved word, the user terminal 110 may perform a command corresponding to the reserved word. In this case, since a reserved word cannot be used as an input word such as a search keyword, in the present invention, a status value of 'reserved word ignored state' is provided so that exception processing can be performed as a preferential branching statement in the uppermost stage of input word processing. When the 'reserved word ignore state' is set, the user terminal 110 does not perform processing related to all reserved words.

The 'reserved word ignore state' can be set for a certain situation forcibly when the user 115 inputs a specific reserved word or a specific situation occurs in the app.

For example, when the user 115 inputs a specific reserved word, if the user 115 inputs the input word 'search' before inputting another input word in the main app, the sub-app is running in the specific reserved word in advance. It may include a case where a reserved word is entered. For example, when the reserved word “search” or “again” is inputted from the user 115 while the 'music app' is running, the 'search app' provides a search result, but the user 115 says "again" This may be the case when the keyword "" is input.

As another example, when a specific situation occurs in an app, each app may artificially implement a 'reserved word ignoring state' even if the user 115 does not input a separate utterance. For example, it may be a case in which no search results exist in an app in which a task such as a search is performed, or access or display is blocked due to an unusual circumstance.

When the above-described case occurs, the user terminal 110 may initialize all previously input character strings and maintain the 'reserved word ignore state' for a predetermined time thereafter.

For example, when the reserved word is ignored (S1009), the user terminal 110 may determine whether a specific app is running (S1010). When the specific app is in the running state (S1011), the user terminal 110 transmits the input word to the running app (S1012), and when the specific app is not in the running state (S1013), the user terminal 110 is all The input word may be transmitted to the app (S1014). The continuous input word may be continuously maintained so that voice input is possible regardless of the user's total input time or interval time from when the system is initially activated until it is deactivated (or terminated). If the reserved word is not ignored (S1015), the user terminal 110 may determine whether the system is a reserved word (S1016). If it is a system reserved word (S1017), the user terminal 110 may process the reserved word (S1018), and if it is not a system reserved word (S1019), it may determine whether it is an app reserved word (S1020).

If it is not an app reserved word (S1021), the user terminal 110 goes through a step of determining the execution state of a specific app (S1010), and if it is an app reserved word (S1022), the user terminal 110 determines whether the app call word is It can be determined (S1023). Through the app call word, the user terminal 110 may change the currently running app to the standby state and change the calling target app from the normal standby state to the running state.

In the case of the app call word (S1024), the user terminal 110 executes the app that is the target of the app call word (S1025), and if it is not the app call word (S1026), the user terminal 110 determines the execution state of the specific app and (S1027), when the specific app is not running (S1028), ignore the input word (S1029), and when the specific app is running (S1030), the input word can be processed in the running app (S1031) ). Based on whether a specific app is being executed, the user terminal 110 may determine whether an active or running app exists through the previously inputted word.

In the above description, steps S1000 to S1031 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as needed, and the order between the steps may be switched.

11 is a flowchart of a method of processing an input word in a user terminal according to an embodiment of the present invention. The method of processing the input word performed by the user terminal 110 according to the embodiment shown in FIG. 11 is a method that is time-series processed in the input word processing system 1 according to the embodiment shown in FIGS. 1 to 10 . includes steps. Therefore, even if omitted below, it is also applied to the method of processing an input word performed by the user terminal 110 according to the embodiment shown in FIGS. 1 to 10 .

In step S1110, the user terminal 110 may display the main screen and a plurality of preset apps to be exposed on the main screen.

In step S1120, the user terminal 110 may receive an input word from the user.

In step S1130, the user terminal 110 may transmit the input word input to the plurality of apps. At this time, when the user terminal 110 ignores the reserved word, the input word may be transmitted to the plurality of apps without determining whether the input word corresponds to the reserved word.

In step S1140 , the user terminal 110 may receive a return value for the input word from at least one app capable of processing the input word among a plurality of apps.

In step S1150, the user terminal 110 may change the at least one app to the standby state based on the return value.

In step S1160 , when receiving an additional input word related to any one of the apps in the standby state, the user terminal 110 may execute an app corresponding to the additional input word. For example, the input word may be processed by an app corresponding to the additional input word.

Although not shown in FIG. 11 , the user terminal 110 changes the app corresponding to the additional input word from the standby state to the running state, switching from the main screen to the running screen of the running app, and executing the input word It may further include a step of processing by the app in the state.

Although not shown in FIG. 11 , the user terminal 110 performs rendering processing for the transition from the main screen to the running screen of the running app by the main system of the user terminal 110 and the running app. After the conversion, the method may further include performing rendering processing on the result screen of the input word by the running app.

Although not shown in FIG. 11 , the user terminal 110 further performs the steps of determining whether the input word corresponds to a reserved word that should be systematically prioritized over processing in the app, and processing the reserved word when the input word corresponds to the reserved word. may include In this case, in step S1130, if the input word corresponds to a reserved word, the input word is not transmitted to the plurality of apps.

In the above description, steps S1110 to S1160 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as needed, and the order between the steps may be switched.

The method of processing an input word in the user terminal described with reference to FIGS. 1 to 11 may be implemented in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by the computer. In addition, the method of processing an input word in the user terminal described with reference to FIGS. 1 to 11 may be implemented in the form of a computer program stored in a medium executed by a computer.

Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

110: user terminal
120: display device
130: a plurality of service providing servers
210: display unit
220: input unit
230: reserved word judgment unit
240: reserved word execution unit
250: input word transfer unit
260: state change unit
270: transition part
280: app launcher

Claims (20)

In the user terminal processing the input word,
a display unit displaying a plurality of apps;
an input unit for receiving an input word from a user;
an input word transfer unit that transmits the input word to the plurality of apps and receives a return value for the input word from at least one app capable of processing the input word among the plurality of apps; and
a state change unit for changing the at least one app to a standby state based on the return value; and
When receiving an additional input word related to any one of the apps in the standby state through the input unit, the app execution unit executes an app corresponding to the additional input word
That comprising a, the user terminal.
The method of claim 1,
The input word transmission unit assigns an index to the input word, classifies the word unit by word, and transmits the input word divided into word units in the order of input to the plurality of apps.
The method of claim 1,
The return value includes a result value that can be obtained when each app processes the input word,
The state change unit will display the result value to be overlaid on the at least one app, the user terminal.
The method of claim 1,
and a switching unit that changes the app corresponding to the additional input word from the standby state to the running state, and switches from the main screen on which the plurality of apps are displayed to the running screen of the app in the running state,
The input word will be processed by the app in the running state, the user terminal.
5. The method of claim 4,
Rendering processing during the transition from the main screen to the running screen of the app in the running state is performed by the switching unit,
The rendering process for the result screen of the input word is performed by the running app after being switched to the running app, the user terminal.
5. The method of claim 4,
When an input word for screen initialization is received through the input unit, the conversion unit transmits the input word to the app in the running state,
The app in the running state will perform screen initialization based on the input word, the user terminal.
5. The method of claim 4,
When any one of the plurality of apps is in the running state, the input word transfer unit transmits the input word to the app in the running state, the user terminal.
The method of claim 1,
When the input word includes a keyword that can be automatically executed for any one of the plurality of apps after a preset time,
The input word transfer unit receives information about the automatic execution from the app corresponding to the automatic execution,
The return value will include information about the automatic execution, the user terminal.
9. The method of claim 8,
The state change unit will display the information on the automatic execution to be overlaid on the app corresponding to the automatic execution, the user terminal.
10. The method of claim 9,
When an additional input word is not input within the preset time, the app corresponding to the automatic execution is executed, the user terminal.
The method of claim 1,
When the system is activated by an activation command, the input word transmitting unit transmits the input word to the plurality of apps, the user terminal.
12. The method of claim 11,
a reserved word determination unit for determining whether the input word corresponds to a reserved word that should be systematically processed prior to processing in the app; and
When the input word corresponds to the reserved word, a reserved word execution unit that processes the reserved word
further comprising,
When the input word corresponds to a reserved word, the input word transmitting unit does not transmit the input word to the plurality of apps, the user terminal.
13. The method of claim 12,
When the user terminal ignores the reserved word, the input word transmitting unit transmits the input word to the plurality of apps without determining whether the input word corresponds to a reserved word.
14. The method of claim 13,
When any one of the plurality of apps is in the running state, the input word transfer unit transmits the input word to the app in the running state, the user terminal.
13. The method of claim 12,
The reserved word includes an app call word,
The reserved word execution unit, when the input word corresponds to the app call word, changes the currently running app to the standby state, and changes the app that is the target of the app call word to the running state, the user terminal.
In the method of processing an input word in a user terminal,
displaying a plurality of preset apps;
receiving an input word from a user;
transmitting the received input word to the plurality of apps;
receiving a return value for the input word from at least one app capable of processing the input word among the plurality of apps;
changing the at least one app to a standby state based on the return value; and
When receiving an additional input word related to any one of the apps in the standby state, executing an app corresponding to the additional input word
That comprising a, input word processing method.
17. The method of claim 16,
changing the app corresponding to the additional input word from the standby state to the running state;
switching from the main screen on which the plurality of apps are displayed to the running screen of the app in the running state; and
processing the input word by the app in the running state
Which will further include, the input word processing method.
18. The method of claim 17,
performing rendering processing for the transition from the main screen to the running screen of the app in the running state by the main system of the user terminal; and
After switching to the app in the running state, performing rendering processing on the result screen of the input word by the app in the running state
Which will further include, the input word processing method.
17. The method of claim 16,
determining whether the input word corresponds to a reserved word to be systematically processed prior to processing in the app; and
processing the reserved word when the input word corresponds to the reserved word
further comprising,
The step of transmitting the input word input to the plurality of apps,
When the input word corresponds to a reserved word, the input word processing method is not to transmit the input word to the plurality of apps.
20. The method of claim 19,
The step of transmitting the input word input to the plurality of apps,
When the user terminal ignores the reserved word, the input word processing method is to transmit the input word to the plurality of apps without determining whether the input word corresponds to the reserved word.

Images