KR102038147B1 - Mobile terminal for managing app/widget based voice recognition and method for the same - Google Patents

Abstract

An embodiment of the present invention provides a portable terminal for performing voice recognition application/widget management, which comprises: a terminal screen displaying a plurality of applications and widgets sorted by category; a terminal microphone receiving a user voice command; and an AI controller extracting an application or a widget matching the inputted user voice command and executing the extracted application or widget.

Description

Mobile terminal for managing app / widget based voice recognition and method for the same}

The present invention relates to a mobile terminal for performing voice recognition app / widget management and an app / widget management method, and to a mobile terminal and method for managing an app or a widget based on voice recognition.

The portable terminal market is growing rapidly due to various designs and applications that stimulate purchase. In particular, unlike conventional mobile phones that only use a given function, a smart phone can download and install various applications from the online market. In such a mobile terminal, multitasking (Multi Tasking) that performs several tasks (Task) at the same time is possible. For example, a user may surf the web while listening to music, and may be informed of receiving a text message while watching a movie.

Accordingly, the portable terminal requires an interface technology that allows a user to quickly access a corresponding application of a desired task.

Widgets are popularized and used. Widget is a graphic interface that supports the interaction between the user and the application / operating system more smoothly. The user can directly use the desired service on the standby screen in the terminal without having to visit the portal service or the content providing site. To help. For example, there are various widgets that provide services such as 'MSN Messenger, Stock, Keyword Search Box, Weather, Clock, Game, Notepad, Saying', etc. Users can access widgets that provide services that they mainly use. After the first installation on the terminal, the service can be easily used on the standby screen of the terminal through the widget without visiting the website.

However, conventional mobile terminals are equipped with unique widgets, so users are inconvenient to limit their use.

For example, Google Play, Apple Store, Samsung Apps, LG U + service, such as each manufacturer, carrier, operating system (OS) to provide a unique widget, each operating, there is a problem that users can not use it efficiently.

Korea Patent Publication 10-2012-0092487

An object of the present invention is to provide a mobile terminal and an app / widget management method that integrally performs various app / widget management based on voice recognition.

An embodiment of the present invention provides a terminal screen in which a plurality of apps and widgets are sorted by category; A terminal microphone for receiving a user voice command; And an AI controller which extracts an app or widget matching the input user voice command and executes the extracted app or widget.

The apps and widgets are sorted by category, wherein the apps and icons are sorted by operating system, manufacturer, carrier, and app theme, and all apps and widgets mounted on the mobile terminal are displayed by sorting by category in a circular graphic. Alternatively, all the apps and widgets mounted on the mobile terminal may be displayed arranged in categories in the microphone graphic.

The AI controller requests and receives biometric information from a user when an app or a widget requiring biometric information is executed, and automatically executes without asking the user when an app or widget requiring NFC, Bluetooth communication, or GPS is executed. It may be characterized by executing NFC communication, Bluetooth communication, GPS.

Extracting an app or widget matching the user voice command may extract an app or widget having the highest consumer rating when there are a plurality of apps or widgets matching the user voice command.

The mobile terminal may include a command allocation DB to which an app or a widget to be executed for each command is allocated; and extracting an app or a widget matching the user voice command includes an app or a widget assigned to an input user voice command. Can be extracted from the command assignment DB.

When there are a plurality of apps or widgets to be executed in the same command, an app or a widget to be executed for each command may be assigned by a user setting, or an app or a widget to be executed for each command may be differently assigned for each place.

Embodiment of the present invention, the mobile terminal, the category-specific display process for displaying a plurality of apps and widgets by category arranged on the screen; A user voice input process in which the mobile terminal receives a user voice command; And an app / widget execution process for each user voice command that extracts an app or widget matching the input user voice command and executes the extracted app or widget.

According to an embodiment of the present invention, a unique widget or an app may be integrated and managed for each manufacturer, a carrier, and an operating system such as the Apple Store, Samsung Apps, and LG U + service based on voice recognition.

1 is a perspective view showing a mobile terminal for performing voice recognition app / widget management according to an embodiment of the present invention.
Figure 2 is a block diagram of a mobile terminal for performing voice recognition app / widget management according to an embodiment of the present invention.
3 is a diagram illustrating a state in which apps and widgets are arranged by category according to an embodiment of the present invention.
4 and 5 are diagrams showing examples of category display according to an embodiment of the present invention;
6 is a flowchart illustrating a voice recognition app / widget management method according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms, and is provided to fully inform the scope of the invention to those skilled in the art. The invention is defined only by the scope of the claims. In addition, in the following description of the present invention, if it is determined that related related technologies and the like may obscure the gist of the present invention, detailed description thereof will be omitted.

1 is a perspective view of a mobile terminal for performing voice recognition app / widget management according to an embodiment of the present invention, Figure 2 is a configuration of a mobile terminal for performing voice recognition app / widget management according to an embodiment of the present invention 3 is a block diagram illustrating an example in which apps and widgets are sorted by category according to an exemplary embodiment of the present invention, and FIGS. 4 and 5 illustrate examples of displaying a category according to an exemplary embodiment of the present invention. to be.

The mobile terminal 100 that performs voice recognition app / widget management of the present invention refers to a mobile terminal in which an app or a widget is installed, and is illustrated in the smart phone shown in FIG. 1 (a) or FIG. 1 (b). One tablet PC or the like may correspond. In addition to smartphones and tablet PCs, slate PCs, notebook computers, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), and navigation devices are also available. (Navigation), such as a digital camera (Digital Camera), the terminal to which the present invention can be applied is not limited to the above-described type, it is natural that it can include all the various terminals that can be installed apps or widgets.

As illustrated in FIG. 2, the mobile terminal 100 performing voice recognition app / widget management of the present invention includes a terminal communication unit 110, a terminal location determiner 120, a terminal storage unit 130, and a terminal microphone ( 140, the terminal screen 150, the terminal input unit 160, and the terminal controller 180 in which the AI controller is installed. In addition, the command allocation DB 170 may be further included.

The terminal communication unit 110 is a module that performs a function of communicating through a mobile communication network. When performing a mobile communication such as 3G or 4G, an RF transmitter (not shown) which upconverts and amplifies a frequency of a wirelessly transmitted signal. And an RF receiver (not shown) for low noise amplifying the received wireless signal and down converting the frequency.

The terminal location determiner 120 is a module for identifying the location of the user terminal. Receive GPS information from the GPS satellites to determine the location of the terminal, to determine the location of the terminal based on the location of the mobile communication base station, or to determine the location of the terminal based on the beacon location determined through beacon communication from a nearby beacon device .

The terminal storage unit 130 is a storage device that stores an app, a widget execution program, and screen graphic interface (GUI) information. Such a storage body may be provided inside the device as a module capable of inputting / outputting information such as a flash memory, a compact flash card, a secure digital card, or the like, or may be provided in a separate device. It may be.

The terminal input unit 160 is a module that receives a user input according to the execution of an app or a widget displayed through the terminal screen 150.

The terminal screen 150 is a module in which an app or a widget is displayed. The terminal input unit 160 and the terminal screen 150 may be implemented in a single form of a touch screen panel. The touch screen panel is a display window provided on the front of the terminal to display a work screen by providing a touch screen screen that can simultaneously perform input and display, and provides a graphical user interface (GUI) for communication with a user. Display.

In particular, the terminal screen 150 of the present invention, the plurality of apps and widgets are arranged to be displayed by category, as shown in Figure 3 so that the apps and icons by the operating system, manufacturers, carriers, app themes are arranged and displayed . Therefore, in the mobile terminal, all operating systems, manufacturers, carriers, and apps may be sorted by category and displayed on the screen.

In addition, in the display by category, as shown in FIG. 4, all apps and widgets mounted in the mobile terminal may be displayed in a circular graphic sorted by category. In another embodiment, in the display by category, as shown in FIG. 5, all apps and widgets mounted in the mobile terminal may be displayed in a microphone graphic sorted by category. Various other types of categories may be displayed.

The terminal microphone 140 is a module for receiving a voice of a user and receiving a user voice command. The voice analog signal input from the microphone is converted into a data form and transmitted to the audio codec of the data processor. The terminal microphone 140 monitors the user's voice for 24 hours even when the screen of the portable terminal is turned off.

The terminal controller 180 is implemented as a main control unit (MCU) for controlling each function module of the mobile terminal, and is a module in which various apps or widgets are installed. Widgets are mini applications that allow users to access weather, calendars, calculators, news, games, stock information, etc. directly from a PC, mobile phone, blog or cafe without using a web browser.

For reference, a mobile terminal implemented with a smart phone or the like can install and add or delete hundreds of various apps or applications as desired by the user, and can be suited to them through various apps or widgets. You can implement the interface. Therefore, you can download apps and widgets from Google Market, Apple Store, etc. and install them on your smartphone.

On the other hand, the terminal controller 180 is equipped with an AI controller of the present invention. The AI controller extracts an app or widget that matches the input voice command and executes the extracted app or widget. For example, if a user calls the AI controller's name (for example, 'App Doll') and then voices the user's voice command 'Calculator', the calculator widget of the manufacturer Samsung Apps or the mobile operator's calculator widget To be executed. Likewise, if the user calls the AI controller's name (eg, 'App Doll') and then voicely commands the user's voice command 'Gallery', the picture view widget of the manufacturer Samsung Apps or the picture view widget of the carrier is executed. Be sure to

In addition, when the app or widget is executed, the voice prompts that the app or widget has been executed, and when using the app or widget, multiple apps or widgets with different functions can be operated at the same time. Quit.

In addition, after executing the app or widget, the biometric information is requested and received from the user when the app or widget requiring the biometric information is executed.

In addition, after running the app or widget, when running an app or widget that requires NFC communication, Bluetooth communication, GPS, let the user automatically execute NFC communication, Bluetooth communication, GPS without querying.

Meanwhile, according to a user's voice command, multiple apps or widgets having different functions may be executed at the same time, or one of the apps or widgets may be executed.

When implementing to execute any one app or widget, extracting the app or widget matching the user voice command can be implemented in two ways as follows.

One is to extract an app or widget having the highest consumer rating among a plurality of apps or widgets matching the user voice command. To this end, when there are a plurality of apps or widgets matching the user voice command, the app or widget having the highest consumer rating may be extracted. Extract and run apps or widgets that other consumers have rated as good.

Another method is to utilize DB (DataBase).

To this end, the mobile terminal includes a command assignment DB 170 to which an app or a widget to be executed for each command is assigned. For example, a calculator may be assigned a calculator widget of a manufacturer, Samsung Apps, and a gallery may be assigned a picture view widget of an operating system, Google. Here, the command allocation DB 170 may include a hard disk drive, a solid state drive, a flash memory, a compact flash card, a secure digital card, and an SM card. (Smart Media Card), an MMC card (Multi-Media Card), a memory stick (Memory Stick), such as a module that can input and output information may be provided inside the device, or may be provided in a separate device.

Therefore, to extract the app or widget matching the user voice command, to extract the app or widget assigned to the input user voice command from the command assignment DB (170). For example, if the user calls the name of the AI controller (eg, 'App Doll') and then voices the user's voice command 'Gallery', the picture view widget of the manufacturer Samsung Apps is extracted.

If there are a plurality of apps or widgets to be executed in the same command, the apps or widgets to be executed for each command are assigned by the user setting. The user sets and assigns the app or widget to use by command.

In addition, when there are a plurality of apps or widgets to be executed in the same command, the apps or widgets to be executed for each command are allocated differently for each location (terminal location). The same command allows each place to have a different app or widget. For example, in the 'Gallery' command, the picture view widget of the manufacturer Samsung Apps is assigned to the first location range, the picture view widget of LG U Plus, the carrier, is assigned to the second location range, Assuming that the picture view widget of Google, which is an operating system, is assigned, when the user voice command 'Gallery' is input in the first place, the picture view widget of Samsung Apps is executed, and the user voice called 'Gallery' in the second place. If a command is entered, the picture view widget of the carrier LG U Plus is executed, and if a user voice command called 'gallery' is entered in the third place, the picture view widget of the operating system Google is executed.

Therefore, even if the same voice command is different widgets or apps that are executed for each place, the user can run an efficient widget or app for each place. For example, use a Samsung widget at home and a Google widget at the office.

6 is a flowchart illustrating a voice recognition app / widget management method according to an embodiment of the present invention.

As shown in FIG. 6, the voice recognition app / widget management method of the present invention includes a display process for each category (S610), a user voice input process (S620), and an app / widget execution process for each user voice command (S630). can do.

The display process for each category (S610) is a process in which the mobile terminal displays a plurality of apps and widgets sorted by category on the screen.

The sorting display by category may allow apps and icons to be sorted and displayed according to operating systems, manufacturers, carriers, and app themes. Therefore, in the mobile terminal, all operating systems, manufacturers, carriers, and apps may be sorted by category and displayed on the screen.

In addition, in displaying by category, all apps and widgets mounted on the mobile terminal may be displayed in a circular graphic sorted by category. Also, in another embodiment, in displaying by category, all apps and widgets mounted on the portable terminal may be displayed by sorting by category in the microphone graphic. Various other types of categories may be displayed.

The user voice input process (S620) is a process in which the mobile terminal receives a user voice command.

The app / widget execution process for each user voice command (S630) is a process in which the AI controller of the mobile terminal extracts an app or widget matching the input user voice command and executes the extracted app or widget. For example, if a user calls the AI controller's name (for example, 'App Doll') and then voices the user's voice command 'Calculator', the calculator widget of the manufacturer Samsung Apps or the mobile operator's calculator widget To be executed. Likewise, if the user calls the AI controller's name (eg, 'App Doll') and then voicely commands the user's voice command 'Gallery', the picture view widget of the manufacturer Samsung Apps or the picture view widget of the carrier is executed. Be sure to

If there are a plurality of apps or widgets to be executed in the same command, the apps or widgets to be executed for each command may be executed differently according to user settings.

In addition, when there are a plurality of apps or widgets to be executed in the same command, the apps or widgets to be executed for each command may be executed differently for each place (terminal location). Even the same command causes different apps or widgets to run in different places. For example, in the 'Gallery' command, the picture view widget of the manufacturer Samsung Apps is executed in the first place range, and the picture view widget of LG U Plus, the carrier, is executed in the second place range.

The embodiments in the above description of the present invention are presented by selecting the most preferred examples to help those skilled in the art from the various possible examples, and the technical spirit of the present invention is not necessarily limited or limited only by the embodiments. However, various changes, modifications, and other equivalent embodiments may be made without departing from the technical spirit of the present invention.

100: portable terminal 110: terminal communication unit
120: terminal location determining unit 130: terminal storage unit
140: terminal microphone 150: terminal screen
160: terminal input unit 170: command assignment DB
180: terminal control unit

Claims (7)

A terminal screen displaying a plurality of apps and widgets sorted by category;
A terminal microphone which receives a user voice command and converts the input voice analog signal into a data form and transmits the same to an audio codec of the data processor; And
An AI controller which extracts an app or widget that matches the input user voice command, executes the extracted app or widget, and guides the execution of the executed app or widget by voice;
Including,
The apps and widgets are arranged by category,
The apps and icons are arranged by operating system, manufacturer, carrier, and app theme.
All apps and widgets on your mobile device are displayed sorted by category in a circular graphic, or all apps and widgets on your mobile device are displayed sorted by category within a microphone graphic,
The mobile terminal,
A command allocation DB to which an app or a widget to be executed for each command is allocated;
Equipped with,
Extracting an app or widget matching the user voice command, extracts the app or widget assigned to the input user voice command from the command assignment DB,
The mobile terminal,
A terminal communication unit performing a communication function through a mobile communication network;
A module for determining the location of a user terminal, which receives GPS information to determine the location of the terminal, based on the location of the mobile communication base station, based on the location of the terminal, or based on the beacon location determined through beacon communication from the beacon device. A terminal location determiner for detecting a location of the terminal;
A module capable of inputting and outputting information, comprising: a terminal storage unit provided inside or separately provided;
A terminal input unit configured to receive a user input according to the execution of an app or a widget displayed on the terminal screen;
A mobile terminal for performing voice recognition app / widget management, characterized in that it further comprises.
delete The method of claim 1, wherein the AI controller,
Request and receive biometric information from the user when running an app or widget that requires biometric information,
A mobile terminal that performs voice recognition app / widget management, which executes NFC communication, Bluetooth communication, and GPS automatically without querying a user when executing an app or widget that requires NFC communication, Bluetooth communication, or GPS.
The method of claim 1, wherein the extracting the app or widget matching the user voice command,
If there are a plurality of apps or widgets that match the user voice command, the mobile terminal performs voice recognition app / widget management, characterized in that for extracting the app or widget with the highest consumer rating.
delete The method according to claim 1,
If you have multiple apps or widgets to run on the same command,
A mobile terminal performing voice recognition app / widget management, wherein an app or a widget to be executed by a command is assigned by a user setting, or an app or a widget to be executed by a command is individually assigned to each place.
delete

Images