KR102114064B1 - Mobile terminal - Google Patents

Abstract

The mobile terminal according to an embodiment of the present invention includes a microphone for receiving a user's voice command, a memory, and a control unit for controlling the microphone and the memory, wherein the control unit receives the first voice command and receives the first If the recognition of the voice command fails, the first voice pattern of the first voice command is stored in the memory, the second voice command is received, and the second voice pattern of the second voice command is the first voice pattern In the case of a pattern similar to the one, the voice recognition threshold indicating the ability of speech recognition is adjusted, and when the second voice command is recognized according to the adjustment of the speech recognition threshold, the first voice pattern stored in the memory and the adjusted The voice recognition threshold can be sent to the server.

Description

Mobile terminal {MOBILE TERMINAL}

The present invention relates to a mobile terminal, and more particularly, to a mobile terminal capable of automatically training a speech pattern in which speech recognition has failed.

Data mining and pattern recognition are based on context-aware technologies that include devices that can recognize or predict a user's needs based on past information, the user's environment, the user's current status, or a user's specific schedule using machine learning. And other intelligent algorithms and technologies.

Recently, there have been many technologies that utilize devices through voice recognition through mobile terminals such as smart phones.

To improve the existing speech recognition technology, data that is not well-recognized is collected and trained again through a speech recognition model, which is a representative example of evolving artificial intelligence.

However, it is not easy to collect data that speech recognition has failed while being used by real users.

It is very cumbersome for a user to collect data whose speech recognition has failed through the user UI.

In addition, it is a privacy problem to send data to the server after automatic login through the application for which voice recognition has failed.

Therefore, there is a need for a method for utilizing the training to notify that the current speech recognition has failed and to use the data at that time for privacy.

The present invention aims to solve the above and other problems.

An object of the present invention is to provide a mobile terminal capable of automatically training by recognizing a duplicate speech pattern in which speech recognition has failed.

An object of the present invention is to provide a mobile terminal capable of improving a speech recognition rate through automatic training.

The mobile terminal according to an embodiment of the present invention includes a microphone for receiving a user's voice command, a memory, and a control unit for controlling the microphone and the memory, wherein the control unit receives the first voice command and receives the first If the recognition of the voice command fails, the first voice pattern of the first voice command is stored in the memory, the second voice command is received, and the second voice pattern of the second voice command is the first voice pattern In the case of a pattern similar to the one, the voice recognition threshold indicating the ability of speech recognition is adjusted, and when the second voice command is recognized according to the adjustment of the speech recognition threshold, the first voice pattern stored in the memory and the adjusted The voice recognition threshold can be sent to the server.

Further scope of applicability of the present invention will become apparent from the following detailed description. However, various changes and modifications within the spirit and scope of the present invention may be clearly understood by those skilled in the art, and thus, it should be understood that specific embodiments such as detailed description and preferred embodiments of the present invention are given as examples only.

According to an embodiment of the present invention, a voice recognition threshold may be lowered for repeated speech patterns in which speech recognition has failed, so that the performance of speech recognition can be effectively improved.

1A is a block diagram illustrating a mobile terminal related to the present invention.
1B and 1C are conceptual views of an example of a mobile terminal related to the present invention viewed from different directions.
2 is a conceptual diagram illustrating another example of a deformable mobile terminal according to the present invention.
3 is a perspective view showing an example of a watch-type mobile terminal related to another embodiment of the present invention.
4 is a perspective view showing an example of a glass-type mobile terminal according to another embodiment of the present invention.
5 is a flowchart illustrating a method of operating a mobile terminal according to an embodiment of the present invention.
6 is a flowchart illustrating a method of operating a mobile terminal according to another embodiment of the present invention.
7 is a flowchart illustrating a method of operating a mobile terminal according to another embodiment of the present invention.
8 is a diagram for explaining a process of performing voice recognition re-training in a server according to an embodiment of the present invention.
9 is a view for explaining cases in which the recognition of a conventional voice activated word fails.

Hereinafter, exemplary embodiments disclosed herein will be described in detail with reference to the accompanying drawings, but the same or similar elements are assigned the same reference numbers regardless of the reference numerals, and overlapping descriptions thereof will be omitted. The suffixes "modules" and "parts" for components used in the following description are given or mixed only considering the ease of writing the specification, and do not have meanings or roles distinguished from each other in themselves. In addition, in describing the embodiments disclosed in this specification, detailed descriptions of related known technologies are omitted when it is determined that the gist of the embodiments disclosed in this specification may be obscured. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed in the specification is not limited by the accompanying drawings, and all modifications included in the spirit and technical scope of the present invention , It should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from other components.

When an element is said to be "connected" or "connected" to another component, it is understood that other components may be directly connected to or connected to the other component, but there may be other components in between. It should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that no other component exists in the middle.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as "comprises" or "have" are intended to indicate the presence of features, numbers, steps, actions, components, parts or combinations thereof described in the specification, but one or more other features. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Mobile terminals described herein include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, and slate PCs. , Tablet PC (tablet PC), ultrabook (ultrabook), wearable device (wearable device, for example, a watch-type terminal (smartwatch), glass-type terminal (smart glass), HMD (head mounted display), etc. may be included have.

However, the configuration according to the embodiment described in the present specification can be easily recognized by those skilled in the art that it can be applied to a fixed terminal such as a digital TV, a desktop computer, and a digital signage, except when applicable only to a mobile terminal. will be.

1A to 1C, FIG. 1A is a block diagram illustrating a mobile terminal related to the present invention, and FIGS. 1B and 1C are conceptual views of an example of a mobile terminal related to the present invention viewed from different directions.

The mobile terminal 100 includes a wireless communication unit 110, an input unit 120, a running data unit 130, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit ( 180) and a power supply unit 190.

The components shown in FIG. 1A are not essential for implementing a mobile terminal, so the mobile terminal described herein may have more or fewer components than those listed above.

More specifically, the wireless communication unit 110 among the components, between the mobile terminal 100 and the wireless communication system, between the mobile terminal 100 and another mobile terminal 100, or the mobile terminal 100 and an external server It may include one or more modules that enable wireless communication between. In addition, the wireless communication unit 110 may include one or more modules connecting the mobile terminal 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short-range communication module 114, and a location information module 115. .

The input unit 120 may include a camera 121 for inputting a video signal or a video input unit, a microphone for inputting an audio signal (microphone 122), or an audio input unit, a user input unit 123 for receiving information from a user, for example , A touch key, a mechanical key, and the like. The voice data or image data collected by the input unit 120 may be analyzed and processed by a user's control command.

The learning data unit 130 may be configured to receive, classify, store, and output information to be used for data mining, data analysis, intelligent decision making, and machine learning algorithms and techniques.

The running data unit 130 may be received, detected, detected, generated, predefined, or otherwise output by the terminal or communicated with other components, devices, terminals, or terminals in a received, detected, detected, generated, predefined, or otherwise manner It may include one or more memory units configured to store data output by an entity.

The running data unit 130 may include a memory integrated or implemented in a terminal. In some embodiments, the running data unit 130 may be implemented using the memory 170.

Alternatively or additionally, the running data unit 130 may be implemented using memory associated with the terminal, such as external memory coupled directly to the terminal or memory maintained in a server communicating with the terminal.

  In another embodiment, the running data unit 130 may be implemented using memory maintained in a cloud computing environment, or another remote memory location accessible by a terminal through a communication method such as a network.

The learning data unit 130 generally includes one or more data for identifying, indexing, categorizing, manipulating, storing, retrieving, and outputting data for use in supervised or unsupervised learning, data mining, predictive analytics, or other machines. It can be configured to store in a database.

The information stored in the running data unit 130 may be used by the controller 180 or one or more other controllers of the terminal using any of a variety of different types of data analysis and machine learning algorithms and techniques.

 Examples of such algorithms and techniques include k-recent neighbor systems, fuzzy logic (eg probability theory), neural networks, Boltzmann machines, vector quantization, pulse neural networks, support vector machines, maximum margin classifier, hill climbing, inductive logic system Bayesian Networks, peritnets (e.g. finite state machines, milli machines, Moore finite state machines), classifier trees (e.g. perceptron trees, support vector trees, Markov trees, decision tree forests, random forests), stake models and systems, Various types of information can be provided, including artificial fusion, sensor fusion, image fusion, reinforcement learning, augmented reality, pattern recognition, and automated planning.

The controller 180 requests data of the running data unit 130 to determine or predict at least one executable operation of the terminal based on the generated or determined information using data analysis and machine learning algorithms and techniques, It can be searched, received or utilized, and the terminal can be controlled to perform a predicted or desirable operation among the at least one executable operation.

The controller 180 may perform various functions for implementing emulation of intelligence (ie, a knowledge-based system, a reasoning system, and a knowledge acquisition system). It can be applied to various types of systems (eg, fuzzy logic systems), including adaptive systems, machine learning systems, artificial neural networks, and the like.

The control unit 180 also carries voice and natural language speech processing, such as an I / O processing module, an environmental condition module, a voice-text (STT) processing module, a natural language processing module, a work flow processing module, and a service processing module. It can include submodules that enable performance and / or execution.

Each of these sub-modules can also have access to one or more systems or data and models, or a subset or superset of them, in a terminal, vocabulary index, user data, work flow model, service model and automatic speech recognition It can provide various functions including (ASR) system.

In another embodiment, the controller 180 or another aspect of the terminal may be implemented as the sub-module, system, or data and model.

In some examples, based on data from the running data unit 130, the controller 180 can be configured to detect and detect needs based on the user's intention or contextual conditions expressed in user input or natural language input.

The controller 180 can actively derive and acquire information necessary to completely determine a need based on context conditions or a user's intention (eg, historical input and output, pattern matching, unambiguous words, input) By analyzing historical data including intent, etc.).

The controller 180 may determine a task flow for executing a function in response to a need based on the context condition or the user's intention, and execute the task flow to satisfy the need according to the context condition or the user's intention.

In some embodiments, the controller 180 may be a member, memristor, mutual conductance amplifier, pulsed neural circuit, artificially intelligent nanotechnology system (eg autonomous nanomachine) or artificially intelligent quantum mechanical system (eg quantum neural network) And the like.

In some embodiments, the controller 180 may include a machine vision system, an acoustic recognition system, a handwriting recognition system, a data fusion system, a sensor fusion system, and a pattern recognition system such as a soft sensor. Machine vision systems can also include content-based image retrieval, optical character recognition, augmented reality, egomotion, tracking or optical flow.

The control unit 180 collects, detects, and extracts signals or data used for data analysis and machine learning operations through one or more sensing components in the terminal, in order to collect information for processing and storage in the running data unit 130. It can be configured to detect and / or receive.

Collecting information may include sensing information through a sensor, extracting information stored in the memory 170, or receiving information from another terminal, entity, or external storage device through communication means.

Accordingly, in one example, the controller 180 may collect history usage information at the terminal and store history usage information for use in data analysis.

The controller 180 may determine the best match for executing a specific function using predictive modeling based on the stored information when the future occurs, based on the stored historical usage information.

The controller 180 may also receive or detect surrounding environment information or other information through the sensing unit 140.

Also, the controller 180 may receive a broadcast signal and / or broadcast-related information, a radio signal, and radio data through the radio communication unit 110.

The controller 180 may receive image information (or a corresponding signal), audio information (or a corresponding signal), data, or user input information from the input unit.

The control unit 180 collects information in real time, processes or classifies information (for example, a knowledge graph, a command policy, a personalization database, a conversation engine, etc.), and processes the processed information in a memory 170 or a running data unit ( 130).

When the operation of the terminal is determined based on data analysis and machine learning algorithms and techniques, the controller 180 can control components of the terminal to execute the determined operation. In addition, the controller 180 may perform the determined operation by controlling the terminal according to the control command.

When a specific operation is performed, the controller 180 analyzes historical information indicating execution of a specific operation through data analysis and machine learning algorithms and techniques, and performs updating of previously learned information based on the analyzed information. Can be.

Accordingly, the control unit 180 may improve the accuracy of future performance of the data analysis and machine learning algorithms and techniques based on the updated information together with the learning data unit 130.

The sensing unit 140 may include one or more sensors for sensing at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information.

For example, the sensing unit 140 includes a proximity sensor 141, an illumination sensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, and gravity G-sensor, gyroscope sensor, motion sensor, RGB sensor, infrared sensor (IR sensor), fingerprint scan sensor, ultrasonic sensor , Optical sensor (e.g., camera (see 121)), microphone (see 122), battery gauge, environmental sensor (e.g. barometer, hygrometer, thermometer, radioactivity sensor, Thermal sensor, gas sensor, and the like), and a chemical sensor (eg, an electronic nose, a health care sensor, a biometric sensor, etc.). Meanwhile, the mobile terminal disclosed in this specification may combine and use information sensed by at least two or more of these sensors.

The output unit 150 is for generating output related to vision, hearing, or tactile sense, and includes at least one of a display unit 151, an audio output unit 152, a hap tip module 153, and an optical output unit 154 can do. The display unit 151 may form a mutual layer structure with the touch sensor or may be integrally formed, thereby realizing a touch screen. The touch screen may function as a user input unit 123 that provides an input interface between the mobile terminal 100 and a user, and at the same time, provide an output interface between the mobile terminal 100 and the user.

The interface unit 160 serves as a passage with various types of external devices connected to the mobile terminal 100. The interface unit 160 connects a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, and an identification module. It may include at least one of a port, an audio input / output (I / O) port, a video input / output (I / O) port, and an earphone port. In the mobile terminal 100, in response to an external device being connected to the interface unit 160, appropriate control related to the connected external device may be performed.

In addition, the memory 170 stores data supporting various functions of the mobile terminal 100. The memory 170 is a plurality of application programs (application programs or applications) driven in the mobile terminal 100, data for the operation of the mobile terminal 100, instructions, the operation of the running data unit 130 Data (eg, at least one algorithm information for machine learning) may be stored. At least some of these applications can be downloaded from external servers via wireless communication. In addition, at least some of these application programs may exist on the mobile terminal 100 from the time of shipment for the basic functions of the mobile terminal 100 (for example, an incoming call, a calling function, a message reception, and a calling function). Meanwhile, the application program may be stored in the memory 170 and installed on the mobile terminal 100 to be driven by the controller 180 to perform an operation (or function) of the mobile terminal.

In addition to the operations related to the application program, the controller 180 generally controls the overall operation of the mobile terminal 100. The controller 180 may provide or process appropriate information or functions to the user by processing signals, data, information, etc. input or output through the above-described components or by driving an application program stored in the memory 170.

In addition, the controller 180 may control at least some of the components discussed with reference to FIG. 1A to drive an application program stored in the memory 170. Furthermore, the controller 180 may operate by combining at least two or more of the components included in the mobile terminal 100 to operate the application program.

Under the control of the controller 180, the power supply unit 190 receives external power and internal power to supply power to each component included in the mobile terminal 100. The power supply unit 190 includes a battery, and the battery may be a built-in battery or a replaceable battery.

Hereinafter, before looking at various embodiments implemented through the mobile terminal 100 described above, the components listed above will be described in more detail with reference to FIG. 1A.

First, referring to the wireless communication unit 110, the broadcast reception module 111 of the wireless communication unit 110 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or switching of broadcast channels for at least two broadcast channels.

The broadcast management server may refer to a server that generates and transmits broadcast signals and / or broadcast-related information or a server that receives previously generated broadcast signals and / or broadcast-related information and transmits them to a terminal. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, as well as a TV broadcast signal or a radio broadcast signal combined with a data broadcast signal.

The broadcast signal may be encoded according to at least one of technical standards (or broadcast methods, for example, ISO, IEC, DVB, ATSC, etc.) for transmitting and receiving digital broadcast signals, and the broadcast receiving module 111 may The digital broadcast signal may be received using a method suitable for the technical standard determined by the technical standards.

The broadcast related information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast-related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast-related information may exist in various forms, such as Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H). The broadcast signal and / or broadcast-related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 includes technology standards or communication methods for mobile communication (eg, Global System for Mobile Communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV -Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (DO), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced) transmits and receives wireless signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network constructed according to (Long Term Evolution-Advanced).

The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text / multimedia message.

The wireless Internet module 113 refers to a module for wireless Internet access, and may be built in or external to the mobile terminal 100. The wireless Internet module 113 is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

Wireless Internet technologies include, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World) Interoperability for Microwave Access (HSDPA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), and the wireless Internet module ( 113) transmits and receives data according to at least one wireless Internet technology in a range including the Internet technology not listed above.

In view of the wireless Internet access through WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc., through the mobile communication network, the wireless Internet module 113 that performs wireless Internet access through the mobile communication network ) May be understood as a type of the mobile communication module 112.

The short-range communication module 114 is for short-range communication, Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, NFC (Near Field Communication), by using at least one of Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus) technology, it can support short-range communication. The short-range communication module 114 may be provided between the mobile terminal 100 and the wireless communication system, between the mobile terminal 100 and another mobile terminal 100, or through the mobile terminal 100 through wireless area networks. ) And other mobile terminals (100, or an external server) is located between the network can support wireless communication. The short-range wireless communication network may be wireless personal area networks.

Here, the other mobile terminal 100 is a wearable device capable of exchanging data with (or interlocking with) the mobile terminal 100 according to the present invention (eg, a smartwatch), smart glasses (smart glass), HMD (head mounted display). The short-range communication module 114 may detect (or recognize) a wearable device capable of communicating with the mobile terminal 100 around the mobile terminal 100. Furthermore, when the sensed wearable device is a device authenticated to communicate with the mobile terminal 100 according to the present invention, the controller 180 may include at least a portion of data processed by the mobile terminal 100, the short-range communication module ( 114) to the wearable device. Therefore, the user of the wearable device can use data processed by the mobile terminal 100 through the wearable device. For example, according to this, the user performs a phone call through the wearable device when a call is received by the mobile terminal 100 or, when a message is received by the mobile terminal 100, the received message through the wearable device It is possible to check the message.

The location information module 115 is a module for obtaining a location (or current location) of a mobile terminal, and examples thereof include a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, if the mobile terminal utilizes a GPS module, the location of the mobile terminal can be obtained by using a signal from a GPS satellite.

As another example, if the mobile terminal utilizes a Wi-Fi module, the location of the mobile terminal may be obtained based on information of a Wi-Fi module and a wireless access point (AP) that transmits or receives a wireless signal. If necessary, the location information module 115 may perform any function of other modules of the wireless communication unit 110 in order to obtain data regarding the location of the mobile terminal by substitution or additionally. The location information module 115 is a module used to acquire the location (or current location) of the mobile terminal, and is not limited to a module that directly calculates or acquires the location of the mobile terminal.

Next, the input unit 120 is for input of image information (or signals), audio information (or signals), data, or information input from a user. For input of image information, the mobile terminal 100 is one Alternatively, a plurality of cameras 121 may be provided. The camera 121 processes image frames such as still images or moving pictures obtained by an image sensor in a video call mode or a shooting mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170. Meanwhile, the plurality of cameras 121 provided in the mobile terminal 100 may be arranged to form a matrix structure, and through the camera 121 forming the matrix structure, various angles or focuses may be applied to the mobile terminal 100. A plurality of image information may be input. Also, the plurality of cameras 121 may be arranged in a stereo structure to acquire left and right images for realizing a stereoscopic image.

The microphone 122 processes external sound signals as electrical voice data. The processed voice data may be used in various ways according to a function (or a running application program) being performed by the mobile terminal 100. Meanwhile, various noise reduction algorithms for removing noise generated in the process of receiving an external sound signal may be implemented in the microphone 122.

The user input unit 123 is for receiving information from a user. When information is input through the user input unit 123, the controller 180 may control the operation of the mobile terminal 100 to correspond to the input information. . The user input unit 123 is a mechanical input means (or a mechanical key, for example, a button located on the front or rear or side of the mobile terminal 100, a dome switch, and a jog wheel) , Jog switch, etc.) and a touch input means. As an example, the touch-type input means is made of a virtual key, a soft key or a visual key displayed on the touch screen through software processing, or a part other than the touch screen On the other hand, the virtual key or the visual key may be displayed on the touch screen while having various forms, for example, a graphic or text. ), An icon, a video, or a combination thereof.

Meanwhile, the sensing unit 140 senses at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information, and generates a sensing signal corresponding thereto. The controller 180 may control driving or operation of the mobile terminal 100 based on the sensing signal, or perform data processing, function, or operation related to an application program installed in the mobile terminal 100. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object in the vicinity using mechanical force or infrared light without mechanical contact. The proximity sensor 141 may be provided with the proximity sensor 141 in the inner region of the mobile terminal or the proximity of the touch screen surrounded by the touch screen.

Examples of the proximity sensor 141 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor. When the touch screen is capacitive, the proximity sensor 141 may be configured to detect the proximity of the object with a change in electric field according to the proximity of the conductive object. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of description, the act of allowing the object to be recognized as being located on the touch screen without being touched on the touch screen is called a "proximity touch", and the touch The act of actually touching an object on the screen is called "contact touch". The location where an object is in close touch on the touch screen means a location where the object is vertically corresponding to the touch screen when the object is touched in close proximity. The proximity sensor 141 may detect a proximity touch and a proximity touch pattern (eg, proximity touch distance, proximity touch direction, proximity touch speed, proximity touch time, proximity touch position, proximity touch movement state, etc.). have.

Meanwhile, as described above, the controller 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern detected through the proximity sensor 141, and further, visual information corresponding to the processed data. It can be output on the touch screen. Furthermore, the controller 180 may control the mobile terminal 100 such that different operations or data (or information) are processed according to whether a touch on the same point on the touch screen is a proximity touch or a touch touch. .

The touch sensor uses a touch (or touch input) applied to the touch screen (or the display unit 151) using at least one of various touch methods such as a resistive film method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method. Detect.

As an example, the touch sensor may be configured to convert a change in pressure applied to a specific part of the touch screen or capacitance generated in the specific part into an electrical input signal. The touch sensor may be configured to detect a location, an area, a pressure when a touch is touched, a capacitance when a touch object is touched on the touch screen, and the like. Here, the touch object is an object that applies a touch to the touch sensor, and may be, for example, a finger, a touch pen or a stylus pen, a pointer, and the like.

As such, when there is a touch input to the touch sensor, the signal (s) corresponding to the touch is sent to the touch controller. The touch controller processes the signal (s) and then transmits corresponding data to the controller 180. Accordingly, the control unit 180 can know which area of the display unit 151 has been touched, and the like. Here, the touch controller may be a separate component from the controller 180 or may be the controller 180 itself.

Meanwhile, the controller 180 may perform different controls or perform the same control according to the type of the touch object that touches the touch screen (or a touch key provided in addition to the touch screen). Whether to perform different control or the same control according to the type of the touch object, it may be determined according to the current operating state of the mobile terminal 100 or an application program being executed.

On the other hand, the touch sensors and proximity sensors described above are independently or in combination, such as short (or tap) touch on the touch screen, long touch, multi touch, drag touch ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, etc. You can sense the touch.

The ultrasonic sensor may recognize location information of a sensing target using ultrasonic waves. Meanwhile, the control unit 180 may calculate the position of the wave generating source through information sensed by the optical sensor and the plurality of ultrasonic sensors. The position of the wave generator can be calculated by using a property in which light is much faster than ultrasonic waves, that is, a time when light reaches the optical sensor is much faster than a time when ultrasonic waves reach the ultrasonic sensor. More specifically, the position of the wave generator may be calculated by using a time difference from the time at which the ultrasound reaches the light as a reference signal.

On the other hand, the camera 121, which is considered as the configuration of the input unit 120, includes at least one of a camera sensor (eg, CCD, CMOS, etc.), a photo sensor (or image sensor), and a laser sensor.

The camera 121 and the laser sensor are combined with each other to detect a touch of a sensing target for a 3D stereoscopic image. The photo sensor may be stacked on a display element, which is configured to scan the movement of the sensing object close to the touch screen. More specifically, the photo sensor mounts photo diodes and TRs (transistors) in rows / columns to scan the contents placed on the photo sensor using electrical signals that change according to the amount of light applied to the photo diode. That is, the photo sensor performs coordinate calculation of the sensing object according to the change amount of light, and through this, location information of the sensing object may be obtained.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, the display unit 151 may display execution screen information of an application program driven by the mobile terminal 100, or user interface (UI) or graphical user interface (GUI) information according to the execution screen information. .

In addition, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image. A three-dimensional display method such as a stereoscopic method (glasses method), an auto stereoscopic method (glasses-free method), or a projection method (holographic method) may be applied to the stereoscopic display unit.

Generally, a 3D stereoscopic image is composed of a left image (image for the left eye) and a right image (image for the right eye). Depending on how the left and right images are combined into a 3D stereoscopic image, a top-down method of arranging left and right images up and down in one frame, left and right images in one frame L-to-R (left-to-right, side by side) method that is arranged in, checker board method that arranges pieces of left and right images in tile form, and left and right images in columns Or, it is divided into an interlaced method in which rows are alternately arranged, and a time sequential (frame by frame) method in which left and right images are alternately displayed by time.

Also, the 3D thumbnail image may be generated as a left image thumbnail and a right image thumbnail, respectively, from the left image and the right image of the original image frame, and may be generated as one image as they are combined. Generally, a thumbnail refers to a reduced image or a reduced still image. The thus generated left image thumbnail and right image thumbnail can be displayed with a distance difference between left and right on the screen as much as a depth corresponding to the parallax between the left image and the right image.

The left and right images necessary for the realization of the 3D stereoscopic image may be displayed on the stereoscopic display unit by the stereoscopic processing unit. The stereoscopic processing unit is configured to receive a 3D image (an image at a reference point and an image at an extended point of view) and set a left image and a right image therefrom, or receive a 2D image and convert it into a left image and a right image.

The audio output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception, call mode or recording mode, voice recognition mode, broadcast reception mode, or the like. The sound output unit 152 may also output sound signals related to functions (for example, call signal reception sound, message reception sound, etc.) performed by the mobile terminal 100. The sound output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that the user can feel. A typical example of the tactile effect generated by the haptic module 153 may be vibration. The intensity and pattern of vibration generated in the haptic module 153 may be controlled by a user's selection or setting of a control unit. For example, the haptic module 153 may synthesize different vibrations and output them sequentially or sequentially.

In addition to vibration, the haptic module 153 is used for stimulation such as pin arrangement that vertically moves with respect to the contact skin surface, jet or suction power of air through a jet or intake, grazing on the skin surface, contact with an electrode, and electrostatic force. Various tactile effects can be generated, such as an effect and an effect of reproducing a feeling of cold and warm using an element capable of absorbing heat or generating heat.

The haptic module 153 may not only deliver a tactile effect through direct contact, but may also be implemented so that the user can feel the tactile effect through muscle sensations such as fingers or arms. Two or more haptic modules 153 may be provided according to a configuration aspect of the mobile terminal 100.

The light output unit 154 outputs a signal for notifying the occurrence of an event using the light of the light source of the mobile terminal 100. Examples of events generated in the mobile terminal 100 may include receiving messages, receiving call signals, missed calls, alarms, schedule notifications, receiving emails, and receiving information through applications.

The signal output from the light output unit 154 is implemented as the mobile terminal emits light of a single color or a plurality of colors to the front or rear. The signal output may be terminated by the mobile terminal detecting the user's event confirmation.

The interface unit 160 serves as a passage with all external devices connected to the mobile terminal 100. The interface unit 160 receives data from an external device, receives power, transfers it to each component inside the mobile terminal 100, or allows data inside the mobile terminal 100 to be transmitted to the external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module (port), audio I / O (Input / Output) port, video I / O (Input / Output) port, earphone port, and the like may be included in the interface unit 160.

On the other hand, the identification module is a chip that stores various information for authenticating the usage rights of the mobile terminal 100, a user identification module (UIM), a subscriber identity module (SIM), universal user authentication It may include a module (universal subscriber identity module; USIM). The device provided with the identification module (hereinafter referred to as 'identification device') may be manufactured in a smart card format. Accordingly, the identification device may be connected to the terminal 100 through the interface unit 160.

In addition, when the mobile terminal 100 is connected to an external cradle, the interface unit 160 becomes a passage through which power from the cradle is supplied to the mobile terminal 100 or is input by the user from the cradle. Various command signals may be passages to the mobile terminal 100. Various command signals or power inputted from the cradle may be operated as signals for recognizing that the mobile terminal 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180, and may temporarily store input / output data (eg, a phone book, a message, a still image, a video, etc.). The memory 170 may store data related to various patterns of vibration and sound output when a touch is input on the touch screen.

The memory 170 is a flash memory type, a hard disk type, a solid state disk type, an SDD type (Silicon Disk Drive type), a multimedia card micro type ), Card type memory (e.g. SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read (EPMROM) It may include a storage medium of at least one type of -only memory (PROM), programmable read-only memory (PROM), magnetic memory, magnetic disk and optical disk. The mobile terminal 100 may be operated in connection with a web storage performing a storage function of the memory 170 on the Internet.

On the other hand, as described above, the controller 180 controls the operation related to the application program and the overall operation of the mobile terminal 100 in general. For example, when the state of the mobile terminal satisfies a set condition, the controller 180 may execute or release a lock state that restricts input of a user's control command to applications.

In addition, the controller 180 performs control and processing related to voice calls, data communication, video calls, etc., or performs pattern recognition processing capable of recognizing handwriting input or drawing input on the touch screen as text and images, respectively. Can be. Furthermore, in order to implement various embodiments described below on the mobile terminal 100 according to the present invention, the controller 180 may control any one or a combination of the above-described components.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power required for the operation of each component. The power supply unit 190 includes a battery, and the battery may be a built-in battery made to be rechargeable, and may be detachably coupled to a terminal body for charging and the like.

Further, the power supply unit 190 may include a connection port, and the connection port may be configured as an example of the interface 160 to which an external charger that supplies power for charging the battery is electrically connected.

As another example, the power supply unit 190 may be configured to charge the battery wirelessly without using the connection port. In this case, the power supply unit 190 uses one or more of an inductive coupling method based on a magnetic induction phenomenon from an external wireless power transmission device or a magnetic resonance coupling method based on an electromagnetic resonance phenomenon. Power can be delivered.

Meanwhile, various embodiments below may be implemented in a recording medium readable by a computer or a similar device using, for example, software, hardware, or a combination thereof.

1B and 1C, the disclosed mobile terminal 100 has a bar-shaped terminal body. However, the present invention is not limited to this, and can be applied to various structures such as a watch type, a clip type, a glass type, or a folder type, a flip type, a slide type, a swing type, and a swivel type in which two or more bodies are coupled to move relative to each other . It will be related to a specific type of mobile terminal, but the description of a specific type of mobile terminal can be generally applied to other types of mobile terminals.

Here, the terminal body may be understood as a concept of referring to the mobile terminal 100 as at least one aggregate.

The mobile terminal 100 includes a case (for example, a frame, a housing, a cover, etc.) forming an exterior. As illustrated, the mobile terminal 100 may include a front case 101 and a rear case 102. Various electronic components are disposed in the inner space formed by the combination of the front case 101 and the rear case 102. At least one middle case may be additionally disposed between the front case 101 and the rear case 102.

A display unit 151 is disposed on the front of the terminal body to output information. As illustrated, the window 151a of the display unit 151 is mounted on the front case 101 to form the front surface of the terminal body together with the front case 101.

In some cases, electronic components may also be mounted on the rear case 102. Electronic components that can be mounted on the rear case 102 include a removable battery, an identification module, and a memory card. In this case, the rear case 102 may be detachably coupled to the rear cover 103 for covering the mounted electronic components. Therefore, when the rear cover 103 is separated from the rear case 102, the electronic components mounted on the rear case 102 are exposed to the outside.

As shown, when the rear cover 103 is coupled to the rear case 102, a part of the side surface of the rear case 102 may be exposed. In some cases, the rear case 102 may be completely covered by the rear cover 103 when the coupling is performed. Meanwhile, an opening for exposing the camera 121b or the sound output unit 152b to the outside may be provided in the rear cover 103.

These cases (101, 102, 103) may be formed by injection of synthetic resin or may be formed of metal, for example, stainless steel (STS), aluminum (Al), titanium (Ti), or the like.

Unlike the above example in which a plurality of cases provide an internal space in which a plurality of cases accommodate various electronic components, the mobile terminal 100 may be configured such that one case provides the interior space. In this case, the mobile terminal 100 of a uni-body in which synthetic resin or metal extends from the side to the rear may be implemented.

On the other hand, the mobile terminal 100 may be provided with a waterproof portion (not shown) to prevent water from entering the terminal body. For example, the waterproof portion is provided between the window 151a and the front case 101, between the front case 101 and the rear case 102, or between the rear case 102 and the rear cover 103, the combination of these It may include a waterproof member for sealing the interior space of the city.

The mobile terminal 100 includes a display unit 151, first and second sound output units 152a and 152b, a proximity sensor 141, an illuminance sensor 142, a light output unit 154, the first and second units Cameras 121a and 121b, first and second operation units 123a and 123b, a microphone 122, and an interface unit 160 may be provided.

Hereinafter, as shown in FIGS. 1B and 1C, a display unit 151, a first sound output unit 152a, a proximity sensor 141, an illuminance sensor 142, and an optical output unit on the front surface of the terminal body ( 154), the first camera (121a) and the first operation unit (123a) is disposed, the second operation unit (123b), the microphone 122 and the interface unit 160 is disposed on the side of the terminal body, the terminal body The mobile terminal 100 in which the second sound output unit 152b and the second camera 121b are disposed on the rear surface of the mobile terminal 100 will be described as an example.

However, these configurations are not limited to this arrangement. These configurations may be excluded or replaced as necessary, or may be disposed on other sides. For example, the first operation unit 123a may not be provided on the front of the terminal body, and the second sound output unit 152b may be provided on the side of the terminal body rather than the rear of the terminal body.

The display unit 151 displays (outputs) information processed by the mobile terminal 100. For example, the display unit 151 may display execution screen information of an application program driven by the mobile terminal 100, or user interface (UI) or graphical user interface (GUI) information according to the execution screen information. .

The display unit 151 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (flexible). display), a three-dimensional display (3D display), an electronic ink display (e-ink display).

Also, two or more display units 151 may be present depending on the implementation form of the mobile terminal 100. In this case, a plurality of display units may be spaced apart from one surface or integrally disposed in the mobile terminal 100, or may be respectively disposed on different surfaces.

The display unit 151 may include a touch sensor that senses a touch on the display unit 151 so that a control command can be input by a touch method. Using this, when a touch is made to the display unit 151, the touch sensor detects the touch, and the controller 180 can be configured to generate a control command corresponding to the touch based on the touch. The content input by the touch method may be a letter or a number, or an instruction or designable menu item in various modes.

On the other hand, the touch sensor is composed of a film having a touch pattern, is disposed between the display (not shown) on the back surface of the window 151a and the window 151a, or a metal wire patterned directly on the back surface of the window 151a. It may be. Alternatively, the touch sensor may be integrally formed with the display. For example, the touch sensor may be disposed on the substrate of the display, or may be provided inside the display.

As such, the display unit 151 may form a touch screen together with a touch sensor, and in this case, the touch screen may function as a user input unit 123 (see FIG. 1A). In some cases, the touch screen may replace at least some functions of the first operation unit 123a.

The first sound output unit 152a may be implemented as a receiver that delivers a call sound to the user's ear, and the second sound output unit 152b is a loud speaker that outputs various alarm sounds or multimedia playback sounds. ).

An acoustic hole for emitting sound generated from the first sound output unit 152a may be formed in the window 151a of the display unit 151. However, the present invention is not limited thereto, and the sound may be configured to be emitted along an assembly gap between structures (for example, a gap between the window 151a and the front case 101). In this case, the appearance of the mobile terminal 100 may be simpler because the holes formed independently for the sound output are not visible or hidden.

The light output unit 154 is configured to output light to notify when an event occurs. Examples of the event include message reception, call signal reception, missed calls, alarm, schedule notification, email reception, and information reception through an application. The controller 180 may control the light output unit 154 so that the output of light is terminated when the user's event confirmation is detected.

The first camera 121a processes an image frame of a still image or video obtained by an image sensor in a shooting mode or a video call mode. The processed image frame may be displayed on the display unit 151, and may be stored in the memory 170.

The first and second operation units 123a and 123b are examples of the user input unit 123 that is operated to receive a command for controlling the operation of the mobile terminal 100, and may also be collectively referred to as a manipulating portion. have. The first and second operation units 123a and 123b may be employed in any manner as long as the user operates the touch, push, scroll, etc. while receiving a tactile feeling. Also, the first and second manipulation units 123a and 123b may be employed in such a way that the user operates without a tactile feeling through a proximity touch, a hovering touch, or the like.

In this figure, the first operation unit 123a is illustrated as a touch key, but the present invention is not limited thereto. For example, the first operation unit 123a may be a mechanical key or a combination of a touch key and a push key.

Contents input by the first and second operation units 123a and 123b may be variously set. For example, the first operation unit 123a receives commands such as a menu, a home key, a cancellation, and a search, and the second operation unit 123b is output from the first or second sound output units 152a and 152b. Commands such as adjusting the volume of the sound and switching to the touch recognition mode of the display unit 151 may be received.

Meanwhile, as another example of the user input unit 123 on the rear surface of the terminal body, a rear input unit (not shown) may be provided. The rear input unit is operated to receive a command for controlling the operation of the mobile terminal 100, and the input content may be variously set. For example, commands such as power on / off, start, end, scroll, etc., control the volume of sound output from the first and second sound output units 152a, 152b, and touch recognition mode of the display unit 151. You can receive commands such as conversion of. The rear input unit may be implemented in a form capable of input by touch input, push input, or a combination thereof.

The rear input unit may be disposed to overlap the front display unit 151 in the thickness direction of the terminal body. For example, when the user holds the terminal body with one hand, the rear input unit may be disposed on the rear upper portion of the terminal body so as to be easily operated using the index finger. However, the present invention is not necessarily limited to this, and the position of the rear input unit may be changed.

When the rear input unit is provided on the rear surface of the terminal body, a new type of user interface using the rear input unit may be implemented. In addition, when the touch screen or rear input unit described above replaces at least some functions of the first operation unit 123a provided on the front surface of the terminal body, when the first operation unit 123a is not disposed on the front surface of the terminal body, The display unit 151 may be configured as a larger screen.

Meanwhile, the mobile terminal 100 may be provided with a fingerprint recognition sensor for recognizing a user's fingerprint, and the controller 180 may use fingerprint information detected through the fingerprint recognition sensor as an authentication means. The fingerprint recognition sensor may be embedded in the display unit 151 or the user input unit 123.

The microphone 122 is configured to receive a user's voice, other sounds, and the like. The microphone 122 may be provided at a plurality of locations and configured to receive stereo sound.

The interface unit 160 is a passage through which the mobile terminal 100 can be connected to an external device. For example, the interface unit 160 is a connection terminal for connection with other devices (eg, earphones, external speakers), a port for short-range communication (for example, an infrared port (IrDA Port), a Bluetooth port (Bluetooth) Port, Wireless LAN Port, etc.], or at least one of a power supply terminal for supplying power to the mobile terminal 100. The interface unit 160 may be implemented in the form of a socket that accommodates an external card such as a subscriber identification module (SIM) or a user identity module (UIM) or a memory card for storing information.

A second camera 121b may be disposed on the rear side of the terminal body. In this case, the second camera 121b has a shooting direction substantially opposite to the first camera 121a.

The second camera 121b may include a plurality of lenses arranged along at least one line. The plurality of lenses may be arranged in a matrix format. Such a camera may be referred to as an 'array camera'. When the second camera 121b is configured as an array camera, images may be photographed in various ways using a plurality of lenses, and images of a better quality may be obtained.

The flash 124 may be disposed adjacent to the second camera 121b. When the flash 124 photographs the subject with the second camera 121b, the light is directed toward the subject.

A second sound output unit 152b may be additionally disposed on the terminal body. The second sound output unit 152b may implement a stereo function together with the first sound output unit 152a, or may be used to implement a speakerphone mode during a call.

The terminal body may be provided with at least one antenna for wireless communication. The antenna may be built in the terminal body or may be formed in the case. For example, the antenna forming part of the broadcast receiving module 111 (see FIG. 1A) may be configured to be pulled out from the terminal body. Alternatively, the antenna may be formed of a film type and attached to the inner surface of the rear cover 103, or a case including a conductive material may be configured to function as an antenna.

The terminal body is provided with a power supply unit 190 (see FIG. 1A) for supplying power to the mobile terminal 100. The power supply unit 190 may include a battery 191 built in the terminal body or configured to be detachable from the outside of the terminal body.

The battery 191 may be configured to receive power through a power cable connected to the interface unit 160. Also, the battery 191 may be configured to be wirelessly charged through a wireless charger. The wireless charging may be implemented by a magnetic induction method or a resonance method (magnetic resonance method).

Meanwhile, in this drawing, the rear cover 103 is coupled to the rear case 102 so as to cover the battery 191 to limit the detachment of the battery 191, and is configured to protect the battery 191 from external shocks and foreign objects. For example. When the battery 191 is configured to be detachably attached to the terminal body, the rear cover 103 may be detachably coupled to the rear case 102.

An accessory that protects the appearance or assists or expands the function of the mobile terminal 100 may be added to the mobile terminal 100. An example of such an accessory is a cover or pouch that covers or accommodates at least one side of the mobile terminal 100. The cover or pouch may be configured to expand the function of the mobile terminal 100 in conjunction with the display unit 151. Another example of an accessory is a touch pen for assisting or extending a touch input to a touch screen.

Meanwhile, in the present invention, information processed in a mobile terminal can be displayed using a flexible display. Hereinafter, this will be described in more detail based on the accompanying drawings.

2 is a conceptual diagram illustrating another example of a deformable mobile terminal 200 according to the present invention.

As illustrated, the display unit 251 may be configured to be deformable by external force. The deformation may be at least one of bending, bending, folding, twisting, and curling of the display unit 251. The deformable display unit 251 may be referred to as a 'flexible display unit'. Here, the flexible display unit 251 may include both a general flexible display, an e-paper, and a combination thereof. In general, the mobile terminal 200 may include features similar to or similar to those of the mobile terminal 100 of FIGS. 1A-1C.

A general flexible display refers to a durable display that is light and easy to break, being made on a thin and flexible substrate that can bend, bend, fold, twist, or curl like paper, while maintaining the characteristics of a conventional flat panel display.

In addition, the electronic paper is a display technology to which general ink characteristics are applied, and the point of using reflected light may be different from a conventional flat panel display. The electronic paper can change information using a twist ball or electrophoresis using a capsule.

In a state in which the flexible display unit 251 is not deformed (for example, a state having an infinite curvature radius, hereinafter referred to as a first state), the display area of the flexible display unit 251 is flat. In the state deformed by an external force in the first state (for example, a state having a finite curvature radius, hereinafter referred to as a second state), the display area may be a curved surface. As illustrated, information displayed in the second state may be visual information output on a curved surface. This visual information is realized by independently controlling the light emission of sub-pixels arranged in a matrix form. The unit pixel refers to a minimum unit for realizing one color.

The flexible display unit 251 may be placed in a curved state (for example, a vertically or horizontally curved state) rather than a flat state in the first state. In this case, when an external force is applied to the flexible display unit 251, the flexible display unit 251 may be transformed into a flat state (or a less curved state) or a more curved state.

Meanwhile, the flexible display unit 251 may be combined with a touch sensor to implement a flexible touch screen. When a touch is made on the flexible touch screen, the controller 180 (see FIG. 1A) may perform control corresponding to the touch input. The flexible touch screen may be configured to sense a touch input in the first state as well as the second state.

Meanwhile, the mobile terminal 200 according to the present modification example may be provided with a deformation detecting means capable of detecting deformation of the flexible display unit 251. The deformation detecting means may be included in the sensing unit 140 (see FIG. 1A).

The deformation detecting means is provided in the flexible display unit 251 or the case 201, and can detect information related to deformation of the flexible display unit 251. Here, the information related to the deformation may be a direction in which the flexible display unit 251 is deformed, a deformed degree, a deformed position, a deformed time, and an acceleration in which the deformed flexible display unit 251 is restored, and the like. In addition, the flexible display unit 251 may be various pieces of information that can be sensed due to bending.

In addition, the controller 180 changes information displayed on the flexible display unit 251 based on information related to the deformation of the flexible display unit 251 sensed by the deformation detecting means, or the mobile terminal 200. It is possible to generate a control signal for controlling the function of.

Meanwhile, the mobile terminal 200 according to the present modification example may include a case 201 accommodating the flexible display unit 251. The case 201 may be configured to be deformable together with the flexible display unit 251 by external force in consideration of the characteristics of the flexible display unit 251.

In addition, a battery (not shown) provided in the mobile terminal 200 may also be configured to be deformable together with the flexible display unit 251 by an external force in consideration of characteristics of the flexible display unit 251. In order to implement the battery, a stack and folding method in which battery cells are stacked up may be applied.

Deformation of the state of the flexible display unit 251 is not limited only to external force. For example, when the flexible display unit 251 has a first state, it may be transformed into a second state by a user or an application command.

On the other hand, the mobile terminal can be extended to a wearable device that can be worn on a body, beyond a dimension that is mainly held and used by a user. Such wearable devices include smartwatches, smart glasses, and head mounted displays (HMDs). Hereinafter, examples of the mobile terminal extended to the wearable device will be described.

The wearable device may be made to exchange (or interlock) data with other mobile terminals 100. The short-range communication module 114 may detect (or recognize) a wearable device capable of communicating around the mobile terminal 100. Furthermore, when the detected wearable device is a device authenticated to communicate with the mobile terminal 100, the controller 180 may wear at least a portion of data processed by the mobile terminal 100 through the short-range communication module 114. Can be transferred to. Therefore, the user can use the data processed by the mobile terminal 100 through the wearable device. For example, it is possible to perform a phone call through a wearable device when a call is received by the mobile terminal 100, or to confirm the received message through a wearable device when a message is received by the mobile terminal 100. .

3 is a perspective view showing an example of a watch-type mobile terminal 300 related to another embodiment of the present invention.

Referring to FIG. 3, the watch-type mobile terminal 300 includes a body 301 having a display unit 351 and a band 302 connected to the body 301 and configured to be worn on a wrist. In general, the mobile terminal 300 may include features similar to or similar to those of the mobile terminal 100 of FIGS. 1A-1C.

The body 301 includes a case forming an exterior. As illustrated, the case may include a first case 301a and a second case 301b that provide an interior space for accommodating various electronic components. However, the present invention is not limited to this, and one case is configured to provide the interior space, so that the unibody mobile terminal 300 may be implemented.

The watch-type mobile terminal 300 is configured to enable wireless communication, and an antenna for the wireless communication may be installed in the main body 301. Meanwhile, the antenna can expand its performance using a case. For example, a case including a conductive material may be configured to electrically connect to the antenna to expand the ground area or the radiation area.

A display unit 351 may be disposed on the front surface of the main body 301 to output information, and a touch sensor may be provided on the display unit 351 to be implemented as a touch screen. As illustrated, the window 351a of the display unit 351 is mounted on the first case 301a to form the front surface of the terminal body together with the first case 301a.

The body 301 may be provided with an audio output unit 352, a camera 321, a microphone 322, a user input unit 323, and the like. When the display unit 351 is implemented as a touch screen, it may function as the user input unit 323, and accordingly, a separate key may not be provided in the main body 301.

The band 302 is worn on the wrist to cover the wrist, and may be formed of a flexible material for easy wearing. As such an example, the band 302 may be formed of leather, rubber, silicone, synthetic resin material, or the like. In addition, the band 302 is configured to be detachably attached to the main body 301, and the user can be configured to be replaceable with various types of bands according to preference.

Meanwhile, the band 302 may be used to expand the performance of the antenna. For example, a ground extension unit (not shown) that is electrically connected to the antenna to expand the ground area may be built in the band.

The band 302 may be provided with a fastener 302a. The fastener 302a may be implemented by a buckle, a hook structure capable of snap-fit, or a velcro (trade name), and may include an elastic section or material. . In this figure, an example in which the fastener 302a is implemented in the form of a buckle is presented.

4 is a perspective view showing an example of a glass-type mobile terminal 400 related to another embodiment of the present invention.

The glass-type mobile terminal 400 is configured to be worn on the head of the human body, and may include a frame part (case, housing, etc.) for this purpose. The frame portion may be formed of a flexible material to facilitate wearing. In this drawing, it is illustrated that the frame portion includes a first frame 401 and a second frame 402 of different materials. In general, the mobile terminal 400 may include features similar to or similar to those of the mobile terminal 100 of FIGS. 1A-1C.

The frame part is supported on the head, and provides a space in which various parts are mounted. As illustrated, electronic components such as a control module 480 and an audio output module 452 may be mounted on the frame unit. Further, a lens 403 covering at least one of the left eye and the right eye may be detachably mounted to the frame portion.

The control module 480 is configured to control various electronic components provided in the mobile terminal 400. The control module 480 may be understood as a configuration corresponding to the controller 180 described above. In this figure, it is illustrated that the control module 480 is installed on the frame portion on one side of the head. However, the position of the control module 480 is not limited thereto.

The display unit 451 may be implemented in the form of a head mounted display (HMD). The HMD type is a display method mounted on the head and displaying an image directly in front of the user's eyes. When the user wears the glass-type mobile terminal 400, the display unit 451 may be disposed to correspond to at least one of the left eye and the right eye so as to directly provide an image in front of the user. In this figure, it is illustrated that the display unit 451 is located in a portion corresponding to the right eye so that an image is output toward the right eye of the user.

The display unit 451 may project an image to the user's eyes using a prism. In addition, the prism may be formed to be translucent so that the user can view the projected image and the general field of view in front (the range that the user sees through the eyes) together.

As such, an image output through the display unit 451 may be viewed overlapping with a general field of view. The mobile terminal 400 may provide augmented reality (AR) by displaying a virtual image on a background or a real image by using the characteristics of the display.

The camera 421 is disposed adjacent to at least one of the left eye and the right eye, and is formed to photograph an image in front. Since the camera 421 is located adjacent to the eye, the camera 421 may acquire a scene viewed by the user as an image.

In this drawing, although the camera 421 is provided in the control module 480, it is not necessarily limited thereto. The camera 421 may be installed in the frame unit, or may be provided in plural to obtain a stereoscopic image.

The glass-type mobile terminal 400 may include user input units 423a and 423b operated to receive a control command. The user input units 423a and 423b may be employed in any manner as long as the user operates with a tactile feeling, such as touch or push. In this figure, it is illustrated that the frame unit and the control module 480 are provided with user input units 423a and 423b of a push and touch input method, respectively.

Also, a glass type mobile terminal 400 may be provided with a microphone (not shown) that receives sound and processes it as electrical voice data, and an audio output module 452 for outputting sound. The sound output module 452 may be configured to transmit sound in a general sound output method or bone conduction method. When the sound output module 452 is implemented in a bone conduction method, when the user wears the mobile terminal 400, the sound output module 452 is in close contact with the head and vibrates the skull to transmit sound.

Next, a communication system that can be implemented through the mobile terminal 100 according to the present invention will be described.

First, the communication system may use different radio interfaces and / or physical layers. For example, in a radio interface usable by a communication system, frequency division multiple access (FDMA), time division multiple access (TDMA), code division multiple access (CDMA) ), Universal Mobile Telecommunications Systems (UMTS) (especially Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A)), Global System for Mobile Communications (GSM), etc. This can be included.

Hereinafter, for convenience of description, the description will be limited to CDMA. However, it is apparent that the present invention can be applied to all communication systems including an orthogonal frequency division multiplexing (OFDM) wireless communication system as well as a CDMA wireless communication system.

CDMA wireless communication system, at least one terminal 100, at least one base station (Base Station, BS (also may be referred to as Node B or Evolved Node B)), at least one base station controller (Base Station Controllers, BSCs) ), A Mobile Switching Center (MSC). The MSC is configured to connect to Public Switched Telephone Networks (PSTNs) and BSCs. BSCs can be connected in pairs with the BS through a backhaul line. The backhaul line may be provided according to at least one of E1 / T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL or xDSL. Accordingly, a plurality of BSCs can be included in the CDMA wireless communication system.

Each of the plurality of BSs may include at least one sector, and each sector may include an omni-directional antenna or an antenna pointing in a particular direction radially from the BS. Further, each sector may include two or more antennas of various types. Each BS may be configured to support multiple frequency allocations, and the multiple frequency allocations may each have a specific spectrum (eg, 1.25 MHz, 5 MHz, etc.).

The intersection of sector and frequency allocation can be referred to as a CDMA channel. The BS may be called Base Station Transceiver Subsystems (BTSs). In this case, one BSC and at least one BS may be collectively referred to as a “base station”. The base station may also indicate "cell site". Alternatively, each of the plurality of sectors for a particular BS may be referred to as a plurality of cell sites.

The broadcasting transmitter (BT) transmits a broadcast signal to terminals 100 operating in the system. The broadcast receiving module 111 shown in FIG. 1A is provided in the terminal 100 to receive a broadcast signal transmitted by BT.

In addition, a satellite positioning system (Global Positioning System, GPS) for confirming the location of the mobile terminal 100 may be connected to the CDMA wireless communication system. The satellite 300 helps to locate the mobile terminal 100. Useful location information may be obtained by two or more satellites. Here, the location of the mobile terminal 100 may be tracked using all techniques capable of tracking a location, as well as GPS tracking technology. In addition, at least one of the GPS satellites may selectively or additionally be responsible for satellite DMB transmission.

The location information module 115 provided in the mobile terminal is for detecting, calculating, or identifying the location of the mobile terminal, and examples include a Global Position System (GPS) module and a Wireless Fidelity (WiFi) module. If necessary, the location information module 115 may perform any function of other modules of the wireless communication unit 110 in order to obtain data regarding the location of the mobile terminal by substitution or additionally.

The GPS module 115 calculates distance information and accurate time information from three or more satellites, and then applies trigonometry to the calculated information to accurately calculate three-dimensional current location information according to latitude, longitude, and altitude. can do. Currently, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another one satellite is widely used. In addition, the GPS module 115 may calculate speed information by continuously calculating the current position in real time. However, it is difficult to accurately measure the position of the mobile terminal in the shaded area of the satellite signal, such as indoors, using a GPS module. Accordingly, a WPS (WiFi Positioning System) may be used to compensate for GPS positioning.

The WiFi Positioning System (WPS) is a mobile terminal 100 using a WiFi module provided in the mobile terminal 100 and a wireless AP (Wireless Access Point) that transmits or receives a wireless signal with the WiFi module. As a technology for tracking the location of, it means a location localization technology based on a wireless local area network (WLAN) using WiFi.

The Wi-Fi location tracking system may include a Wi-Fi location server, a mobile terminal 100, a wireless AP connected to the mobile terminal 100, and a database in which arbitrary AP information is stored.

The mobile terminal 100 accessing the wireless AP may transmit a location information request message to a Wi-Fi location location server.

The Wi-Fi location server extracts information of a wireless AP connected to the mobile terminal 100 based on the location information request message (or signal) of the mobile terminal 100. Information of the wireless AP connected to the mobile terminal 100 may be transmitted to the Wi-Fi location server through the mobile terminal 100, or may be transmitted from the wireless AP to the Wi-Fi location server.

Based on the location information request message of the mobile terminal 100, extracted wireless AP information includes MAC Address, SSID (Service Set IDentification), RSSI (Received Signal Strength Indicator), RSRP (Reference Signal Received Power), RSRQ ( Reference Signal Received Quality), channel information, Privacy, Network Type, signal strength (Signal Strength) and noise strength (Noise Strength).

As described above, the Wi-Fi location server may receive information of the wireless AP connected to the mobile terminal 100 and extract wireless AP information corresponding to the wireless AP being accessed by the mobile terminal from a pre-built database. At this time, the information of arbitrary wireless APs stored in the database includes MAC address, SSID, channel information, privacy, network type, latitude and longitude coordinates of the wireless AP, building name where the wireless AP is located, floor number, and indoor detailed location information (GPS coordinates) Available), AP owner's address, and phone number. In this case, in order to remove a wireless AP provided using a mobile AP or an illegal MAC address in the positioning process, the Wi-Fi location server may extract only a predetermined number of wireless AP information in order of high RSSI.

Thereafter, the Wi-Fi location server may extract (or analyze) location information of the mobile terminal 100 using at least one wireless AP information extracted from the database. The included information is compared with the received wireless AP information to extract (or analyze) location information of the mobile terminal 100.

As a method for extracting (or analyzing) location information of the mobile terminal 100, a Cell-ID method, a finger print method, a triangulation method, and a landmark method may be used.

The Cell-ID method is a method of determining the location of the wireless AP having the strongest signal strength among the surrounding wireless AP information collected by the mobile terminal as the location of the mobile terminal. It has the advantage of simple implementation, no extra cost, and the ability to quickly obtain location information, but has the disadvantage of low positioning accuracy when the installation density of the wireless AP is low.

The fingerprint method is a method of selecting a reference location in a service area, collecting signal strength information, and estimating a location through signal strength information transmitted from a mobile terminal based on the collected information. In order to use the fingerprint method, it is necessary to database the propagation characteristics in advance.

The triangulation method is a method of calculating the position of the mobile terminal based on the coordinates of at least three wireless APs and the distance between the mobile terminals. To measure the distance between the mobile terminal and the wireless AP, the signal strength is converted to distance information, or the time at which the wireless signal is transmitted (Time of Arrival, ToA), and the time at which the signal is transmitted (Time Difference of Arrival, TDoA) , The angle at which the signal is transmitted (Angle of Arrival, AoA) may be used.

The landmark method is a method of measuring the location of a mobile terminal using a landmark transmitter that knows its location.

In addition to the listed methods, various algorithms may be used as a method for extracting (or analyzing) location information of a mobile terminal.

The extracted location information of the mobile terminal 100 is transmitted to the mobile terminal 100 through the Wi-Fi location location server, so that the mobile terminal 100 can obtain location information.

The mobile terminal 100 may acquire location information by being connected to at least one wireless AP. At this time, the number of wireless APs required to obtain location information of the mobile terminal 100 may be variously changed according to the wireless communication environment in which the mobile terminal 100 is located.

1A, the mobile terminal according to the present invention includes Bluetooth (Bluetooth ™), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, NFC (Near Field Communication), Wireless USB (Wireless Universal Serial Bus), and other short-range communication technologies can be applied.

Among them, the NFC module provided in the mobile terminal supports contactless short-range wireless communication between terminals at a distance of about 10 cm. The NFC module can operate in any one of card mode, reader mode and P2P mode. In order for the NFC module to operate in the card mode, the mobile terminal 100 may further include a security module that stores card information. Here, the security module may be a physical medium such as a Universal Integrated Circuit Card (UICC) (for example, a Subscriber Identification Module (SIM) or a Universal SIM (USIM)), a Secure micro SD, and a sticker, or a logical medium embedded in the mobile terminal ( For example, it may be an embedded secure element (SE). Data exchange based on a SWP (Single Wire Protocol) may be performed between the NFC module and the security module.

When the NFC module is operated in the card mode, the mobile terminal can transmit the stored card information to the outside like a traditional IC card. Specifically, when a mobile terminal storing card information of a payment card such as a credit card or a bus card is brought close to a toll machine, mobile short-range payment can be processed, and the mobile terminal storing card information of the access card is granted access. When approaching the flag, the process of approval for entry and exit may begin. Cards such as credit cards, transportation cards, and access cards are mounted in the security module in the form of an applet, and the security module can store card information for the loaded card. Here, the card information of the payment card may be at least one of a card number, a balance, and a usage history, and the card information of the access card may include at least one of a user's name, number (for example, a user's student number or four), and access history. It can be one.

When the NFC module is operated in a reader mode, the mobile terminal can read data from an external tag. At this time, the data received by the mobile terminal from the tag may be coded in a data exchange format (NFC Data Exchange Format) determined by the NFC forum. In addition, the NFC Forum defines four record types. Specifically, the NFC Forum defines four RTDs (Record Type Definition), such as Smart Poster, Text, Uniform Resource Identifier (URI), and General Control. When the data received from the tag is a smart poster type, the control unit executes a browser (eg, an internet browser), and when the data received from the tag is text type, the control unit may execute a text viewer. When the data received from the tag is a URI type, the controller may execute a browser or make a call, and when the data received from the tag is a general control type, an appropriate operation may be performed according to the control content.

When the NFC module is operated in a peer-to-peer (P2P) mode, the mobile terminal can perform P2P communication with other mobile terminals. At this time, Logical Link Control Protocol (LLCP) may be applied to P2P communication. For P2P communication, a connection may be created between a mobile terminal and another mobile terminal. In this case, the generated connection may be divided into a connectionless mode in which one packet is exchanged and terminated, and a connection-oriented mode in which packets are continuously exchanged. Through P2P communication, data such as electronic business cards, contact information, digital photos, URLs, and setup parameters for Bluetooth and Wi-Fi connection can be exchanged. However, since the available distance of NFC communication is short, the P2P mode may be effectively used for exchanging small data.

Hereinafter, embodiments related to a control method that can be implemented in the mobile terminal configured as described above will be described with reference to the accompanying drawings. It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit and essential features of the present invention.

5 is a flowchart illustrating a method of operating a mobile terminal according to an embodiment of the present invention.

The microphone 122 of the mobile terminal 100 receives the user's first voice command (S501).

The controller 180 determines whether the recognition of the received first voice command has failed (S503).

In one embodiment, the controller 180 may determine whether the first voice pattern corresponding to the received first voice command is stored in the memory 170.

In one embodiment, the voice pattern may be a signal pattern corresponding to a voice command.

When the first voice pattern corresponding to the first voice command and a voice pattern matching a predetermined reference value or more are stored in the memory 170, the controller 180 may determine that the recognition of the first voice command is successful.

The controller 180 may determine that the recognition of the first voice command has failed when the first voice pattern corresponding to the first voice command and a voice pattern matching a predetermined reference value or more are not stored in the memory 170.

In one embodiment, when the degree of matching between the first voice pattern and the voice pattern stored in the memory 170 is greater than or equal to a predetermined reference value, the controller 180 may determine that the recognition of the first voice command is successful.

If the recognition of the first voice command is successful, the controller 180 performs a function corresponding to the first voice command (505).

The memory 170 may store a function corresponding to a voice pattern.

The memory 170 may store functions corresponding to each of a plurality of voice patterns.

For example, the function may be a specific function that can be performed in the mobile terminal 100, such as power on / off, opening a web browser, or making a call.

If the recognition of the first voice command fails, the controller 180 stores the first voice pattern of the first voice command in the memory 170 (S507).

In one embodiment, the controller 180 may store the first voice pattern in order to increase the voice recognition rate during speech recognition training, due to pronunciation, intonation, and speed out of the assumed range.

Thereafter, the microphone 122 receives the second voice command (S509), and the controller 180 determines whether the second voice pattern of the received second voice command is similar to the first voice pattern (S511).

When the degree of similarity between the first voice pattern and the second voice pattern is equal to or greater than the reference similarity, the controller 180 may determine that the second voice pattern is similar to the first voice pattern.

When the similarity between the first voice pattern and the second voice pattern is less than the reference similarity, the controller 180 may determine that the second voice pattern is not similar to the first voice pattern.

If it is determined that the second voice pattern and the first voice pattern are similar, the controller 180 adjusts the voice recognition threshold (S513).

In one embodiment, the speech recognition threshold may be a value indicating confidence in the speech recognition rate. The speech recognition threshold may indicate the ability of speech recognition.

The higher the voice recognition threshold is, the more accurate pronunciation and intonation of voice commands must be received before the voice commands can be recognized.

As the voice recognition threshold is lowered, voice commands can be recognized even if relatively, less accurate pronunciation and intonation voice commands are received.

The default speech recognition threshold may be 0.9, but this is only an example.

When it is determined that the second voice pattern and the first unrecognized voice pattern are similar, the controller 180 may lower the voice recognition threshold to increase the recognition rate of the voice command.

Lowering the speech recognition threshold may be determined to lower the standard similarity for determining the degree of matching between speech patterns.

Thereafter, the controller 180 determines whether the second voice command is recognized based on the adjusted voice recognition threshold (S515).

When the similarity between the second voice pattern of the second voice command and the voice pattern stored in the memory 170 is equal to or greater than the changed reference similarity according to a decrease in the voice recognition threshold, the controller 180 determines that the recognition of the second voice command is successful. can do.

If the similarity between the second voice pattern of the second voice command and the voice pattern stored in the memory 170 is less than the changed reference similarity, it may be determined that the recognition of the second voice command has failed. In this case, the controller 180 has failed to recognize the corresponding voice command, and may output a notification to utter the voice command again.

If the second voice command is recognized, the controller 180 transmits the first voice pattern and the adjusted voice recognition threshold to the server (S517).

The controller 180 may transmit the first voice pattern or the second voice pattern and the adjusted voice recognition threshold to the server.

The server may retrain the user-specific speech recognition model using the received information. For example, the server may match and store the adjusted voice recognition threshold and the first voice pattern to the corresponding user.

6 is a flowchart illustrating a method of operating a mobile terminal according to another embodiment of the present invention.

Referring to FIG. 6, the controller 180 continuously acquires a voice pattern of a voice command (S601).

The controller 180 determines whether a repeated voice pattern has been searched (S603).

In one embodiment, if the similarity between the first voice pattern of the first voice command and the second voice pattern of the second voice command received through the microphone 122 is greater than or equal to the reference similarity, the control unit 180 may repeat the voice pattern. It can be judged as being discovered.

In one embodiment, the time point of acquiring the first voice pattern and the time point of acquiring the second voice pattern may be within a certain time period. The period of time may be 5 seconds, but this is only an example.

The controller 180 counts the number of times the repeated voice pattern is searched (S605).

The controller 180 can count the number of times the repeated voice pattern is searched in real time.

The controller 180 determines the priority according to the counting number and transmits it to the server (S607).

In one embodiment, the controller 180 may give a higher priority to the corresponding voice pattern as the counting number increases.

For example, when the counting number is 1, the controller 180 may set the priority to 2, and when the counting number is 2, the priority may be set to 1.

The controller 180 may transmit information about the repeated voice pattern, the counting number of voice patterns, and priority through the wireless communication unit 110 to the server.

The server may perform voice recognition retraining based on the information received from the mobile terminal 100.

7 is a flowchart illustrating a method of operating a mobile terminal according to another embodiment of the present invention.

Referring to FIG. 7, the controller 180 continuously acquires a voice pattern of a voice command (S701).

The controller 180 determines whether a repeated voice pattern has been searched (S703).

The controller 180 counts the number of times the repeated voice pattern is searched (S705).

The controller 180 adjusts the voice recognition threshold according to the counting number (S707).

In one embodiment, the controller 180 may lower the voice recognition threshold value as the counting number increases.

It is assumed that the initial speech recognition threshold is 0.9.

When the number of repetitions of the voice pattern is 1, the controller 180 may change the voice recognition threshold from 0.9 to 0.8.

When the number of repetitions of the voice pattern is 2, the controller 180 may change the voice recognition threshold from 0.8 to 0.7.

If the number of repetitions of the voice pattern is 0, the controller 180 may re-adjust the voice recognition threshold to 0.9, which is the initial voice recognition threshold.

Generally, when speech recognition fails, the recognition request is repeated. Therefore, according to an embodiment of the present invention, when a similar pattern is retried, the voice recognition threshold may be lowered to a specific value, thereby improving the voice recognition rate.

8 is a diagram for explaining a process of performing voice recognition re-training in a server according to an embodiment of the present invention.

The server 800 may receive failure data for which speech recognition has failed from each of the plurality of mobile terminals.

The server 800 may cluster (group) failure data based on a plurality of classification items.

The reason speech recognition fails is because the characteristics of the speech may vary depending on the specific region, gender, and specific household.

Accordingly, the server 800 may group failure data according to a plurality of classification items, that is, a specific region, gender, and specific household.

The server 800 may generate a retrained speech recognition model according to the grouping result. The retrained speech recognition model may be a process of setting a speech recognition threshold for a specific speech pattern.

That is, the server 800 may set a voice recognition threshold of each group classified according to the grouping result.

For example, the server 800 sets the first voice recognition threshold for the voice pattern corresponding to the first classification item, sets the second voice recognition threshold for the voice pattern corresponding to the second classification item, and the third classification A third voice recognition threshold may be set in the voice pattern corresponding to the item.

That is, the server 800 may set different voice recognition thresholds according to classification items for the same voice pattern.

The server 800 may transmit the set voice recognition threshold to the corresponding terminal in response to the voice pattern and the voice pattern.

9 is a view for explaining cases in which the recognition of a conventional voice activated word fails.

Compared to the pattern of the general spoken words, the user may speak the spoken words slowly or quickly, and speech recognition may fail.

In addition, if a unique accent is given according to a user to utter a spoken word, speech recognition may fail.

To solve this case, according to various embodiments of FIGS. 5 to 8, failed cases of speech recognition are automatically labeled to increase efficiency of retraining.

The existing method could only retrain through a cloud server through manual labeling. However, when applying the method according to an embodiment of the present invention, there is an effect that retraining can be quickly performed.

In addition, the voice recognition threshold may be lowered according to a failure to recognize the repeated voice pattern, thereby increasing the voice recognition rate.

If the voice recognition threshold is lowered, the probability of malfunction of voice recognition may increase. According to an embodiment of the present invention, when necessary, that is, only when a speech pattern that fails to recognize is repeated, the speech recognition threshold is adjusted, so that recognition performance may be increased.

The present invention described above can be embodied as computer readable codes on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data readable by a computer system is stored. Examples of computer-readable media include a hard disk drive (HDD), solid state disk (SSD), silicon disk drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. There is this. In addition, the computer may include a control unit 180 of the terminal.

Accordingly, the above detailed description should not be construed as limiting in all respects, but should be considered illustrative. The scope of the invention should be determined by rational interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

Claims (8)

In the mobile terminal,
A communication unit for communication with the server;
A microphone that receives a user's voice command;
Memory; And
It includes a control unit for controlling the microphone and the memory,
The control unit
If a first voice command is received and recognition of the received first voice command fails, the first voice pattern of the first voice command is stored in the memory,
If a second voice command is received, and the second voice pattern of the second voice command is a pattern similar to the first voice pattern, the voice recognition threshold indicating the capability of voice recognition is adjusted,
When the second voice command is recognized according to the adjustment of the voice recognition threshold, transmit the first voice pattern and the adjusted voice recognition threshold stored in the memory to the server,
The control unit
Based on the number of repetitions of speech patterns for which speech recognition has failed, priority is given to each of the speech patterns, and information on the given priority is transmitted to the server through the communication unit.
Mobile terminal. According to claim 1,
The control unit
When the similarity between the second speech patterns is equal to or greater than a predetermined similarity, the threshold of the speech recognition is lowered.
Mobile terminal. According to claim 2,
The control unit
Decreasing the voice recognition threshold according to the number of repetitions of a voice pattern in which voice recognition has failed
Mobile terminal. According to claim 1,
The control unit
As the number of repetitions of the speech pattern increases, the rate of reduction of the speech recognition threshold increases.
Mobile terminal. According to claim 1,
The speech recognition threshold
Represented by similarities, similar degrees between speech patterns
Mobile terminal. According to claim 3,
The control unit
The voice pattern for which speech recognition has failed, and the voice recognition threshold adjusted according to the number of repetitions and the number of repetitions of the voice pattern are transmitted to the server
Mobile terminal. delete According to claim 1,
The control unit
As the number of repetitions increases, a higher priority is given to the corresponding voice pattern.
Mobile terminal.

Images