KR20200011796A - Voice recognition system - Google Patents

Abstract

Disclosed is a voice recognition system capable of learning a voice of a user. According to an embodiment of the present invention, the voice recognition system comprises: a voice recognition agent receiving voice data from a user and transmitting the voice data to an artificial intelligence server; and an artificial intelligence server inputting the voice data into a voice recognition model, transmitting a recognition result for the voice data to the voice recognition agent, and learning the voice data. The voice recognition agent requests the user for additional data to learn the voice of the user when the voice recognition rate for the voice data is lower than a previously set criterion.

Description

Speech Recognition System {VOICE RECOGNITION SYSTEM}

The present invention relates to a speech recognition system that enables a user to directly participate in learning a speech recognition model to secure speech data or text and to learn the user's speech with the obtained data.

 Artificial intelligence is a branch of computer science and information technology that studies how to enable computers to do things like thinking, learning, and self-development that human intelligence can do. It means to be able to imitate.

In addition, artificial intelligence does not exist by itself, but is directly or indirectly related to other fields of computer science. Particularly in modern times, attempts are being actively made to introduce artificial intelligence elements in various fields of information technology and use them to solve problems in those fields.

On the other hand, in the related art, the situation awareness (Context Awareness) technology for recognizing a user's situation using the artificial intelligence and providing the desired information in a desired form has been actively studied.

As the situation-aware technology is developed as described above, there is an increasing demand for a system capable of performing a function suitable for a user's situation.

On the other hand, the voice recognition system that combines the user's voice recognition and situational awareness technology, to provide various operations and functions to the user through the voice recognition is increasing.

Voice recognition refers to converting a speech signal into a string or identifying linguistic semantic content by interpreting the speech signal and combining it with a patterned database.

The speech recognition technology analyzes the received speech data by the speech recognition model, extracts features, and measures similarity with the previously collected speech model database to convert the most similar text or commands.

Speech recognition technology is a kind of pattern recognition process. Since voices, pronunciation, and intonation differ from person to person, the conventional voice recognition technology collects voice data from as many people as possible, extracts common features, and generates reference patterns.

However, since the learning model is composed of the training data created in the laboratory environment, the reference pattern is not optimized for the voice or tone of the real user.

Therefore, additional adaptive learning is needed so that the speech recognition model is personalized to the user who directly uses the speech recognition apparatus.

Accordingly, the present invention proposes a method for improving the accuracy and efficiency of adaptive learning.

SUMMARY OF THE INVENTION The present invention relates to a speech recognition system that enables a user to directly participate in learning a speech recognition model to secure speech data or text data and to learn the user's speech with the obtained data.

A voice recognition system according to an embodiment of the present invention includes a voice recognition agent for receiving voice data from a user, transmitting the voice data to an artificial intelligence server, and inputting the voice data into a voice recognition model, And an artificial intelligence server for transmitting the recognition result to the voice recognition agent and learning the voice data, wherein the voice recognition agent includes: when the voice recognition rate of the voice data is lower than a preset criterion; Ask the user for additional data to learn the voice.

In this case, the voice recognition agent provides a specific sentence to the user, and when the second voice data corresponding to the specific sentence is received, transmits the second voice data to the artificial intelligence server, and the artificial intelligence server, The second voice data corresponding to the specific sentence may be learned.

In this case, the artificial intelligence server may transmit the specific sentence corresponding to the characteristic of the speech data among the plurality of sentences to the speech recognition agent based on the characteristic of the speech data.

In this case, the plurality of sentences are classified into categories including at least one of a product function, a country, a region, an age, a dialect, a gender, and a foreign language, and the artificial intelligence server is configured to generate a plurality of sentences based on the characteristics of the voice data. The specific sentence included in a category requiring additional learning by the user among categories may be transmitted to the voice recognition agent.

The specific sentence may include a command corresponding to a function of the speech recognition agent.

The voice recognition system may further include a mobile terminal, and the voice recognition agent may transmit the specific sentence to the mobile terminal of the user, and the mobile terminal may display text corresponding to the specific sentence. have.

Meanwhile, when the speech recognition rate is lower than a preset criterion, the speech recognition agent may request the user to input text corresponding to the speech data.

In this case, the artificial intelligence server stores the voice data, and when the text corresponding to the voice data is input, the artificial intelligence server transmits the text corresponding to the voice data to the artificial intelligence server, and the artificial intelligence The server may learn the stored voice data corresponding to the text.

In this case, the artificial intelligence server converts the text into speech data, determines the stored speech data as valid data based on the similarity between the converted speech data and the stored speech data, and determines the speech data as the valid data. Can learn.

The voice recognition system may further include a mobile terminal that receives an input of a text corresponding to the voice data and transmits a text corresponding to the voice data to the voice recognition agent.

Meanwhile, when the user inputs specific text and third voice data corresponding to the specific text, the voice recognition agent transmits the specific text and third voice data corresponding to the specific text to the artificial intelligence server, The artificial intelligence server may learn the third voice data corresponding to the specific text.

The voice recognition agent may provide a first option of speaking according to the presented voice, a second option of speaking according to the presented sentence, and a third option of directly writing and speaking along the sentence, wherein the speech recognition rate of the first to third options may be The highest option may be to request the additional data.

The artificial intelligence server may learn the additional data and transmit the changed speech recognition rate to the speech recognition agent according to a result of learning the additional data.

On the other hand, the voice recognition apparatus according to an embodiment of the present invention, the input unit for receiving voice data from a user, and inputs the voice data to a voice recognition model, obtains the recognition result for the voice data, And an artificial intelligence unit for learning, wherein the artificial intelligence unit may request additional data for learning the user's voice when the speech recognition rate of the voice data is lower than a preset criterion.

According to the present invention, unlike the conventional method of manually collecting and learning a user's voice, the present invention presents a sentence that best grasps a user's speech habit, requests a voice input, or directly inputs a sentence spoken by the user as text. request. Therefore, according to the present invention, it is possible to greatly improve the learning performance, there is an advantage capable of fast personalization.

1 is a view for explaining a speech recognition system according to an embodiment of the present invention.
2 is a block diagram illustrating a speech recognition agent related to the present invention.
3 is a block diagram showing the configuration of the artificial intelligence server 200 according to an embodiment of the present invention.
4 is a diagram illustrating a problem that may occur in the speech recognition system.
5 is a diagram for describing a method of requesting additional data for further learning from a user according to an exemplary embodiment of the present invention.
FIG. 6 is a diagram for describing an operating method when option 1 or option 2 is selected according to an embodiment of the present disclosure.
7 is a diagram illustrating a word unit recognition rate of a spoken sentence.
8 is a view for explaining an operation when the option 1 is selected.
9 is a view for explaining an operation when the option 2 is selected.
FIG. 10 is a diagram for describing an operation when option 3 is selected.
FIG. 11 is a diagram for describing a method of requesting additional data for additional learning from a user according to another exemplary embodiment.
12 is a diagram for describing an operation when a text input request is made.
13 is a diagram for describing an operation of a voice recognition system according to an exemplary embodiment of the present invention.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, and the same or similar components will be given the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or mixed in consideration of ease of specification, and do not have distinct meanings or roles. In addition, in the following description of the embodiments disclosed herein, when it is determined that the detailed description of the related known technology may obscure the gist of the embodiments disclosed herein, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easily understanding the embodiments disclosed in the present specification, the technical idea disclosed in the specification by the accompanying drawings are not limited, and all changes included in the spirit and scope of the present invention. It should be understood to include equivalents and substitutes.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

When a component is said to be "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that another component may be present in the middle. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprises" or "having" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

The mobile terminal described herein includes a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant, a portable multimedia player, a navigation, a slate PC , Tablet PCs, ultrabooks, wearable devices, such as smartwatches, glass glasses, head mounted displays, and the like. have.

1 is a view for explaining a speech recognition system according to an embodiment of the present invention.

The voice recognition system 10 according to an exemplary embodiment of the present invention may include a voice recognition agent 100, an artificial intelligence server 200, and a mobile terminal 300.

The voice recognition agent 100 may communicate with the artificial intelligence server 200. In more detail, the voice recognition agent 100 may provide an interface for connecting the voice recognition agent 100 with a wired / wireless network including an internet network. The voice recognition agent 100 may transmit or receive data with the server through the connected network or another network linked to the connected network.

In addition, the voice recognition agent 100 may communicate with the mobile terminal 300. In more detail, the voice recognition agent 100 may provide an interface for connecting the voice recognition agent 100 with a wired / wireless network including an internet network. The voice recognition agent 100 may transmit or receive data with the mobile terminal 300 through the connected network or another network linked to the connected network.

In addition, the voice recognition agent 100 may communicate with the mobile terminal 300 through the short range communication described with reference to FIG. 2.

Meanwhile, the voice recognition agent 100 may learn voice data in various ways or perform a function corresponding to the voice data.

For example, when the speech recognition model is mounted on the artificial intelligence server 200, and the speech recognition agent 100 receives the speech data and transmits the speech data to the artificial intelligence server 200, the artificial intelligence server 200 learns the speech data. Alternatively, the recognition result of the voice data may be output and transmitted to the voice recognition agent 100, and the voice recognition agent 100 may be implemented by generating a control command corresponding to the recognition result and performing control.

In another example, when the voice recognition model is mounted on the artificial intelligence server 200, and the voice recognition agent 100 receives the voice data and transmits the voice data to the artificial intelligence server 200, the artificial intelligence server 200 transmits the voice data. Learning or outputting a recognition result for the voice data, and may be implemented by transmitting a control command corresponding to the recognition result to the voice recognition agent 100.

For example, a recognition model is mounted on the voice recognition agent 100, and the voice recognition agent 100 receives voice data to learn voice data or outputs a recognition result for the voice data to the artificial intelligence server 200. And transmits a control command corresponding to the recognition result from the artificial intelligence server 200 to the voice recognition agent 100.

In addition, the speech recognition agent 100 may perform an artificial intelligence function independently of the artificial intelligence server 200.

For example, a voice recognition model is mounted on the voice recognition agent 100, and the voice recognition agent 100 receives voice data to learn voice data or to output voice recognition results, and controls corresponding to the recognition result. You can create a command.

2 is a block diagram illustrating a speech recognition agent related to the present invention.

The voice recognition agent 100 may include a wireless communication unit 110, an input unit 120, an artificial intelligence unit 130, a detection unit 140, an output unit 150, an interface unit 160, a memory 170, and a control unit ( 180 and the power supply unit 190 may be included.

The components shown in FIG. 2 are not essential to implementing a speech recognition agent, so the speech recognition agent described herein may have more or fewer components than the components listed above.

More specifically, the wireless communication unit 110 of the components, between the voice recognition agent 100 and the wireless communication system, between the voice recognition agent 100 and another voice recognition agent 100, or the voice recognition agent 100 ) And one or more modules that enable wireless communication between the external server and the external server. In addition, the wireless communication unit 110 may include one or more modules for connecting the voice recognition agent 100 to one or more networks.

The wireless communication unit 110 may include at least one of the broadcast receiving module 111, the mobile communication module 112, the wireless internet module 113, the short range communication module 114, and the location information module 115. .

The input unit 120 may include a camera 121 or an image input unit for inputting an image signal, a microphone 122 for inputting an audio signal, an audio input unit, or a user input unit 123 for receiving information from a user. , Touch keys, mechanical keys, and the like. The voice data or the image data collected by the input unit 120 may be analyzed and processed as a control command of the user.

The artificial intelligence unit 130 performs a role of processing information based on artificial intelligence technology, and includes one or more modules that perform at least one of learning information, inferring information, perceiving information, and processing natural language. It may include.

The artificial intelligence unit 130 uses machine learning technology to generate a large amount of information (big data, big data, etc.) stored in the voice recognition agent, environment information around the voice recognition agent, and information stored in an external storage that can be communicated with. at least one of data, learning, inference, and processing. The artificial intelligence unit 130 predicts (or infers) an operation of at least one voice recognition agent executable by using the information learned using the machine learning technique, and among the at least one predicted operations. The voice recognition agent can be controlled to execute the most feasible operation.

Machine learning technology is a technology that collects and learns a large amount of information based on at least one algorithm, and determines and predicts information based on the learned information. Learning information is an operation of grasping characteristics, rules, and judgment criteria of information, quantifying the relationship between information, and predicting new data using the quantized pattern.

The algorithms used by these machine learning techniques can be algorithms based on statistics, for example, decision trees using tree structures as predictive models, artificial trees that mimic the neural network structure and function of organisms. Neural networks, genetic programming based on living evolutionary algorithms, clustering that distributes observed examples into subsets of clusters, and Monte Carlo, which randomly computes function values through randomized random numbers Monte carlo method.

As a field of machine learning technology, deep learning technology is a technology that performs at least one of learning, determining, and processing information by using an artificial neural network algorithm. The artificial neural network may have a structure that connects layers to layers and transfers data between layers. Such deep learning technology can learn a huge amount of information through an artificial neural network using a graphic processing unit (GPU) optimized for parallel computing.

Meanwhile, the artificial intelligence unit 130 collects signals, data, information, etc. input or output from the components of the voice recognition agent in order to collect a large amount of information for applying the machine learning technology (detection, monitoring, Extraction, detection, reception). In addition, the artificial intelligence unit 130 may collect (detect, monitor, extract, detect, receive) data, information, and the like stored in an external storage (eg, a cloud server) connected through communication. More specifically, the collection of information may be understood as a term including an operation of sensing information through a sensor, extracting information stored in the memory 170, or receiving information from an external storage through communication.

The artificial intelligence unit 130 may detect information in the voice recognition agent, surrounding environment information surrounding the voice recognition agent, and user information through the sensing unit 140. In addition, the artificial intelligence unit 130 may receive a broadcast signal and / or broadcast related information, a wireless signal, wireless data, and the like through the wireless communication unit 110. In addition, the artificial intelligence unit 130 may receive image information (or signal), audio information (or signal), data, or information input from a user from the input unit.

The artificial intelligence unit 130 collects a large amount of information in real time on the background, learns it, and stores the processed information (for example, knowledge graph, command policy, personalization database, conversation engine, etc.) in an appropriate form. Can be stored at 170.

And, based on the information learned using the machine learning technology, the artificial intelligence unit 130, if the operation of the speech recognition agent is predicted, to execute the predicted operation, to control the components of the speech recognition agent, The control command for executing the predicted operation may be transmitted to the controller 180. The controller 180 may execute the predicted operation by controlling the voice recognition agent based on the control command.

Meanwhile, when a specific operation is performed, the artificial intelligence unit 130 analyzes historical information indicating performance of a specific operation through machine learning technology, and updates the previously learned information based on the analysis information. Can be. Thus, the artificial intelligence unit 130 may improve the accuracy of the information prediction.

Meanwhile, in the present specification, the artificial intelligence unit 130 and the controller 180 may be understood as the same component. In this case, a function performed by the controller 180 described herein may be expressed as being performed by the artificial intelligence unit 130, and the controller 180 may be named as the artificial intelligence unit 130 or vice versa. The intelligent unit 130 may be referred to as the controller 180.

Alternatively, in the present specification, the artificial intelligence unit 130 and the controller 180 may be understood as separate components. In this case, the artificial intelligence unit 130 and the controller 180 may perform various controls on the voice recognition agent through data exchange with each other. The controller 180 may perform at least one function on the voice recognition agent or control at least one of the components of the voice recognition agent based on the result derived from the artificial intelligence unit 130. Furthermore, the artificial intelligence unit 130 may also be operated under the control of the controller 180.

The sensing unit 140 may include one or more sensors for sensing at least one of information in the voice recognition agent, surrounding environment information surrounding the voice recognition agent, and user information.

For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, an illumination sensor, a touch sensor, an acceleration sensor, a magnetic sensor, and gravity. G-sensor, Gyroscope Sensor, Motion Sensor, RGB Sensor, Infrared Sensor, Infrared Sensor, Finger Scan Sensor, Ultrasonic Sensor Optical sensors (e.g. cameras 121), microphones (see 122), battery gauges, environmental sensors (e.g. barometers, hygrometers, thermometers, radiation detection sensors, Thermal sensors, gas sensors, etc.), chemical sensors (eg, electronic noses, healthcare sensors, biometric sensors, etc.). Meanwhile, the voice recognition agent disclosed in the present specification may use a combination of information sensed by at least two or more of these sensors.

The output unit 150 is used to generate an output related to sight, hearing, or tactile sense, and includes at least one of a display unit 151, an audio output unit 152, a hap tip module 153, and an optical output unit 154. can do. The display unit 151 forms a layer structure with or is integrally formed with the touch sensor, thereby implementing a touch screen. The touch screen may function as a user input unit 123 that provides an input interface between the voice recognition agent 100 and the user, and may also provide an output interface between the voice recognition agent 100 and the user.

The interface unit 160 serves as a path to various types of external devices connected to the voice recognition agent 100. The interface unit 160 connects a device equipped with a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, and an identification module. It may include at least one of a port, an audio input / output (I / O) port, a video input / output (I / O) port, and an earphone port. In response to the external device being connected to the interface unit 160, the voice recognition agent 100 may perform appropriate control related to the connected external device.

In addition, the memory 170 stores data supporting various functions of the voice recognition agent 100. Memory 170 is a plurality of application programs (application program or application) running in the voice recognition agent 100, data for the operation of the voice recognition agent 100, instructions, the artificial intelligence unit 130 Data for operation of (eg, at least one algorithm information for machine learning, etc.). At least some of these applications may be downloaded from an external server through wireless communication. In addition, at least some of these applications may exist on the voice recognition agent 100 from the time of shipment for the basic functions (eg, call forwarding, call forwarding, message reception, call forwarding) of the voice recognition agent 100. have. The application program may be stored in the memory 170 and installed on the voice recognition agent 100 to be driven by the controller 180 to perform an operation (or function) of the voice recognition agent.

In addition to the operation related to the application program, the controller 180 typically controls the overall operation of the voice recognition agent 100. The controller 180 may provide or process information or a function appropriate to a user by processing signals, data, information, and the like, which are input or output through the above-described components, or driving an application program stored in the memory 170.

In addition, the controller 180 may control at least some of the components described with reference to FIG. 1A in order to drive an application program stored in the memory 170. In addition, the controller 180 may operate by combining at least two or more of the components included in the voice recognition agent 100 to drive the application program.

The power supply unit 190 receives power from an external power source and an internal power source under the control of the controller 180 to supply power to each component included in the voice recognition agent 100. The power supply unit 190 includes a battery, which may be a built-in battery or a replaceable battery.

Hereinafter, the components listed above will be described in detail with reference to FIG. 2 before looking at various embodiments implemented through the voice recognition agent 100 described above.

First, referring to the wireless communication unit 110, the broadcast receiving module 111 of the wireless communication unit 110 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 100 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The broadcast management server may mean a server that generates and transmits a broadcast signal and / or broadcast related information or a server that receives a previously generated broadcast signal and / or broadcast related information and transmits the same to a terminal. The broadcast signal may include not only a TV broadcast signal, a radio broadcast signal, and a data broadcast signal, but also a broadcast signal having a data broadcast signal combined with a TV broadcast signal or a radio broadcast signal.

The broadcast signal may be encoded according to at least one of technical standards for transmitting / receiving a digital broadcast signal (or a broadcast method, for example, ISO, IEC, DVB, ATSC, etc.), and the broadcast receiving module 111 may The digital broadcast signal may be received using a method suitable for the technical standard set by the technical standards.

The broadcast associated information may mean information related to a broadcast channel, a broadcast program, or a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms such as an electronic program guide (EPG) of digital multimedia broadcasting (DMB) or an electronic service guide (ESG) of digital video broadcast-handheld (DVB-H). The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the memory 160.

The mobile communication module 112 may include technical standards or communication schemes (eg, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV, etc.) for mobile communication. Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (DO), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced) and the like to transmit and receive a radio signal with at least one of a base station, an external terminal, a server on a mobile communication network.

The wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text / multimedia message.

The wireless internet module 113 refers to a module for wireless internet access and may be embedded or external to the voice recognition agent 100. The wireless internet module 113 is configured to transmit and receive wireless signals in a communication network according to wireless internet technologies.

Wireless Internet technologies include, for example, Wireless LAN (WLAN), Wireless-Fidelity (Wi-Fi), Wireless Fidelity (Wi-Fi) Direct, Digital Living Network Alliance (DLNA), Wireless Broadband (WiBro), and WiMAX (World Interoperability for Microwave Access (HSDPA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), and the like. 113) transmits and receives data according to at least one wireless Internet technology in a range including the Internet technologies not listed above.

In view of the fact that the wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A, etc. is made through a mobile communication network, the wireless Internet module 113 for performing a wireless Internet access through the mobile communication network 113 ) May be understood as a kind of mobile communication module 112.

The short range communication module 114 is for short range communication, and includes Bluetooth ™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, and NFC. (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus) by using at least one of the technologies, it can support near field communication. The short-range communication module 114 may be configured between a voice recognition agent 100 and a wireless communication system, between the voice recognition agent 100 and another voice recognition agent 100, or a voice through a wireless area network. Wireless communication between the recognition agent 100 and the network in which the other voice recognition agent 100 or an external server is located may be supported. The short range wireless communication network may be short range wireless personal area networks.

Here, the other voice recognition agent 100 may be a wearable device capable of exchanging data (or interworking with) the voice recognition agent 100 according to the present invention, for example, a smartwatch, It may be a smart glass, a head mounted display (HMD). The short range communication module 114 may detect (or recognize) a wearable device that can communicate with the voice recognition agent 100 around the voice recognition agent 100. In addition, when the detected wearable device is a device that is authenticated to communicate with the voice recognition agent 100 according to the present invention, the controller 180 transmits at least a portion of data processed by the voice recognition agent 100 to the near field communication. The module 114 may transmit the data to the wearable device. Therefore, the user of the wearable device may use data processed by the voice recognition agent 100 through the wearable device. For example, according to this, when a call is received by the voice recognition agent 100, the user performs a phone call through the wearable device or when a message is received by the voice recognition agent 100, the wearable device provides the call. It is possible to confirm the received message.

The location information module 115 is a module for obtaining the location (or current location) of the voice recognition agent, and a representative example thereof is a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, if the voice recognition agent utilizes a GPS module, the voice recognition agent may acquire a location of the voice recognition agent by using a signal transmitted from a GPS satellite.

As another example, when the voice recognition agent utilizes the Wi-Fi module, the voice recognition agent may acquire the location of the voice recognition agent based on the information of the wireless access point (AP) transmitting or receiving the Wi-Fi module and the wireless signal. have. If necessary, the location information module 115 may perform any function of the other modules of the wireless communication unit 110 to obtain data regarding the location of the voice recognition agent. The location information module 115 is a module used to obtain the location (or current location) of the voice recognition agent, and is not limited to a module that directly calculates or obtains the location of the voice recognition agent.

Next, the input unit 120 is for inputting image information (or signal), audio information (or signal), data, or information input from a user. For input of image information, the voice recognition agent 100 is One or a plurality of cameras 121 may be provided. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video call mode or the photographing mode. The processed image frame may be displayed on the display unit 151 or stored in the memory 170. On the other hand, the plurality of cameras 121 provided in the voice recognition agent 100 may be arranged to form a matrix structure, through the camera 121 forming a matrix structure in this way, the voice recognition agent 100 has a variety of angles or A plurality of image information having a focus may be input. In addition, the plurality of cameras 121 may be arranged in a stereo structure to acquire a left image and a right image for implementing a stereoscopic image.

The microphone 122 processes external sound signals into electrical voice data. The processed voice data may be utilized in various ways depending on the function (or running application program) being performed by the voice recognition agent 100. Meanwhile, various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in the process of receiving an external sound signal.

The user input unit 123 is for receiving information from a user. When information is input through the user input unit 123, the controller 180 may control an operation of the voice recognition agent 100 to correspond to the input information. have. The user input unit 123 may be a mechanical input unit (or a mechanical key, for example, a button, a dome switch, or a jog wheel located at the front or rear or side of the voice recognition agent 100). , Jog switch, etc.) and touch input means. As an example, the touch input means may include a virtual key, a soft key, or a visual key displayed on the touch screen through a software process, or a portion other than the touch screen. The virtual key or the visual key may be displayed on the touch screen while having various forms, for example, a graphic or text. ), An icon, a video, or a combination thereof.

Meanwhile, the sensing unit 140 senses at least one of information in the voice recognition agent, surrounding environment information surrounding the voice recognition agent, and user information, and generates a sensing signal corresponding thereto. The controller 180 may control driving or operation of the voice recognition agent 100 or perform data processing, function or operation related to an application program installed in the voice recognition agent 100 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 140 will be described in more detail.

First, the proximity sensor 141 refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or an object present in the vicinity without using a mechanical contact by using an electromagnetic force or infrared rays. The proximity sensor 141 may be disposed in the inner region of the voice recognition agent covered by the touch screen described above or near the touch screen.

Examples of the proximity sensor 141 include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. In the case where the touch screen is capacitive, the proximity sensor 141 may be configured to detect the proximity of the object with the change of the electric field according to the proximity of the conductive object. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of description, the action of allowing the object to be recognized without being in contact with the touch screen so that the object is located on the touch screen is referred to as "proximity touch", and the touch The act of actually touching an object on the screen is called a "contact touch." The position where an object is in close proximity touch on the touch screen means a position where the object is perpendicular to the touch screen when the object is in close proximity touch. The proximity sensor 141 may detect a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, and a proximity touch movement state). have.

Meanwhile, the controller 180 processes data (or information) corresponding to the proximity touch operation and the proximity touch pattern detected through the proximity sensor 141 as described above, and further, provides visual information corresponding to the processed data. It can be output on the touch screen. Furthermore, the controller 180 may control the voice recognition agent 100 to process different operations or data (or information) according to whether the touch on the same point on the touch screen is a proximity touch or a touch touch. have.

The touch sensor applies a touch (or touch input) applied to the touch screen (or display unit 151) using at least one of various touch methods such as a resistive film method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method. Detect.

As an example, the touch sensor may be configured to convert a change in pressure applied to a specific portion of the touch screen or capacitance generated at the specific portion into an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the touch, a capacitance at the touch, and the like, when the touch object applying the touch on the touch screen is touched on the touch sensor. Here, the touch object is an object applying a touch to the touch sensor and may be, for example, a finger, a touch pen or a stylus pen, a pointer, or the like.

As such, when there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and then transmits the corresponding data to the controller 180. As a result, the controller 180 can determine which area of the display unit 151 is touched. Here, the touch controller may be a separate component from the controller 180 or may be the controller 180 itself.

The controller 180 may perform different control or perform the same control according to the type of the touch object, which touches the touch screen (or a touch key provided in addition to the touch screen). Whether to perform different control or the same control according to the type of touch object may be determined according to the operation state of the voice recognition agent 100 or an application program being executed.

Meanwhile, the touch sensor and the proximity sensor described above may be independently or combined, and may be a short (or tap) touch, a long touch, a multi touch, a drag touch on a touch screen. ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, etc. A touch can be sensed.

The ultrasonic sensor may recognize location information of a sensing object using ultrasonic waves. On the other hand, the controller 180 can calculate the position of the wave generation source through the information detected from the optical sensor and the plurality of ultrasonic sensors. The position of the wave source can be calculated using the property that light is much faster than ultrasonic waves, i.e., the time that the light reaches the optical sensor is much faster than the time when the ultrasonic wave reaches the ultrasonic sensor. More specifically, the position of the wave generation source may be calculated using a time difference from the time when the ultrasonic wave reaches the light as the reference signal.

On the other hand, the camera 121, which is described as the configuration of the input unit 120, includes at least one of a camera sensor (for example, CCD, CMOS, etc.), a photo sensor (or an image sensor) and a laser sensor.

The camera 121 and the laser sensor may be combined with each other to detect a touch of a sensing object on a 3D stereoscopic image. The photo sensor may be stacked on the display element, which is configured to scan the movement of the sensing object in proximity to the touch screen. More specifically, the photo sensor mounts a photo diode and a transistor (TR) in a row / column and scans contents mounted on the photo sensor by using an electrical signal that varies according to the amount of light applied to the photo diode. That is, the photo sensor calculates coordinates of the sensing object according to the amount of change of light, and thus the position information of the sensing object can be obtained.

The display unit 151 displays (outputs) information processed by the voice recognition agent 100. For example, the display unit 151 may display execution screen information of an application program driven by the voice recognition agent 100 or UI (User Interface) or Graphic User Interface (GUI) information according to the execution screen information. have.

In addition, the display unit 151 may be configured as a stereoscopic display unit for displaying a stereoscopic image. The stereoscopic display unit may be a three-dimensional display method such as a stereoscopic method (glasses method), an auto stereoscopic method (glasses-free method), a projection method (holographic method).

Generally, a 3D stereoscopic image is composed of a left image (left eye image) and a right image (right eye image). A top-down method in which the left and right images are arranged up and down in one frame according to the way in which the left and right images are combined into a 3D stereoscopic image, and the left and right images are in the left and right in one frame. L-to-R (left-to-right, side by side) method that is arranged as a checker board method to arrange the pieces of the left and right images in the form of tiles, and the left and right images in columns Or an interlaced method of alternately arranging rows, and a time sequential (frame by frame) method of alternately displaying left and right images by time.

In addition, the 3D thumbnail image may generate a left image thumbnail and a right image thumbnail from the left image and the right image of the original image frame, respectively, and may be generated as one image as they are combined. In general, a thumbnail refers to a reduced image or a reduced still image. The left image thumbnail and the right image thumbnail generated as described above may be displayed with a three-dimensional space by displaying left and right distances on the screen by a depth corresponding to the parallax between the left and right images.

The left image and the right image necessary for implementing the 3D stereoscopic image may be displayed on the stereoscopic display by the stereoscopic processing unit. The stereoscopic processor is configured to receive a 3D image (an image of a reference time point and an image of an extended time point) and set a left image and a right image therefrom, or to receive a 2D image and convert it to a left image and a right image.

The sound output unit 152 may output audio data received from the wireless communication unit 110 or stored in the memory 170 in a call signal reception, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, and the like. The sound output unit 152 may also output a sound signal related to a function (eg, a call signal reception sound, a message reception sound, etc.) performed by the voice recognition agent 100. The sound output unit 152 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 153 generates various tactile effects that a user can feel. A representative example of the tactile effect generated by the haptic module 153 may be vibration. The intensity and pattern of vibration generated by the haptic module 153 may be controlled by the user's selection or the setting of the controller. For example, the haptic module 153 may synthesize different vibrations and output them or sequentially output them.

In addition to the vibration, the haptic module 153 may be used for stimulation such as a pin array vertically moving with respect to the contact skin surface, a jetting force or suction force of air through the injection or inlet, grazing to the skin surface, contact of an electrode, and electrostatic force. Various tactile effects can be generated, such as the effects of the heat-absorption and the reproduction of the sense of cold using the element capable of generating heat.

The haptic module 153 may not only deliver a tactile effect through direct contact, but also may allow a user to feel the tactile effect through a muscle sense such as a finger or an arm. The haptic module 153 may be provided with two or more according to the configuration of the speech recognition agent 100.

The light output unit 154 outputs a signal for notifying occurrence of an event by using light of a light source of the voice recognition agent 100. Examples of events generated by the voice recognition agent 100 may be message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output from the light output unit 154 is implemented as the voice recognition agent emits light of a single color or a plurality of colors to the front or the rear. The signal output may be terminated by the voice recognition agent detecting the user's event confirmation.

The interface unit 160 serves as a path to all external devices connected to the voice recognition agent 100. The interface unit 160 receives data from an external device, receives power, transfers the power to each component inside the voice recognition agent 100, or transmits the data inside the voice recognition agent 100 to an external device. For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, or a port that connects a device equipped with an identification module. The port, an audio input / output (I / O) port, a video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 160.

On the other hand, the identification module is a chip that stores a variety of information for authenticating the authority of the voice recognition agent 100, a user identification module (UIM), subscriber identity module (SIM), universal user A universal subscriber identity module (USIM) or the like. A device equipped with an identification module (hereinafter referred to as an 'identification device') may be manufactured in the form of a smart card. Therefore, the identification device may be connected to the terminal 100 through the interface unit 160.

In addition, the interface unit 160 may be a passage through which power from the cradle is supplied to the voice recognition agent 100 when the voice recognition agent 100 is connected to an external cradle, or by a user in the cradle. Various command signals inputted may be passages to the voice recognition agent 100. Various command signals or power input from the cradle may operate as signals for recognizing that the voice recognition agent 100 is correctly mounted on the cradle.

The memory 170 may store a program for the operation of the controller 180 and may temporarily store input / output data (for example, a phone book, a message, a still image, a video, etc.). The memory 170 may store data relating to various patterns of vibration and sound output when a touch input on the touch screen is performed.

The memory 170 may include a flash memory type, a hard disk type, a solid state disk type, an SSD type, a silicon disk drive type, and a multimedia card micro type. ), Card type memory (e.g. SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read It may include at least one type of storage medium of -only memory (PROM), programmable read-only memory (PROM), magnetic memory, magnetic disk and optical disk. The voice recognition agent 100 may be operated in connection with a web storage that performs a storage function of the memory 170 on the Internet.

On the other hand, as described above, the controller 180 controls the operation related to the application program, and generally the overall operation of the voice recognition agent 100. For example, if the state of the voice recognition agent satisfies a set condition, the controller 180 may execute or release a lock state that restricts input of a user's control command to applications.

In addition, the controller 180 may perform control and processing related to a voice call, data communication, video call, or the like, or may perform pattern recognition processing for recognizing handwriting or drawing input performed on a touch screen as text and images, respectively. Can be. Furthermore, the controller 180 may control any one or a plurality of components described above in order to implement the various embodiments described below on the voice recognition agent 100 according to the present invention.

The power supply unit 190 receives an external power source and an internal power source under the control of the controller 180 to supply power for operation of each component. The power supply unit 190 includes a battery, and the battery may be a built-in battery configured to be rechargeable, and may be detachably coupled to the terminal body for charging.

In addition, the power supply unit 190 may include a connection port, the connection port may be configured as an example of the interface 160 to which the external charger for supplying power for charging the battery is electrically connected.

As another example, the power supply unit 190 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 190 may use at least one of an inductive coupling based on a magnetic induction phenomenon or a magnetic resonance coupling based on an electromagnetic resonance phenomenon from an external wireless power transmitter. Power can be delivered.

Meanwhile, various embodiments of the present disclosure may be implemented in a recording medium readable by a computer or a similar device using, for example, software, hardware, or a combination thereof.

Meanwhile, the description of the voice recognition agent 100 described with reference to FIG. 2 may be applied to the mobile terminal 300 as it is.

In this specification, the term memory 170 may be used interchangeably with the term storage unit 170.

The controller 180 may control the operation of each component of the mobile terminal 100 under the control of the artificial intelligence unit 130.

Meanwhile, the input unit 120 of the mobile terminal 100 may include the sensing unit 140, and may perform all the functions performed by the sensing unit 140. For example, the input unit 120 may detect a user touch input.

3 is a block diagram showing the configuration of the artificial intelligence server 200 according to an embodiment of the present invention.

The communicator 210 may communicate with an external device.

In detail, the communication unit 210 may be connected to the voice recognition agent 100 to transmit / receive data with the voice recognition agent 100 under the control of the artificial intelligence unit 220.

In addition, the communication unit 210 may be connected to the mobile terminal 300 to transmit / receive data with the mobile terminal 300 under the control of the artificial intelligence unit 220.

And in the present specification, when the data transmitted from the artificial intelligence server 200 is finally transmitted to the mobile terminal 300, such data may be transmitted through the voice recognition agent 100, the voice recognition agent 100 It may be transmitted directly to the mobile terminal 300 without going through.

In addition, when the data transmitted from the mobile terminal 300 is finally transmitted to the AI server 200, such data may be transmitted through the voice recognition agent 100, without passing through the voice recognition agent 100 It may be sent directly to the artificial intelligence server 200.

The artificial intelligence unit 220 may receive voice data from the voice recognition agent 100 through the communication unit 210.

In addition, the voice recognition unit 222 included in the artificial intelligence unit 220 outputs a recognition result for the voice data using a voice recognition model, and transmits the output recognition result to the voice recognition agent, or outputs the recognition result. The corresponding control command may be sent to the speech recognition agent.

In addition, the voice recognition unit 222 included in the artificial intelligence unit 220 may adaptively learn the voice data, and store the learning result in the voice data database 232 in the storage unit 230.

In addition, the voice recognition unit 222 included in the artificial intelligence unit 220 may label the voice data in a sentence or a word and store the voice data in the voice data database 232.

Meanwhile, the artificial intelligence unit 220 may extract a recognition result by analyzing a speech signal and extracting a feature using a speech recognition model. Herein, the recognition result may be whether the received voice signal is a command or a non-command, or which command of a plurality of commands.

In this case, the command may be a voice recognition agent or another device connected to the voice recognition agent may be pre-registered to perform a specific function, and the non-command may be irrelevant to the performance of the specific function.

On the other hand, the sentence recommender 221 included in the artificial intelligence unit 220 may analyze the characteristics of the voice data using the speech feature analysis model.

The sentence database 231 in the storage unit 230 may hold a plurality of categorized sentences.

The sentence recommender 221 included in the artificial intelligence unit 220 searches for a specific sentence corresponding to the characteristic of the speech data among the plurality of sentences held by the sentence database 231 based on the characteristic of the speech data, The searched specific sentence may be transmitted to the speech recognition agent.

Meanwhile, in this drawing, the sentence recommendation unit 221, the voice recognition unit 222, the sentence database 231, and the voice database 232 are described as configuring one server, but the present disclosure is not limited thereto. Can be.

For example, the sentence recommender 221 and the sentence database 231 may constitute a first server, and the speech recognizer 222 and the speech database 232 may constitute a second server. In this case, the first server and the second server may transmit and receive data with each other.

4 is a diagram illustrating a problem that may occur in the speech recognition system.

Existing products improve the performance of speech recognition models by collecting data from multiple users, re-learning speech recognition models based on big data gathered in the cloud, and upgrading speech recognition software.

However, since the human voice / voice is so diverse, it is necessary to optimize the speech recognition model for specific users in order to increase the recognition rate.

If this optimization process does not exist, as recognition failure is repeated as shown in FIG. 4, the product and the brand may be negatively affected.

Therefore, it is necessary for a user who uses a speech recognition agent to learn his or her own voice.

5 is a diagram for describing a method of requesting additional data for further learning from a user according to an exemplary embodiment of the present invention.

The voice recognition agent 100 may receive voice data from the user (S505).

In addition, the voice recognition agent 100 may transmit the received voice data to the artificial intelligence server (S510).

Meanwhile, the artificial intelligence server 200 may receive voice data, input the received voice data into a voice recognition model, and output at least one of a voice recognition rate and a recognition result for the voice data (S515).

Here, the speech recognition rate may be measured by comparing a confidence score with respect to the voice.

In detail, the artificial intelligence server 200 may calculate a confidence score of the voice data of the user compared to an average of confidence scores extracted from the test data learned in the manufacturing process or the current personalized voice data.

For example, if the average of the confidence score of the previously learned voice data for a specific command or a start word is 70.02, and the confidence score of speech data spoken by a specific user is 52.13, the recognition rate is It can be calculated at about 74 percent.

In addition, the recognition rate may be obtained by comparing an error with a sample and obtaining an average value.

For example, a certain number of samples of speech data that have been previously learned for a specific command or a starting word are extracted, and a mean square error (MSE) or a root mean square error (RMSE) between the speech data and samples spoken by a specific user is extracted. By calculating, the recognition rate for the voice data of the user can be calculated.

Meanwhile, the artificial intelligence server 200 may transmit the acquired speech recognition rate to the speech agent 100 (S520).

Meanwhile, the voice recognition agent 100 may receive a voice recognition rate for the voice data, and if the voice recognition rate is lower than a preset criterion, the voice recognition agent 100 may request the user for additional data for learning the voice of the user.

In more detail, the speech recognition agent 100 may output a query for further learning in the speech recognition model in order to secure additional data (S525). In this case, the voice recognition agent 100 may output a voice recognition rate with respect to the voice data of the user.

For example, the Speech Recognition Agent (100) said, “As I understand the speech recognition rate, my recognition rate for your voice is about 60%. Would you like to optimize my speech recognition for your own voice? ”

On the other hand, if the input of the acceptance of the additional learning is received, the speech recognition agent 100 may provide a plurality of options for further learning (S530).

In more detail, the voice recognition agent may provide a user with a first option of speaking according to the presentation voice, a second option of speaking according to the presented sentence, and a third option of directly writing and speaking the sentence.

Meanwhile, when an input for selecting a specific option is received from the user (S535), the voice recognition agent may request additional data corresponding to the selected option from the user.

FIG. 6 is a diagram for describing an operating method when option 1 or option 2 is selected according to an embodiment of the present disclosure.

The speech recognition agent 100 may transmit a request for a sentence for further learning to the artificial intelligence server 200 (S605).

Meanwhile, when a request for a sentence is received (S610), the artificial intelligence server 200 may analyze characteristics of voice data (S615).

In addition, the artificial intelligence server 200 may search for a specific sentence corresponding to the characteristic of the voice data among the plurality of sentences based on the characteristic of the voice data (S620).

In more detail, a plurality of sentences may be stored in the sentence database 231, and the plurality of sentences may be classified by category. Here, the category may include at least one of product function, country, region, intonation, age, gender, and foreign language.

In addition, the artificial intelligence server 200 may calculate a recognition rate of words included in the voice data of the user.

For example, referring to FIG. 7, when the user utters the sentence “Can you tell me how many water bottle do we have?”, The artificial intelligence server 200 calculates a confidence score in units of words included in the sentence. In addition, a specific score (water, bottle) having a confidence score lower than a predetermined criterion may be obtained.

The artificial intelligence server may acquire the characteristics of the voice data based on the recognition rate of the words included in the voice data of the user and the characteristics of the words.

For example, if the recognition rate of certain words (water, bottle) is low, and certain words (water, bottle) have the characteristic that American English and British English are pronounced differently, the AI server can determine whether the user's country of origin is American or not. It is possible to acquire the characteristics of the voice data that the recognition rate of the words which are pronounced differently depending on whether the UK is low.

In this case, the artificial intelligence server may determine that additional learning is required for a country category among the plurality of categories based on the characteristics of the voice data.

The artificial intelligence server may acquire a specific sentence included in a category for which additional learning is required of the user from among a plurality of categories based on the characteristics of the voice data.

For example, a plurality of sentences including words that can distinguish a country of origin of a user may be classified into a country category. The AI server may acquire a sentence including a word that can be distinguished from the English and the American English among the plurality of sentences.

For example, "schedule" may have a characteristic that is pronounced differently in American English and British English. Therefore, the AI server can obtain the sentence “Can you tell me my schedule of today?” In the country category.

For example, "water" and "bottle" may have different characteristics in American English and British English. Therefore, the AI server can obtain the sentence “Can you tell me how many water bottle do we have?” In the country category.

That is, the words included in the acquired sentence may be words that have the same meaning and letter but may be pronounced by various pronunciations or various intonations.

In addition, a word included in a sentence corresponding to a specific category has the same meaning and letter, but may be a word that may be pronounced with various pronunciations or various accents, depending on the characteristics of the category (depending on the country, region, etc.).

In another example, the user intended to "play quiet music?", But as the user came from a certain area (Gyeongsang-do), she uttered the phrase "can you play quiet music?"

In this case, the artificial intelligence server 200 may calculate a recognition rate of words included in the voice data of the user, and may acquire a specific word (music) having a recognition rate lower than a preset criterion.

The artificial intelligence server may acquire the characteristics of the voice data based on the recognition rate of the words included in the voice data of the user and the characteristics of the words.

For example, if the recognition rate of a specific word (music) is low, and the specific word (music) has a characteristic of being pronounced unusually in a specific region (ordinary degree), the artificial intelligence server has a low voice recognition rate of a differently pronounced word in the current state The characteristics of the data can be obtained.

In this case, the artificial intelligence server may determine that additional learning is needed for a local category among the plurality of categories based on the characteristics of the voice data.

The artificial intelligence server may acquire a specific sentence included in a category for which additional learning is required of the user from among a plurality of categories based on the characteristics of the voice data.

For example, a plurality of sentences including words that can distinguish a region of a user may be classified into a region category. The artificial intelligence server may acquire a sentence including a word for learning that it is from a Gyeongsang province region among the plurality of sentences.

For example, “rice” may have the characteristic of being pronounced “sal” in Gyeongsang-do. Thus, the AI server can obtain a sentence from the local category, "How much rice is left in the house?"

That is, a word included in a sentence corresponding to a local category may be a word that has the same meaning and letter but may be pronounced by various pronunciations or various intonations depending on the region.

In addition, sentences related to product functions may be classified into product function categories. In this case, the sentence related to the product function may include a command corresponding to a function performed by the speech recognition agent or another device linked with the speech recognition agent.

For example, "Can you tell me how many minutes of washing machine dehydration is left?" And "Would you like to raise the air conditioning temperature to 24 degrees?"

When the voice data of the user has a low recognition rate for the command, the AI server may extract a sentence from a product function category.

In addition, a word included in a sentence corresponding to an age category has the same meaning and letter, but may be a word that may be pronounced by various pronunciations or various intonations according to age.

In addition, a word included in a sentence corresponding to a gender category may be a word that has the same meaning and letter but may be pronounced by various pronunciations or various intonations according to gender.

In addition, a word included in a sentence corresponding to a dialect category may be a word that has the same meaning and letter but may be pronounced by various dialects or various intonations by the dialect.

In addition, a word included in a sentence corresponding to a foreign language category may be a word having the same meaning and letter but may be pronounced by various pronunciations or various intonations by the foreign language.

Meanwhile, the artificial intelligence server 200 may acquire characteristics of the voice data based on personal information registered by the user in addition to extracting the characteristics from the voice data.

For example, a user can register personal information such as country, gender, age, region, dialect. If the user registers personal information that the country of origin is the United Kingdom, the artificial intelligence server determines that additional learning is required for the country category, and obtains a sentence containing words that can be distinguished between English English and American English. Can be.

Meanwhile, the specific sentence acquired by the artificial intelligence server may include a command corresponding to the function of the voice recognition agent.

Here, the function of the speech recognition agent may include a function provided by a device interoperating with the speech recognition agent as well as the function provided by the speech recognition agent itself.

In addition, the specific sentence not only includes words for learning the user's country, region, age, etc., but also includes a command to be directly uttered by the user, so that the AI server can collect voice data corresponding to the command. .

Meanwhile, the specific sentence obtained by the artificial intelligence server may include a startup word for calling the voice recognition agent.

The artificial intelligence server may improve the recognition rate of the starting word by extracting and learning only the starting word from the second voice data spoken by the user in response to a specific sentence.

Meanwhile, the artificial intelligence server may transmit the acquired specific sentence to the voice recognition agent (S625).

Meanwhile, when additional learning is required, the process of transmitting a specific sentence may be performed by omitting S520 through S535 and S605.

In detail, when voice data is received, the artificial intelligence server 200 may analyze characteristics of the voice data and obtain a recognition rate of the voice data. In addition, if the recognition rate of the voice data is lower than the preset reference, the artificial intelligence server 200 may search for a specific sentence corresponding to the characteristic of the voice data and transmit the same to the voice recognition agent 100.

Meanwhile, the voice recognition agent 100 may output the received specific sentence (S630).

In detail, as illustrated in FIG. 8, when the user selects the first option of speaking according to the presented speech, the speech recognition agent may output the received specific sentence as a speech signal.

In addition, when the user selects the second option of speaking according to the presented sentence, as shown in FIG. 9, the voice recognition agent may transmit a specific sentence to the user's mobile terminal 300.

In this case, the mobile terminal 300 of the user may display text corresponding to a specific sentence.

Meanwhile, when the user speaks a specific sentence, the voice recognition agent may receive second voice data corresponding to the specific sentence spoken (S635) and transmit the received second voice data to the artificial intelligence server 200 (S640). ).

On the other hand, when the second voice data corresponding to the specific sentence is received, the artificial intelligence server 200 may learn the second voice data corresponding to the specific sentence (S645).

The artificial intelligence server may retain the voice data before learning the second voice data. When the second voice data is received, the voice data before learning the second voice data may be used as the source data, the second voice data may be the target data, and the adaptive data may be adaptively matched to the target data.

In addition, the artificial intelligence server may label the specific sentence on the second voice data and store it in the voice database 232. Here, the voice database 232 is a database personalized to a specific user and may be used to recognize a voice of a specific user.

In this case, the speech recognition model may be updated to reflect the learning results. The artificial intelligence server may transmit the changed speech recognition rate according to the result of learning the additional data (S650).

In more detail, the artificial intelligence server may re-input the voice data received in operation 510 into the updated voice recognition model, calculate the recognition rate, and transmit the received voice data to the voice recognition agent.

Meanwhile, when the changed recognition rate is received, the voice recognition agent may output the changed recognition rate (S655).

For example, the voice recognition agent can output the message, "As a result of learning my algorithm based on the voice data provided by the customer, the learning rate improved from 60% to 70%."

On the other hand, with reference to FIG. 10 will be described an embodiment when the user writes a sentence directly and selected the third option of speaking.

When the user inputs the specific text and the third voice data corresponding to the specific text, the speech recognition agent may transmit the specific text and the third voice data corresponding to the specific text to the artificial intelligence server.

In more detail, at least one of the mobile terminal 300 and the voice recognition agent 100 may receive text input of the user and voice data corresponding to the input text.

In this case, the voice agent may transmit the received text and the voice data corresponding to the text to the artificial intelligence server.

In this case, the artificial intelligence server may learn third voice data corresponding to a specific text.

In detail, the artificial intelligence server may determine words included in the text and voice data corresponding to the words. The AI server may learn voice data corresponding to a word.

FIG. 11 is a diagram for describing a method of requesting additional data for additional learning from a user according to another exemplary embodiment.

The voice recognition agent 100 may receive voice data from the user (S1105).

In addition, the voice recognition agent 100 may transmit the received voice data to the artificial intelligence server (S1110).

Meanwhile, the artificial intelligence server 200 may receive voice data and store the received voice data in the storage unit (S1115).

In addition, the artificial intelligence server 200 may input voice data into a voice recognition model to output at least one of a voice recognition rate and a recognition result for the voice data (S1120).

Meanwhile, the artificial intelligence server 200 may transmit the acquired voice recognition rate to the voice agent 100 (S1125).

Meanwhile, the voice recognition agent 100 may receive a voice recognition rate for the voice data, and if the voice recognition rate is lower than a preset criterion, the voice recognition agent 100 may request the user for additional data for learning the voice of the user.

In detail, as illustrated in FIG. 12, when the speech recognition rate is lower than the preset reference, the speech recognition agent 100 may transmit a request for text input corresponding to the received speech data to the mobile terminal 300. (S1130).

Meanwhile, the mobile terminal 300 may receive an input of text corresponding to the voice data uttered by the user from the user (S1135) and transmit the received text to the voice recognition agent (S1135).

In this case, the voice recognition agent 100 may transmit the received text to the artificial intelligence server 200 (S1140).

Meanwhile, the text transmitted from the mobile terminal 300 is transmitted to the AI server through the voice recognition agent, but the present invention is not limited thereto. The mobile terminal 300 may directly transmit the text to the AI server.

In this case, the artificial intelligence server may learn prestored voice data corresponding to the text (S1145).

In detail, the artificial intelligence server may convert the received text into voice data by using text to speech (TTS). The artificial intelligence server compares the stored voice data with the metric of the converted voice data to calculate similarity, and determines the pre-stored voice data as valid data based on the similarity between the pre-stored voice data and the converted voice data. Can be.

When the pre-stored voice data is determined to be valid data, the artificial intelligence server may label the voice data determined as valid data with text and store the same in the voice data database 232.

On the other hand, the learning of the voice data can be implemented in such a manner as to learn a TTS, output a voice, and then learn a speech recognition model when a user's acceptance request is received.

In more detail, the artificial intelligence server may learn the TTS using voice data of the user. The artificial intelligence server may generate and transmit voice data similar to the user's voice with the learned TTS. The voice recognition agent may output voice data generated in the TTS.

In this case, the user may determine whether the voice generated in the TTS is similar to his voice, and input a request for acceptance if the voice is similar to his voice.

In this case, the voice recognition agent may transmit an acceptance request to the AI server, and the AI server may update the voice recognition model by learning the voice data learned from the TTS.

In addition, when a user who determines that the voice generated in the TTS is not similar to his or her voice inputs a rejection request, the voice recognition agent may request the user for additional data for learning the user's voice.

Meanwhile, the request for text for further learning may be performed when the repetition of speech recognition fails.

For example, if the speech recognition agent fails to recognize the same word or sentence more than a predetermined number of times, or if the recognition rate is lower than the predetermined criterion more than the predetermined number of times, the speech recognition agent responds to the speech data that is issued to the user. You can request to enter text.

For another example, the speech recognition agent first performs a learning by asking a user to present a specific sentence and read along the specific sentence in the manner described with reference to FIG. 6, and is added when the user's voice is still not recognized. You can ask the user for text for learning.

13 is a diagram for describing an operation of a voice recognition system according to an exemplary embodiment of the present invention.

The voice recognition system may receive user information from the user and register the received user information (S1310).

In detail, the voice recognition agent receives the user information and transmits the user information to the server, and the server may store the received user information.

The user information may include at least one of country, region, intonation, age and gender.

Meanwhile, the voice recognition system may receive voice data of a user, recognize voice data, and perform a function corresponding to a voice recognition result (S1320 and S1330).

Meanwhile, the voice recognition system may determine whether the user participates in additional learning and learning options (S1340).

In detail, the voice recognition agent may output a query for further learning and provide a plurality of options for the additional learning method.

And when an input is received from the user to accept further learning and select a specific option, the speech recognition system may register the selected option. If further learning is required later, the voice recognition system may proceed with additional learning with the registered option.

On the other hand, since the option of learning better may vary depending on the user, the speech recognition agent may first learn with all of the plurality of options, and then register the option having a high speech recognition rate after the learning.

For example, if the recognition rate of the first option of speaking according to the presented speech, the second option of speaking according to the presented sentence, and the second option of writing and speaking directly according to the present sentence have the highest recognition rate, the speech recognition system has the highest speech recognition rate. A second option may be to request additional data from the user.

Meanwhile, the criterion of speech recognition rate for performing a specific function may be different depending on what the specific function is.

For example, a home voice based service having a command such as “turn on” or “turn off” may perform a function corresponding to a user's command as long as the voice recognition rate is 55% or more.

For another example, the command for confirming the user's personal message may perform a function corresponding to the user's command when the voice recognition rate is 65% or more.

For another example, the command for payment or authentication may perform a function corresponding to the user's command when the voice recognition rate is 75% or more.

Meanwhile, the present invention has been described as being implemented by a voice recognition agent, an artificial intelligence server, and a mobile terminal, but is not limited thereto.

For example, all the configurations and functions of the above-described AI server may be mounted on the voice recognition agent and performed.

As described above, the present invention, unlike the conventional method of passively collecting and learning the user's voice, requests a voice input by presenting a sentence that can best grasp the user's speech habit, or directly inputs a sentence spoken by the user. Request by text Therefore, according to the present invention, it is possible to greatly improve the learning performance, there is an advantage capable of fast personalization.

Meanwhile, the controller 180 is generally configured to control the device, and may be mixed with terms such as a CPU, a microprocessor, and a processor.

The present invention described above can be embodied as computer readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAM, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. This also includes implementations in the form of carrier waves (eg, transmission over the Internet). In addition, the computer may include the controller 180 of the terminal. Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

10: speech recognition system 100: speech recognition agent
200: AI server 300: mobile terminal

Claims (14)

In the speech recognition system,
A voice recognition agent receiving voice data from a user and transmitting the voice data to an artificial intelligence server; And
An artificial intelligence server for inputting the voice data into a voice recognition model, transmitting a result of recognizing the voice data to the voice recognition agent, and learning the voice data;
The speech recognition agent,
If the speech recognition rate for the voice data is lower than a preset criterion, requesting the user for additional data for learning the voice of the user
Speech recognition system. The method of claim 1,
The speech recognition agent,
Providing a specific sentence to the user, and when the second voice data corresponding to the specific sentence is received, transmitting the second voice data to the artificial intelligence server;
The artificial intelligence server,
Learning the second voice data corresponding to the specific sentence
Speech recognition system. The method of claim 2,
The artificial intelligence server,
Transmitting the specific sentence corresponding to the characteristic of the speech data among the plurality of sentences to the speech recognition agent based on the characteristic of the speech data.
Speech recognition system. The method of claim 3, wherein
The plurality of sentences,
Are categorized into categories that include at least one of product features, country, region, age, dialect, gender, and foreign language,
The artificial intelligence server,
On the basis of the characteristics of the voice data, to transmit the specific sentence included in the category that requires further learning to the user of the plurality of categories to the voice recognition agent
Speech recognition system. The method of claim 3, wherein
The specific sentence,
Including a command corresponding to the function of the speech recognition agent
Speech recognition system. The method of claim 2,
The speech recognition system,
Further comprising a mobile terminal,
The speech recognition agent,
Transmitting the specific sentence to the mobile terminal of the user,
The mobile terminal,
Displaying text corresponding to the specific sentence
Speech recognition system. The method of claim 1,
The speech recognition agent,
If the speech recognition rate is lower than a preset criterion, requesting the user to input text corresponding to the speech data
Speech recognition system. The method of claim 7, wherein
The artificial intelligence server,
Store the voice data,
The speech recognition agent,
When text corresponding to the voice data is input, the text corresponding to the voice data is transmitted to the artificial intelligence server,
The artificial intelligence server,
Learning the stored voice data corresponding to the text
Speech recognition system. The method of claim 8,
The artificial intelligence server,
Converting the text into speech data, determining the stored speech data as valid data based on the similarity between the converted speech data and the stored speech data, and learning the speech data determined as the valid data.
Speech recognition system. The method of claim 8,
The speech recognition system,
The mobile terminal may further include receiving a text input corresponding to the voice data and transmitting the text corresponding to the voice data to the voice recognition agent.
Speech recognition system. The method of claim 1,
The speech recognition agent,
When the user inputs specific text and third voice data corresponding to the specific text, the specific text and the third voice data corresponding to the specific text are transmitted to the artificial intelligence server,
The artificial intelligence server,
Learning the third voice data corresponding to the specific text
Speech recognition system. The method of claim 1,
The speech recognition agent,
Provides a first option of speaking along the presented speech, a second option of speaking along the presented sentence, and a third option of directly writing and speaking along the sentence,
Requesting the additional data with the highest speech recognition rate among the first to third options
Speech recognition system. The method of claim 1,
The artificial intelligence server,
Learning the additional data and transmitting the changed speech recognition rate to the speech recognition agent according to a result of learning the additional data;
Speech recognition system. In the speech recognition device,
An input unit to receive voice data from a user; And
An artificial intelligence unit for inputting the voice data into a voice recognition model, obtaining a recognition result for the voice data, and learning the voice data;
The artificial intelligence unit,
If the speech recognition rate for the voice data is lower than a preset criterion, requesting the user for additional data for learning the voice of the user
Speech recognition device.

Images