KR20220003050U - Electronic apparatus for providing artificial intelligence conversations - Google Patents

Abstract

An electronic device for providing artificial intelligence conversations is provided. An electronic device for providing artificial intelligence conversation receives a user's voice signal, converts the voice signal into a text signal, performs natural language processing on the text signal to extract keywords, and uses the extracted keywords to match with a database. to form a knowledge base response signal, form an emotion base response signal in consideration of the user's age, personality, and intimacy using the knowledge base response signal, and convert the emotion base response signal into a voice output signal.

Description

Electronic device for providing artificial intelligence conversation {ELECTRONIC APPARATUS FOR PROVIDING ARTIFICIAL INTELLIGENCE CONVERSATIONS}

The present invention relates to an electronic device for providing artificial intelligence conversations, and more specifically, to an electronic device for providing artificial intelligence conversations capable of providing emotion-based conversations based on artificial intelligence (AI) learning. .

As the dissemination of AI conversation technology expands and various services related to AI conversation technology appear, the utilization of AI conversation technology continues to increase. Users can use services in various fields through artificial intelligence conversations.

However, when artificial intelligence conversation technology is used, the answer of the artificial intelligence system to the user's inquiry is implemented using a preset tone, accent, accent, etc., so the user does not feel familiar with the artificial intelligence system. there is. In particular, there is a problem that young children can feel fear about the answer of the artificial intelligence system.

One technical problem to be solved by the present invention is to provide an electronic device for providing an artificial intelligence conversation that provides an emotion-based response to a voice received from a user.

Another technical problem to be solved by the present invention is to provide an electronic device for providing an artificial intelligence conversation that provides an emotion-based response in consideration of the age, personality, and intimacy of the user to the voice received from the user.

Another technical problem to be solved by the present invention is to provide an electronic device for providing artificial intelligence conversations that updates a database by assigning a weight according to the degree of a user's reaction to a voice output.

The technical problem to be solved by the present invention is not limited to the above.

An electronic device for providing artificial intelligence conversations according to an embodiment of the present invention includes one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, wherein the one or more processors receive a user's voice signal, and convert the voice signal into a text signal. , performs natural language processing of the text signal to extract keywords, uses the extracted keywords to perform database matching to form a knowledge base response signal, and uses the knowledge base response signal to determine the information of the user. An emotion base response signal may be formed in consideration of age, personality, and intimacy, and the emotion base response signal may be converted into a voice output signal.

As an embodiment, the one or more processors may perform natural language processing on the received voice signal through sentence separation, spacing correction, spelling correction, abbreviation expansion, and symbol removal procedures on the voice signal.

As an embodiment, the one or more processors receive a plurality of voice data according to gender, region, age, and multilateral speech, update the database using the plurality of voice data, and generate the extracted keywords. It is possible to form the knowledge base response signal by performing matching with the database using the database.

As an embodiment, the one or more processors may update the database by assigning a weight according to the degree of the user's reaction to the audio output signal.

As an embodiment, the one or more processors determine that it is impossible to form the knowledge base response signal when it is impossible to match the extracted keyword with the database, and use a machine learning algorithm to determine the knowledge base from the extracted keyword. A response signal may be formed.

One or more processors according to an embodiment of the present invention; and one or more memories storing instructions for causing the one or more processors to perform calculations when executed by the one or more processors. By, receiving the user's voice signal; converting, by the one or more processors, the voice signal into a text signal; extracting, by the one or more processors, keywords by performing natural language processing on the text signal; forming a knowledge base response signal by matching with a database using the extracted keywords, by the one or more processors; forming, by the one or more processors, an emotion base response signal in consideration of the user's age, personality, and intimacy using the knowledge base response signal; and converting the emotional base response signal into a voice output signal by the one or more processors.

one or more processors according to an embodiment of the present invention; And a computer program stored in a computer-readable recording medium so as to be executable by a computer including one or more memories storing instructions for causing the one or more processors to perform operations when executed by the one or more processors, the one or more processors Receiving a user's voice signal by the; converting, by the one or more processors, the voice signal into a text signal; extracting, by the one or more processors, keywords by performing natural language processing on the text signal; forming a knowledge base response signal by matching with a database using the extracted keywords, by the one or more processors;

forming, by the one or more processors, an emotion base response signal in consideration of the user's age, personality, and intimacy using the knowledge base response signal; and converting the emotional base response signal into a voice output signal by the one or more processors.

An electronic device and method for providing artificial intelligence conversations according to an embodiment of the present invention, and a program stored in a computer-readable recording medium to perform the same, provide an emotion-based response to a voice received from a user to provide a user-friendly can make conversation possible.

In addition, an electronic device and method for providing an artificial intelligence conversation according to an embodiment of the present invention, and a program stored in a computer-readable recording medium to perform the same, are related to the user's age, personality, intimacy, etc. with respect to the voice received from the user. It is possible to provide a user-customized conversation considering the artificial intelligence conversation that the user can find again.

In addition, an electronic device and method for providing artificial intelligence conversations according to an embodiment of the present invention, and a program stored in a computer-readable recording medium to perform the same, assign weights according to the degree of user's reaction to voice output to database Since it performs an update of , it is possible to reduce the amount of computation for providing more sophisticated and artificial intelligence conversations.

1 is an exemplary diagram showing the configuration of an electronic device for providing artificial intelligence conversations according to an embodiment of the present invention.
2 is an exemplary diagram showing the configuration of an artificial intelligence conversation providing system according to an embodiment of the present invention.
3 is a diagram for explaining learning of a neural network according to an embodiment of the present invention.
4 and 5 are flowcharts showing procedures of a method for providing artificial intelligence conversations according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the technical spirit of the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosed content will be thorough and complete, and the spirit of the present invention will be sufficiently conveyed to those skilled in the art.

In this specification, when an element is referred to as being on another element, it means that it may be directly formed on the other element or a third element may be interposed therebetween.

In addition, although terms such as first, second, and third are used to describe various elements in various embodiments of the present specification, these elements should not be limited by these terms. These terms are only used to distinguish one component from another. Therefore, what is referred to as a first element in one embodiment may be referred to as a second element in another embodiment. Each embodiment described and illustrated herein also includes its complementary embodiments. In addition, in this specification, 'and/or' is used to mean including at least one of the elements listed before and after.

In the specification, expressions in the singular number include plural expressions unless the context clearly dictates otherwise. In addition, the terms "comprise" or "having" are intended to designate that the features, numbers, steps, components, or combinations thereof described in the specification exist, but one or more other features, numbers, steps, or components. It should not be construed as excluding the possibility of the presence or addition of elements or combinations thereof.

In addition, in this specification, "connection" is used to mean both indirectly and directly connecting a plurality of components. In addition, "connection" is a concept including physical connection as well as electrical connection.

In addition, in the following description of the present invention, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

1 is an exemplary diagram showing the configuration of an electronic device for providing artificial intelligence conversations according to an embodiment of the present invention.

As shown in FIG. 1, an electronic device 100 for providing artificial intelligence conversation includes one or more processors 110, one or more memories 120, a transceiver 130, a microphone 140, and a speaker 150. can include As an example, at least one of these components of the electronic device 100 for providing artificial intelligence conversations may be omitted or another component may be added to the electronic device 100 for providing artificial intelligence conversations. Additionally or alternatively, some of the components may be integrated and implemented, or implemented as a singular or plural entity. At least some of the internal and external components of the electronic device 100 for providing artificial intelligence conversations are system bus, GPIO (general purpose input/output), SPI (serial peripheral interface) or MIPI ( They are connected to each other through a mobile industry processor interface, etc., and can exchange data and/or signals. As an embodiment, the electronic device 100 for providing artificial intelligence conversations may form an emotion-based response signal using machine learning, in particular, a deep reinforcement learning algorithm such as deep learning.

The one or more processors 110 may control at least one component of the electronic device 100 for providing artificial intelligence conversations connected to the processor 110 by driving software (eg, commands, programs, etc.). In addition, the processor 110 may perform operations such as various calculations, processing, data generation, and processing related to the present invention. Also, the processor 110 may load data from one or more memories 120 or store data in one or more memories 120 .

One or more processors 110 may receive a user's voice signal. According to an embodiment, the one or more processors 110 may acquire the user's voice signal through the microphone 140 in the form of data packets that can be processed by a computer. For example, the processor 110 may recognize a voice signal continuously pronounced by the user as several words using a continuous speech recognition algorithm. According to an embodiment, the user's voice signal may include a naturally uttered voice, hesitation, exclamation, breathing, etc. as if talking in a natural voice (Spontaneous Speech). According to another embodiment, the processor 110 may improve convenience through speaker independent recognition of a voice of an arbitrary speaker so that an unspecified number of users can use it. For example, the processor 110 may improve a user's voice recognition rate through a voice training function. According to another embodiment, the processor 110 may improve the accuracy of words and phrases frequently appearing in audio data by implementing speech adaptation technology to which Google Cloud Speech is applied.

One or more processors 110 may convert the received voice signal into a text signal. According to an embodiment, the one or more processors 110 sample the received voice signal by an analog digital (A/D) converter, convert it into a digital signal, remove the noise signal, and convert it into a text signal. can For example, the processor 110 may select and use various languages as a speech to text (STT) recognition engine, and use a BCP-47 identifier in a language code field of a request configuration to use an audio language ( and national or regional dialects) can be designated and developed for use. The processor 110 may activate and use a function using a set button or key. Streaming speech recognition can be used to stream audio into STT, process the streaming speech recognition results into text in real time as the audio is processed, and save it as a translation system file. Streaming STT application programming interface (API) recognition calls, designed for real-time capture and recognition of audio within a bi-directional stream, applications send audio in a request stream and store interim and final recognition results as text in real-time in a response stream; The intermediate result represents the current recognition result of a particular audio section, and the final recognition result may lead to the highest final guess of that audio section. The processor 110 may reduce the network speed to 1/10 by applying a content delivery network (CDN) service. The processor 110 may design and configure a cloud architecture considering service operation. It is a dual configuration in consideration of scalability and stability, and it can be reflected in consideration of service expansion for Clova and VOD Trans that are being provided. You can upload related configuration files to NAVER CLOUD PLATFORM's Object Storage and connect HTTP/HTTPS protocols through CDN+ (CDN+ is a service that delivers content to many users quickly and safely without loss). The processor 110 may perform recognition and service connection by utilizing a call voice file. After calling the service through the API, the result value can be derived when checking.

One or more processors 110 may extract keywords by performing natural language processing on the received text signal. According to an embodiment, the one or more processors 110 may perform natural language processing on the received voice signal through sentence separation, spacing correction, spelling correction, abbreviation expansion, and symbol removal procedures with respect to the text signal. For example, the processor 122 extracts all words included in the received text signal, removes sentence terminating marks for the extracted words, and separates two consecutive words among the extracted words into different sentences. A possible sentence separation probability is calculated, the sentence separation probability is compared with a predetermined reference probability, and when the sentence separation probability is greater than or equal to the predetermined reference probability, two consecutive words may be separated into different sentences.

In addition, the one or more processors 110 remove all punctuation marks from each sentence included in the received text signal, check whether each of the extracted words is included in a predetermined corpus, and determine whether each of the extracted words is included in a predetermined corpus. If included in the determined corpus, the spelling of the word is not corrected, and if each of the extracted words is not included in the corpus, the correction target word that is not included in the corpus may be corrected as a correction candidate word.

The one or more processors 110 determine whether each of the extracted words is included in a pre-determined thesaurus (a dictionary representing a synonym or antonymic relationship between different words), and each of the extracted words is included in the thesaurus If not, the word is not changed, and if each of the extracted words is included in the thesaurus, the word may be changed to an abbreviated extended word included in the thesaurus.

In addition, the one or more processors 110 calculate the probability that each of the extracted words will be classified as a dependent element and a dominant element, compare the probability of being classified as a dependent element and the probability of being classified as a dominant element, and determine the probability of being classified as a dependent element. If the probability of being classified as a dominant element is higher than the probability of being classified as a dominant element, the extracted word is classified as a dependent element, and if the probability of being classified as a dominant element is higher than the probability of being classified as a dependent element, the extracted word can be classified as a dominant element.

For example, when the one or more processors 110 receive a voice signal from the user, “How is the weather at Mt. Seorak tomorrow?”, the one or more processors 110 perform natural language processing to convert “tomorrow”, “Mt. can be extracted by keyword.

According to another embodiment, one or more processors 110 extracts features useful for voice recognition from text signals to generate feature vectors, and references the generated feature vectors to a dictionary composed of words to be recognized previously prepared in a learning process. The most similar words can be extracted as keywords. For example, a small vocabulary of 100 words or less used in general chatting is used, but a large vocabulary of 10,000 words or more may be applied if necessary.

One or more processors 110 may form a knowledge base response signal by performing matching with a database using the extracted keyword. According to an embodiment, the one or more processors 110 receive a plurality of voice data according to gender, region, age, and multilateral speech, update a database using the plurality of voice data, and use a text signal. Accordingly, a knowledge base response signal may be formed by performing matching with a database. For example, the processor 110 may update the database by assigning a weight according to the degree of the user's reaction to the audio output signal. In addition, the processor 110 may build a rule-base dialog system to form a knowledge base response signal. In other words, it is possible to build a dialogue voice data set for the development of artificial intelligence technology that recognizes daily conversation and converts voice signals into text signals in real time, and original voice data (4,000 hours) by field such as gender, region, age, long distance, and multilateral speech. ), 4 million sentences of text data can be secured, and a conversation data set based on the secured voice data can be constructed. In addition, the processor 110 can build a cloud-based large-scale data set capable of flexible answers, and can be advanced to enable appropriate answers to unexpected questions.

For example, the processor 110 performs matching with the database using the extracted keywords such as “tomorrow,” “Mt. Seorak,” and “weather,” and “The temperature of Mt. Seorak is 14 to 23° C. A knowledge base response signal of “cloudy, fine dust/ultra fine dust is good” may be formed.

One or more processors 110 may use the knowledge base response signal to form an emotion base response signal in consideration of the user's age, personality, and intimacy. According to an embodiment, the one or more processors 110 may form the emotion base response signal so that emotions appear in the knowledge base response signal in consideration of the age of the user. For example, the processor 110 may extract age information of the user from the user's voice signal and adjust the tone of the knowledge base response signal using the extracted age information to form an emotion base response signal. That is, when the age of the user is extracted as a teenager, the processor 110 adjusts the tone of the knowledge base response signal "The temperature of Mt. , "The temperature of Mt. Seorak is 14~23℃, it will be cloudy and cloudy, fine dust/ultra fine dust will be good", can form an emotional base response signal. In addition, the processor 110 may extract the user's personality information from the user's voice signal and adjust the tone of the knowledge base response signal using the extracted personality information to form an emotion base response signal. That is, the processor 110 adjusts the tone of the knowledge base response signal, "The temperature of Mt. Seorak is 14-23°C, cloudy and cloudy, fine dust/ultra-fine dust is good," when the user identifies as impatient. The emotional base response signal can be formed in the simplest form as possible: "The temperature of Mt. In addition, the processor 110 may extract the user's intimacy information from the user's voice signal and adjust the tone of the knowledge base response signal using the extracted intimacy information to form an emotion base response signal. That is, when the processor 110 determines that the intimacy with the user is low, the processor 110 adjusts the tone of the knowledge base response signal, "The temperature of Mt. , An emotional base response signal can be formed in the form of a formal sentence, "The temperature of Mt. In addition, one or more processors 110 may perform machine learning on emotion base response signal formation by assigning a weight to the emotion base response signal using a user's feedback on the emotion base response signal. For example, the processor 110 assigns a higher weight to the emotion base response signal when the user smiles with respect to the emotion base response signal, and assigns a higher weight to the emotion base response signal when there is no change in the user's facial expression with respect to the emotion base response signal. may be given a lower weight.

One or more processors 110 may determine that it is impossible to form a knowledge base response signal when it is impossible to match the extracted keyword with the database, and form a knowledge base response signal from the extracted keyword using a machine learning algorithm. According to an embodiment, when the processor 110 receives a voice signal "Where is the favorite ski resort in Africa?" from the user, there is no response matching the keywords of the voice signal "Africa" or "ski resort" in the database. , and a knowledge base response signal such as "The Sahara Desert is the best ski resort in Africa" can be formed. In addition, the one or more processors 110 may form an emotion-based response signal in consideration of the user's age, personality, and intimacy using the knowledge-base response signal.

One or more processors 110 may induce appropriate answers through voice conversation history data and perform keyword training. The disadvantages of each model can be complemented by the flexible use of two types of search-based models (formation of knowledge base response signals using matching with databases) and generative models (formation of knowledge base response signals using machine learning), In order to form a natural response signal, a linguistic context and a physical context may be merged. In order to maintain consistency in answers, it is possible to pursue unification of personality for fixed knowledge and models, and to create an integrated analysis platform that supports text analysis, machine learning, and deep learning algorithms in a single environment for the use of artificial intelligence. It may be possible to build cloud-based large data sets with flexible answers.

One or more processors 110 may derive a 5G cloud immersive content architecture and additionally reflect native services provided by cloud operators based on the existing architecture. By upgrading the architecture, it is possible to design and configure the cloud architecture considering actual service operation. Considering scalability and stability, redundant configuration and service expansion, redundant server configuration is performed, and 5G interoperable architecture is configured to provide metaverse services such as AR (Augmented Reality) and VR (Virtual Reality). It can be configured to enable scaling considering scalability. By utilizing a load balancer (a service that distributes network traffic to multiple servers in consideration of server performance and load), stabilization of service operations and redundant configuration are implemented, and the number of servers is set according to pre-registered settings using auto scaling. can be automatically increased or decreased to maintain stable service.

The one or more processors 110 may develop a voice TTS (Text to Speech) suitable for the AI character and implement a natural voice TTS like a real human voice. A tuned voice can be implemented by reproducing the voice of a specific person and using the Naver Clova TTS voice of a similar level to compare and analyze the spectrum. By utilizing WaveNet technology and Clova TTS technology, personalized speech synthesis technology can be implemented, and technology that directly models the raw waveform of an audio signal one sample at a time can be implemented.

In addition to its original purpose of game development, game engines are used for various developments such as mobile apps, AR, and VR. The electronic device 100 for providing artificial intelligence conversation of the present invention is manufactured as a universal platform that can be applied to and linked to game engines such as Unity and Unreal, and connects voice recognition, STT, and translation systems to provide global metaverse users. communication can be activated.

One or more processors 110 may convert the emotional base response signal into a voice output signal. According to an embodiment, the processor 110 may convert the emotional base response signal into a voice output signal, and output the converted voice output signal to the user through the speaker 150 .

One or more memories 120 may store various data. Data stored in the memory 120 is data obtained, processed, or used by at least one component of the electronic device 100 for providing artificial intelligence conversations, and includes software (eg, commands, programs, etc.) can include Memory 120 may include volatile and/or non-volatile memory. In the present invention, a command or program is software stored in the memory 120, and an operating system, application, and/or application for controlling the resources of the electronic device 100 for providing artificial intelligence conversations can utilize the resources of the electronic device. It can include middleware that provides various functions to applications.

One or more memories 120 may store the above-described user's voice signal, text signal, keyword, knowledge base response signal, emotion base response signal, audio output signal, and the like. Also, the one or more memories 120 may store instructions that, when executed by the one or more processors 110, cause the one or more processors 110 to perform operations.

According to an embodiment, the electronic device 100 for providing artificial intelligence conversations may further include a transceiver 130. The transceiver 130 may perform wireless or wired communication between the electronic device 100 for providing artificial intelligence conversations and a server, database, client devices, and/or other devices. For example, the transceiver 130 may transmit eMBB (enhanced mobile broadband), URLLC (Ultra Reliable Low-Latency Communications), MMTC (Massive Machine Type Communications), LTE (long-term evolution), LTE-A (LTE Advance), UMTS (Universal Mobile Telecommunications System), GSM (Global System for Mobile communications), CDMA (code division multiple access), WCDMA (wideband CDMA), WiBro (Wireless Broadband), WiFi (wireless fidelity), Bluetooth (Bluetooth), NFC ( Wireless communication may be performed according to a method such as near field communication (GPS), global positioning system (GPS), or global navigation satellite system (GNSS). For example, the transceiver 130 may perform wired communication using a universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard 232 (RS-232), or plain old telephone service (POTS). there is.

According to one embodiment, one or more processors 110 may obtain information from a server and a database by controlling the transceiver 130 . Information obtained from the server and database may be stored in one or more memories 120 . As an example, the information obtained from the server and the database may include information required for keyword matching to form a knowledge base response signal.

According to an embodiment, the electronic device 100 for providing artificial intelligence conversations may be various types of devices. For example, the electronic device 100 for providing artificial intelligence conversations may be a portable communication device, a computer device, or a device according to one or more combinations of the above devices. The electronic device 100 for providing artificial intelligence conversations according to the present invention is not limited to the above devices.

Various embodiments of the electronic device 100 for providing artificial intelligence conversations according to the present invention may be combined with each other. Each embodiment may be combined according to the number of cases, and the combined embodiment of the electronic device 100 for providing artificial intelligence conversations also falls within the scope of the present invention. In addition, internal/external components of the electronic device 100 according to the present invention described above may be added, changed, replaced, or deleted according to embodiments. In addition, the aforementioned internal/external components of the electronic device 100 for providing artificial intelligence conversations may be implemented as hardware components.

In the present invention, artificial intelligence (AI) means a technology that imitates human learning ability, reasoning ability, perception ability, etc., and implements it with a computer, and concepts such as machine learning and symbolic logic can include Machine learning (ML) is an algorithm technology that classifies or learns the characteristics of input data by itself. Artificial intelligence technology is a machine learning algorithm that analyzes input data, learns the result of the analysis, and can make judgments or predictions based on the result of the learning. In addition, technologies that mimic the functions of the human brain, such as recognition and judgment, using machine learning algorithms, can also be understood as artificial intelligence. For example, technical fields such as linguistic understanding, visual understanding, inference/prediction, knowledge expression, and motion control may be included.

Machine learning may refer to processing that trains a neural network model using experience of processing data. Through machine learning, computer software could mean improving its own data processing capabilities. A neural network model is constructed by modeling a correlation between data, and the correlation may be expressed by a plurality of parameters. The neural network model derives a correlation between data by extracting and analyzing features from given data, and optimizing the parameters of the neural network model by repeating this process can be referred to as machine learning. For example, a neural network model may learn a mapping (correlation) between an input and an output with respect to data given as an input/output pair. Alternatively, even when only input data is given, the neural network model may learn the relationship by deriving a regularity between given data.

An artificial intelligence learning model or a neural network model may be designed to implement a human brain structure on a computer, and may include a plurality of network nodes having weights while simulating neurons of a human neural network. A plurality of network nodes may have a connection relationship between them by simulating synaptic activities of neurons that transmit and receive signals through synapses. In the artificial intelligence learning model, a plurality of network nodes can send and receive data according to a convolutional connection relationship while being located in layers of different depths. The artificial intelligence learning model may be, for example, an artificial neural network model, a convolutional neural network model (CNN), and the like. As an embodiment, the artificial intelligence learning model may be machine-learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms for performing machine learning include Decision Tree, Bayesian Network, Support Vector Machine, Artificial Neural Network, Ada-boost , Perceptron, Genetic Programming, Clustering, etc. may be used.

Of these, CNNs are a type of multilayer perceptrons designed to use minimal preprocessing. A CNN consists of one or several convolution layers and general artificial neural network layers on top, and additionally utilizes weights and pooling layers. Thanks to this structure, CNN can fully utilize the input data with a two-dimensional structure. Compared to other deep learning structures, CNN shows good performance in both video and audio fields. CNNs can also be trained via standard back-propagation. CNNs are easier to train than other feedforward artificial neural network techniques and have the advantage of using fewer parameters.

Convolutional networks are neural networks that contain sets of nodes with bound parameters. The increasing size of available training data and the availability of computational power, combined with algorithmic advances such as piecewise linear unit and dropout training, have greatly improved many computer vision tasks. With huge data sets, such as those available for many tasks today, overfitting is not critical, and increasing the size of the network improves test accuracy. Optimal use of computing resources becomes a limiting factor. To this end, a distributed, scalable implementation of deep neural networks can be used.

2 is an exemplary diagram showing the configuration of an artificial intelligence conversation providing system according to an embodiment of the present invention.

As shown in FIG. 2, the artificial intelligence dialogue providing system 200 includes a game engine 210, an A/D converter 220, a noise removal unit 230, a feature extraction unit 240, and a vocabulary dictionary 250. ), a text conversion unit 260, a rule-based dialogue system 270, and an AI dialogue system 280.

The game engine 210 is used for various developments such as mobile apps, AR, and VR, in addition to its original purpose of game development. According to an embodiment, the electronic device 100 for providing artificial intelligence conversations of the present invention is manufactured as a general-purpose platform that can be applied to and linked to game engines such as Unity and Unreal, and connects voice recognition, STT, and translation systems. By doing so, communication between users of the global metaverse can be activated.

The A/D converter 220 may sample an audio signal received from a user (UE) and convert it into a digital signal. According to an embodiment, the A/D converter 220 may convert a voice signal, which is an analog signal, into a digital signal recognizable by an electronic device.

The noise removal unit 230 may remove a noise signal from the digital signal converted by the A/D converter 220 . According to an embodiment, the noise removal unit 230 may remove a noise signal from a digital signal by performing band filtering.

The feature extractor 240 may extract keywords (features) by removing postpositions, punctuation marks, spaces, etc. after removing noise signals from the digital signal, and converts the extracted keywords into words stored in the vocabulary dictionary 250. It can be.

The text conversion unit 260 may form a text signal composed of words stored in the vocabulary dictionary 250 .

The rule-based dialogue system 270 may form a knowledge base response signal by performing database matching using keywords extracted from text signals.

The AI conversation system 280 may use the knowledge base response signal to form an emotion-based response signal in consideration of the user's age, personality, intimacy, and the like. The emotional base response signal may be converted into a voice output signal and output to the user UE through the speaker 150 .

3 is a diagram for explaining learning of a neural network according to an embodiment of the present invention.

As shown in FIG. 3 , the learning device may train the neural network 114 to recognize feature points included in the image of the object. According to an embodiment, the learning device may be a separate subject different from the electronic device 100 for providing artificial intelligence conversations, but is not limited thereto.

The neural network 114 includes an input layer 112 into which training samples are input and an output layer 116 which outputs training outputs, and can be learned based on differences between the training outputs and labels. Here, the labels may be defined based on the user's response level corresponding to the audio output signal. The neural network 114 is connected to a group of a plurality of nodes, and is defined by weights between connected nodes and an activation function that activates the nodes.

The learning device may train the neural network 114 using a gradient descent (GD) technique or a stochastic gradient descent (SGD) technique. The learning device may use a loss function designed by outputs and labels of the neural network.

The learning device may calculate a training error using a predefined loss function. The loss function may be predefined with labels, outputs and parameters as input variables, where the parameters may be set by weights in the neural network 114. For example, the loss function may be designed in a mean square error (MSE) form, an entropy form, and the like, and various techniques or methods may be employed in an embodiment in which the loss function is designed.

The learning device may find weights affecting the training error using a backpropagation technique. Here, weights are relationships between nodes in neural network 114 . The learning device may use the SGD technique using labels and outputs to optimize the weights found through the backpropagation technique. For example, the learning device may update weights of a loss function defined based on labels, outputs, and weights using the SGD technique.

According to an embodiment, the learning device may obtain a voice signal of a training object and extract a training voice output signal from the audio signal of the training object. The learning device may acquire pre-labeled information (first labels) for each of the training audio output signals, and the user's reaction degree (e.g., corresponding to a predefined audio output signal) to the training audio output signals. , reactivity high, medium, low, etc.) may be obtained.

According to an embodiment, the learning device may generate first training feature vectors based on appearance features, pattern features, and tone features of the training speech output signals. Various methods may be employed to extract features of the training speech output signals.

According to an embodiment, the learning device may obtain training outputs by applying the first training feature vectors to the neural network 114 . The learning device may train the neural network 114 based on the training outputs and the first labels. The learning device may train the neural network 114 by calculating training errors corresponding to training outputs and optimizing a connection relationship of nodes in the neural network 114 to minimize the training errors. The electronic device 110 may form a voice output signal from a voice signal using the neural network 114 that has been learned.

4 and 5 are flowcharts showing procedures of an artificial intelligence conversation providing method according to an embodiment of the present invention. Although process steps, method steps, algorithms, etc. are described in a sequential order in the flowcharts of FIGS. 4 and 5, such processes, methods, and algorithms may be configured to operate in any suitable order. In other words, steps of processes, methods, and algorithms described in various embodiments of the present invention need not be performed in the order described herein. Also, although some steps are described as being performed asynchronously, in other embodiments some of these steps may be performed concurrently. Further, illustration of a process by depiction in the drawings does not mean that the illustrated process is exclusive of other changes and modifications thereto, and that any of the illustrated process or steps thereof may be one of various embodiments of the present invention. It is not meant to be essential to one or more, and it does not imply that the illustrated process is preferred.

As shown in FIG. 4, in step S410, a user's voice signal is received. For example, referring to Figures 1 and 2, the processor 110 of the electronic device 100 for providing artificial intelligence conversations may receive a user's voice signal. According to an embodiment, the processor 110 may receive a user's voice signal through the microphone 140 of the electronic device 100 for providing artificial intelligence conversation.

In step S420, the voice signal is converted into a text signal. For example, referring to Figure 1 to Figure 2, the processor 110 of the electronic device 100 for providing artificial intelligence conversation may convert the user's voice signal received in step S410 into a text signal. According to an embodiment, the processor 110 samples the received voice signal by an A/D (Analog Digital) converter, converts it into a digital signal, and then removes the noise signal to convert it into a text signal. .

In step S430, keywords are extracted by performing natural language processing on the text signal. For example, referring to Figure 1 to Figure 3, the processor 110 of the electronic device 100 for providing artificial intelligence conversation extracts features useful for voice recognition from the text signal converted in step S420 to generate a feature vector. And, the most similar words can be extracted as keywords by referring to a dictionary composed of words to be recognized prepared in advance in the learning process for the generated feature vectors.

In step S440, a knowledge base response signal is formed by matching with a database using the extracted keyword. For example, referring to 　Do　1 to 3, the processor 110 of the electronic device 100 for providing artificial intelligence conversation receives a plurality of voice data according to 　gender, 　region, 　age, and multiple speeches, and A database may be updated using voice data of , and a knowledge base response signal may be formed by performing database matching using a text signal.

In step S450, an emotion-based response signal is formed in consideration of the user's age, personality, and intimacy using the knowledge-base response signal. For example, referring to Figures 1 to 3, the processor 110 of the electronic device 100 for providing artificial intelligence conversation is machine learning, in particular, deep reinforcement learning such as deep learning. An emotion-based response signal may be formed using an algorithm.

In step S460, the emotional base response signal is converted into a voice output signal. For example, referring to Figures 1 to 3, the processor 110 of the electronic device 100 for providing artificial intelligence conversation uses a TTS (Text to Speech) recognition engine to transmit an emotional base response signal to a voice output signal. , and the audio output signal can be output to the user UE through the speaker 150.

As shown in Fig. 5, in step S510, a plurality of voice data is received. For example, referring to Figures 1 to 4, the processor 110 of the electronic device 100 for providing artificial intelligence conversations may receive a plurality of voice data according to gender, region, age, and multiple speeches. . According to an embodiment, the processor 110 may receive a plurality of voice data through various paths such as the Internet and Social Network Service (SNS).

In step S520, an update of the database is performed. For example, referring to FIGS. 1 to 4 , the processor 110 of the electronic device 100 for providing an artificial intelligence conversation may update a database using a plurality of voice data received in step S510. there is. According to an embodiment, the processor 110 may classify a plurality of received voice data according to gender, region, age, multivocalization, etc., and continuously update them in a database.

In step S530, a knowledge base response signal is formed. For example, referring to Figure 1 to Figure 4, the processor 110 of the electronic device 100 for providing artificial intelligence conversation performs matching with the updated database in step S520 using the keyword extracted in step S430. Thus, a knowledge base response signal may be formed.

Although the method has been described through specific embodiments, the method can also be implemented as computer readable code on a computer readable recording medium. A computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. In addition, the computer-readable recording medium is distributed in computer systems connected through a network, so that computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the above embodiments can be easily inferred by programmers in the art to which the present invention belongs.

In the above, the present invention has been described in detail using preferred embodiments, but the scope of the present invention is not limited to specific embodiments, and should be interpreted according to the appended claims. In addition, those skilled in the art should understand that many modifications and variations are possible without departing from the scope of the present invention.

100: electronic device 110: processor
120: memory 130: transceiver
140: microphone 150: speaker
200: artificial intelligence dialogue providing system 210: game engine
220: A / D converter 230: noise canceling unit
240: feature extraction unit 250: vocabulary dictionary
260: text conversion unit 270: rule-based dialog system
280: AI dialog system 112: Input layer
114: neural network 116: output layer

Claims (2)

In an electronic device for providing artificial intelligence conversation,
one or more processors; and
one or more memories storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations;
The one or more processors,
Receive a user's voice signal;
Converting the voice signal into a text signal;
extracting keywords by performing natural language processing of the text signal;
Forming a knowledge base response signal by performing database matching using the extracted keywords;
Forming an emotion base response signal in consideration of the user's age, personality, and intimacy using the knowledge base response signal;
Converting the emotional base response signal into a voice output signal;
The one or more processors,
assigning a weight to the audio output signal according to the user's response level using the user's feedback on the audio output signal;
When the user shows a positive response to a specific audio output signal and shows no response or negative response to another audio output signal,
A relatively high weight is assigned to a specific audio output signal to which the user has responded positively;
Updating the database by using the specific audio output signal to which the relatively high weight is assigned so that the specific audio output signal to which the relatively high weight is assigned can be considered in forming the emotion base response signal;
The one or more processors induce appropriate answers through voice conversation history data and perform keyword training,
A plurality of voice data according to gender, region, age, and multilateral speech is received, the database is updated using the plurality of voice data, and the database is matched using the extracted keyword to obtain the knowledge If it is impossible to match the search-based model forming the base response signal with the extracted keyword and the database, it is determined that the knowledge base response signal cannot be formed, and the extracted keyword is determined to be impossible to form the knowledge base based on the extracted keyword using a machine learning algorithm. using a generative model that forms a response signal;
The one or more processors derive a 5G cloud immersive content architecture, additionally reflect native services provided by cloud operators based on the existing architecture,
The one or more processors provide a voice text to speech (TTS) that matches a real human voice, an electronic device for providing artificial intelligence conversations.
According to claim 1,
The one or more processors,
An electronic device for providing artificial intelligence conversation, which performs natural language processing on the received voice signal through sentence separation, spacing correction, spelling correction, abbreviation expansion, and symbol removal procedures for the voice signal.

Images