KR102309505B1 - User-customized augmentative and alternative communication apparatus using speech recognition and intelligence and method thereof - Google Patents

Abstract

The present invention relates to a personalized complementary and alternative communication device using voice recognition and artificial intelligence learning and a method thereof. The device comprises: a voice acquisition module for acquiring user's voice information data; a display module for displaying at least one text information data on a display screen; a storage module for storing the user's voice information data and corresponding text information data into a database (DB); and a control module for controlling the voice acquisition module, the display module, and the storage module. The control module performs voice recognition on user's voice information data, compares and analyzes the user's voice information data whose voice recognition has been completed and consistent user's voice information data stored in the storage module to search consistent voice information data identical to the user's voice information data whose voice recognition has been completed, and calls text information data matched with the searched data to be displayed on a display screen, thereby having an effect of enabling rapid communication.

Description

USER-CUSTOMIZED AUGMENTATIVE AND ALTERNATIVE COMMUNICATION APPARATUS USING SPEECH RECOGNITION AND INTELLIGENCE AND METHOD THEREOF

The present invention relates to a supplementary and alternative communication device and method customized for each individual using speech recognition and learning of artificial intelligence.

In general, augmentative and alternative communication (AAC) is a method that helps disabled people with congenital or acquired language disorders to communicate smoothly, for example, speech, vocalization, gestures, and communication behaviors. , means any communication method available to an individual, including a particular communication method or tool.

For example, they range from simple technology (no tech) complementary and alternative communication (AAC) such as symbols or paints to high tech supplementary and alternative communication (AAC) such as smartphones or dedicated terminals.

At this time, in the case of simple technology complementary and alternative communication (AAC), although the price is very low, there is a big limit to various and complex communication, whereas high technology supplemental and alternative communication (AAC) makes relatively specific and rapid communication possible. However, the reality is that it is a burden in terms of cost.

In addition, depending on the degree of disability, there may be restrictions on the complementary and alternative communication (AAC) that can be used. The speech-impaired person can communicate with the other party using a terminal or smartphone equipped with a text to speech (TTS) function.

On the other hand, people with disabilities (eg, cerebral palsy) who have both speech and quadriplegia have considerable difficulties in communicating with others with their own voice due to inaccurate pronunciation, and at the same time cannot use their hands freely, so even if they use the TTS function, text messages If it takes too long to type, it is very difficult to communicate quickly, or if you have severe quadriplegia, you cannot use the tool at all.

Therefore, the communication of the disabled must be possible only with a helper, such as a guardian or a teacher, to help them communicate, which is a reality that greatly limits their independent life.

The pronunciation of these people with cerebral palsy is very difficult for the general public to understand, but those who have lived with them for a long time, such as family members or teachers, understand well. This is to learn the meaning of the pronunciation of the people with cerebral palsy through repeated pattern learning that matches over a long period of time.

Therefore, if the pronunciation and meaning of the person with dysphonia are matched and stored, and the word is retrieved from the storage when the person with a dysphonia speaks later, it can be expressed through the speaker using the text display or TTS function, Accurate communication will be possible.

Domestic Registered Patent No. 10-1169249 (Announced on Aug. 6, 2012)

The present invention has been devised to solve the above problems, and an object of the present invention is to store the voice of a disabled person with dysphonia and quadriplegia, input the corresponding meaning to match it, and then enable the disabled person to use the word or When a sentence is spoken, the voice recognition function recognizes it and displays the word or sentence on the display screen or uses the Text to Speech (TTS) function to express it in normal language through a speaker, etc. An object of the present invention is to provide a supplementary and alternative communication device and method customized for each individual using the learning of voice recognition and artificial intelligence.

In order to achieve the above object, a first aspect of the present invention includes: a voice acquisition module for acquiring voice information data of a user; a display module for displaying at least one text information data on a display screen; a storage module for storing the user's voice information data obtained from the voice acquisition module and text information data corresponding to the database (DB); and controlling the operation of the display module so that at least one text information data is displayed on the display screen through a preset artificial intelligence learning mode, and the user's voice information data corresponding to the text information data displayed on the display screen is displayed a plurality of times. controls the operation of the voice acquisition module so that it is repeatedly acquired with It is to provide a personalized complementary and alternative communication device using voice recognition and artificial intelligence learning, including a control module that controls the consistent user's voice information data to be matched with each other and stored in the storage module.

Here, the control module receives the user's voice information data obtained from the voice acquisition module and performs voice recognition on the acquired user's voice information data using a preset voice recognition technique, and the voice recognition is completed After comparing and analyzing the user's voice information data and the consistent user's voice information data stored in the storage module to search for the same consistent voice information data as the voice information data of the user whose voice recognition has been completed, the searched identical consistent voice information It is preferable to control the operation of the display module so that the character information data matched with the data is called and displayed on the display screen.

Preferably, a voice output module for outputting voice information data may be further included.

Preferably, the control module calls the text information data matched with the searched identical consistent voice information data, and converts the called text information data through a preset text-to-speech (TTS) technique. The operation of the voice output module may be controlled so that it is converted into voice and output to the outside.

Preferably, the preset artificial intelligence learning mode is based on at least one of a neural network, a support vector machine (SVM), a multi layer perception (MLP), or deep learning. Machine learning methods can be applied.

Preferably, a user input module for outputting a specific input signal by a user's manipulation may be further included.

Preferably, the control module controls the operation of the voice acquisition module so that the voice information data of the corresponding user is repeatedly acquired a plurality of times through a preset user voice input mode, and uses the preset voice recognition technique. After confirming whether the user's voice information data repeatedly acquired a plurality of times from the voice acquisition module is consistent, the character information data corresponding to the user's voice information data consistent by a specific input signal output from the user input module is provided, and based on this, the consistent user's voice information data and the text information data provided from the user input module are matched with each other, and it can be controlled to be stored in the storage module.

Preferably, the control module compares and analyzes the voice information data of the voice recognition completed with the consistent user's voice information data stored in the storage module, and fails to search for the same consistent voice information data as the voice information data on which the voice recognition has been completed. In this case, it is possible to control the operation of the display module so that the voice information data of the voice recognition completed and the text information data matched in the order of the highest voice similarity are displayed on the display screen in the form of a list.

Preferably, the control module selects any one of the text information data from among the text information data matched in the order of the high voice similarity to the voice information data that has been recognized by the user by using a separate user input module, , while controlling the operation of the display module so that the selected text information data is displayed on the display screen, the selected text information data is converted into voice through a preset text-to-speech (TTS) technique It is possible to control the operation of a separate voice output module so that it is output as .

Preferably, the control module may control all functions related to supplementary and alternative communication customized for each individual to be performed through the execution of a pre-installed individual customized supplementary and alternative communication related application.

Preferably, the control module receives the user's unique identification information for authentication of the corresponding user when the individual customized complementary and alternative communication related application is executed, and compares it with preset user identification information based on this to authenticate the user After completion, it is possible to control all functions related to complementary and alternative communication tailored to each individual to be performed.

Preferably, a communication module for transmitting text information data corresponding to and matching with the user's voice information data obtained from the voice acquisition module for each user to the outside may be further included.

Preferably, the control module is configured to match text information data displayed on the display screen together with preset user's unique identification information and consistent user's voice information data corresponding to each other through wired/wireless communication to an external terminal or server It is possible to control the operation of the communication module to be transmitted to.

Preferably, the external terminal or server receives, through the communication module, consistent user's voice information data corresponding to and matched with the text information data displayed on the display screen together with the user's unique identification information, based on this As a result, consistent user voice information data corresponding to and matched with text information data for each user can be stored and managed as a database (DB).

Preferably, the control module compares the voice information data of the voice recognition completed with the consistent user's voice information data stored in the storage module. , the voice information data on which the voice recognition has been completed may be transmitted to an external terminal or server through a separate communication module.

Preferably, the external terminal or server receives the voice information data of the voice recognition completed through the communication module, and based on the received voice information data, the same consistent voice information data as the voice information data of the voice recognition completion through a separate voice information DB may be searched for and transmitted to the communication module by calling the text information data matched with the searched identical consistent voice information data.

Preferably, the at least one text information data may be formed of at least one of a word or a sentence.

A second aspect of the present invention is a method for personalized complementary and alternative communication using a device including a voice acquisition module, a display module, a storage module, and a control module, (a) at least one character through the display module displaying information data on a display screen; (b) repeatedly acquiring the user's voice information data corresponding to the text information data displayed in step (a) a plurality of times through the voice acquisition module; (c) checking whether the user's voice information data repeatedly acquired a plurality of times in step (b) is consistent using an artificial intelligence learning mode preset through the control module; (d) matching the text information data displayed in step (a) with the voice information data of the user consistent in step (c) through the control module to correspond to each other and storing in the storage module; (e) performing voice recognition on the user's voice information data obtained from the voice acquisition module using a preset voice recognition technique through the control module; (f) through the control module, the voice information data of the user whose voice recognition is completed in step (e) and the consistent user's voice information data stored in the step (d) are compared and analyzed, and the voice recognition is completed in step (e) retrieving the same consistent voice information data as the user's voice information data; and (g) calling text information data matched with the same consistent voice information data retrieved in step (f) through the control module and displaying it on the display screen of the display module. It is to provide a complementary and alternative communication method customized for each individual using the learning of recognition and artificial intelligence.

Here, after the step (g), (h) calling the text information data matching the same consistent voice information data searched for in the step (f) through the control module, and automatically converting a preset text-to-speech (Text) to Speech, TTS), after converting the called text information data into voice, outputting the converted voice through a separate voice output module.

Preferably, the artificial intelligence learning mode preset in step (c) is at least one artificial intelligence among a neural network, a support vector machine (SVM), a multi-layer perception (MLP), or deep learning. (AI)-based machine learning methods can be applied.

Preferably, before or after the step (a) or after the step (d), a first step of repeatedly acquiring the user's voice information data through the voice acquisition module a plurality of times; a second step of confirming whether the user's voice information data repeatedly acquired a plurality of times in the first step using a preset voice recognition technique through the control module is consistent; a third step of receiving, through the control module, text information data corresponding to the consistent user's voice information data confirmed in the second step according to a specific input signal output from a separate user input module; and the consistent user's voice information data confirmed in the second step based on the text information data corresponding to the consistent user's voice information data provided in the third step through the control module and the text provided in the third step The method may further include a fourth step of matching information data with each other and storing the information data in the storage module.

Preferably, after the step (f), the voice information data of the voice recognition completed in the step (e) and the consistent user's voice information data stored in the step (d) are compared and analyzed through the control module in the step ( When it is not possible to search for the same consistent voice information data as the voice information data of the voice recognition completed in e), the text information data that is matched in the order of the highest voice similarity with the voice information data of the voice recognition completed in step (e) is called The method may further include displaying on the display screen of the display module in the form of a list.

Preferably, through the control module, using a separate user input module, the voice information data that has been recognized by the user in the step (e) and any one character of the character information data matched in the order of the highest voice similarity After selecting information data, the selected text information data is displayed on the display screen of the display module, and at the same time, the selected text information data is converted into voice using a preset text-to-speech (TTS) technique. Then, the method may further include outputting the converted voice through a separate voice output module.

Preferably, the steps (a) to (g) may be performed through the execution of a pre-installed personalized complementary and alternative communication related application under the control of the control module.

Preferably, when the individual customized complementary and alternative communication-related application is executed through the control module, the user-specific identification information preset based on the user-specific identification information for authentication of the corresponding user input through a separate user input module and By comparison, after the authentication of the corresponding user is completed, the steps (a) to (g) may be performed.

Preferably, before or after the step (d), the text information data displayed in step (a) together with the user's unique identification information preset through the control module and the user's voice information consistent in the step (c) The method may further include matching the data so that they correspond to each other and transmitting the data to an external terminal or server using a separate communication module.

Preferably, the user consistent in step (c) corresponding to and matching the character information data displayed in step (a) together with the user's unique identification information transmitted from the communication module through the external terminal or server It may further include the step of storing and managing the consistent user's voice information data corresponding to and matched with text information data for each user based on the voice information data of the database (DB), and storing and managing the same.

Preferably, after the step (f), the voice information data of the voice recognition completed in the step (e) and the consistent user's voice information data stored in the step (d) are compared and analyzed through the control module in the step ( When it is not possible to search for the same consistent voice information data as the voice information data for which voice recognition has been completed in e), transmitting the voice information data for which voice recognition has been completed in step (e) to an external terminal or server through a separate communication module may further include.

Preferably, based on the voice information data of the voice recognition completed in step (e) transmitted from the communication module through the external terminal or server, the voice information of the voice recognition completed in step (e) through a separate voice information DB The method may further include, after searching for the same consistent voice information data as the data, calling text information data matched with the searched identical consistent voice information data and transmitting the same to the communication module.

Preferably, in the step (a), the at least one text information data may be formed of at least one of a word or a sentence.

A third aspect of the present invention provides a computer-readable recording medium in which a program capable of executing a supplementary and alternative communication method customized for each individual using the above-described speech recognition and learning of artificial intelligence is recorded.

The personalized complementary and alternative communication method using voice recognition and artificial intelligence learning according to the present invention can be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored.

For example, computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, hard disk, floppy disk, removable storage device, and non-volatile memory (Flash Memory). , and optical data storage devices.

According to the personalized complementary and alternative communication device and method using speech recognition and artificial intelligence learning of the present invention as described above, the voice of the disabled person with dysphonia and quadriplegia is stored and the corresponding meaning is inputted. After matching, when a person with a disability utters the word or sentence in daily life, the speech recognition function recognizes the word or sentence and displays the word or sentence on the display screen, or through a speaker using the Text to Speech (TTS) function. It has the advantage of being able to communicate quickly by expressing it in language.

1 is an overall block diagram for explaining a supplementary and alternative communication device customized for each individual using speech recognition and learning of artificial intelligence according to an embodiment of the present invention.
2 and 3 are overall flowcharts for explaining a supplementary and alternative communication method customized for each individual using speech recognition and learning of artificial intelligence according to an embodiment of the present invention.

The above-described objects, features and advantages will be described below in detail with reference to the accompanying drawings, and accordingly, those of ordinary skill in the art to which the present invention pertains will be able to easily implement the technical idea of the present invention. In describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Terms including an ordinal number, such as first, second, etc., may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise.

The terms used in the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, but these may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, and the like. In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

When a part "includes" a certain element throughout the specification, this means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit" and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiments of the present invention illustrated below may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. The embodiments of the present invention are provided to more completely explain the present invention to those of ordinary skill in the art.

Each block in the accompanying block diagram and combinations of steps in the flowchart may be executed by computer program instructions (execution engine), which computer program instructions may be executed by a processor of a general-purpose computer, special-purpose computer, or other programmable data processing equipment. It may be mounted so that the instructions, which are executed by the processor of a computer or other programmable data processing equipment, create means for performing the functions described in each block of the block diagram or in each step of the flowchart. These computer program instructions may also be stored in a computer usable or computer readable memory which may direct a computer or other programmable data processing equipment to implement a function in a particular manner, and thus It is also possible for the instructions stored in the block diagram to produce an article of manufacture containing instruction means for performing a function described in each block of the block diagram or each step of the flowchart.

And, since the computer program instructions may be mounted on a computer or other programmable data processing equipment, a series of operating steps are performed on the computer or other programmable data processing equipment to create a computer-executed process to create a computer or other program It is also possible that instructions for performing the possible data processing equipment provide steps for carrying out the functions described in each block of the block diagram and each step of the flowchart.

In addition, each block or step may represent a module, segment, or portion of code comprising one or more executable instructions for executing specified logical functions, and in some alternative embodiments the blocks or steps referred to in the block or steps. It should be noted that it is also possible for functions to occur out of sequence. For example, it is possible that two blocks or steps shown one after another may be performed substantially simultaneously, and also the blocks or steps may be performed in the reverse order of the corresponding functions, if necessary.

1 is an overall block diagram for explaining a supplementary and alternative communication device customized for each individual using speech recognition and learning of artificial intelligence according to an embodiment of the present invention.

Referring to FIG. 1 , a supplementary and alternative communication device customized for each individual using voice recognition and learning of artificial intelligence according to an embodiment of the present invention is largely a voice acquisition module 100 , a display module 200 , and a storage module 300 . ), a control module 400, and a power supply module 500, and the like. In addition, the personalized complementary and alternative communication device using voice recognition and artificial intelligence learning according to an embodiment of the present invention is a voice output module 600 , a user input module 700 , and/or a communication module 800 . and the like may be further included. On the other hand, since the components shown in Fig. 1 are not essential, the personalized complementary and alternative communication device using speech recognition and artificial intelligence learning according to an embodiment of the present invention has more or fewer components than that. It may have elements.

Hereinafter, the components of the personalized complementary and alternative communication device using voice recognition and artificial intelligence learning according to an embodiment of the present invention will be described in detail as follows.

The voice acquisition module 100 performs a function of acquiring the user's voice information data. The voice acquisition module 100 is preferably implemented as a conventional microphone (Microphone).

The display module 200 performs a function of displaying at least one piece of text information data on the display screen. Here, it is preferable that the at least one text information data includes, for example, at least one of a word and/or a sentence.

The display module 200 includes, for example, a liquid crystal display (LCD), a light emitting diode display (LED), a thin film transistor liquid crystal display (TFT LCD), and an organic light emitting diode. (Organic Light Emitting Diode, OLED), Flexible Display, Plasma Display Panel (PDP), Surface Alternate Lighting (ALiS), Digital Light Emitting Diode (DLP), Silicon Liquid Crystal (LCoS), Surface Conduction Electron Emitting Device Display (SED), Field Emission Display (FED), Laser TV (Quantum Dot Laser, Liquid Crystal Laser), Optoelectric Liquid Display (FLD), Interferometric Modulator Display (iMoD), Thick Film Dielectric Electric (TDEL), At least one of a quantum dot display (QD-LED), a telescopic pixel display (TPD), an organic light emitting transistor (OLET), a laser fluorescence display (LPD), a touch screen, and/or a three-dimensional display (3D display) It may include, but is not limited thereto, and any data that can display at least one of image, text, and document data may be included.

The storage module 300 performs a function of storing the user's voice information data obtained from the voice acquisition module 100 and text information data corresponding to the database (DB).

In addition, the storage module 300 may perform a function of storing, for example, preset user unique identification information. Here, the user's unique identification information is, for example, the user's ID, password, user's name, resident number, phone number, address, biometric information, PKI (Public Key Infrastructure), OTP (One Time Password) ), and/or it is preferable to include at least one of the public certificate information.

In addition, the storage module 300 may perform a function of storing and maintaining at least one program code executed through the control module 400 and at least one data set used by the program code.

The storage module 300 may include, for example, a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (eg, SD or XD memory). etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), It may include at least one type of storage medium among a magnetic memory, a magnetic disk, and an optical disk.

The control module 400 is a module responsible for overall control of the personalized complementary and alternative communication device using voice recognition and artificial intelligence learning according to an embodiment of the present invention, and in particular, at least one through a preset artificial intelligence learning mode. It performs a function of controlling the operation of the display module 200 so that the character information data of the display module 200 is displayed on the display screen.

In addition, the control module 400 functions to control the operation of the voice acquisition module 100 so that the user's voice information data corresponding to the text information data displayed on the display screen of the display module 200 is repeatedly acquired a plurality of times. carry out

In addition, the control module 400 checks whether the user's voice information data repeatedly acquired a plurality of times from the voice acquisition module 100 is consistent with the text information data displayed on the display screen of the display module 200 . It performs a function of controlling the consistent user's voice information data to be matched with each other and stored in the storage module 300 .

Here, the preset artificial intelligence learning mode is, for example, at least one of a neural network, a support vector machine (SVM), a multi-layer perception (MLP), and/or deep learning (AI). It is desirable to apply a machine learning-based method.

In addition, the control module 400 receives the user's voice information data acquired from the voice acquisition module 100, and uses a preset voice recognition technique (eg, artificial intelligence-based voice recognition technology, etc.) of the acquired user. Voice recognition is performed on the voice information data, and the voice information data of the user who has completed the voice recognition and the consistent user's voice information data stored in the storage module 300 are compared and analyzed to be the same as the voice information data of the user who has completed the voice recognition. After searching for consistent voice information data, the storage module 300 calls the text information data matched with the searched identical consistent voice information data to be displayed on the display screen of the display module 200 . function to control the operation of

In addition, the control module 400 calls the text information data matched with the searched identical consistent voice information data from the storage module 300, and through a preset text-to-speech (TTS) technique, the It is possible to perform a function of controlling the operation of the voice output module 600 so that the called text information data is converted into voice and outputted to the outside.

In addition, the control module 400 controls the operation of the voice acquisition module 100 so that the user's voice information data is repeatedly acquired a plurality of times through a preset user voice input mode, and uses a preset voice recognition technique. After confirming the consistency of the user's voice information data repeatedly acquired a plurality of times from the voice acquisition module 100, the consistent user's voice information data and A function of controlling to be stored in the storage module 300 by matching the text information data provided from the user input module 700 with the consistent user's voice information data based on the received text information data. can

In addition, the control module 400 compares and analyzes the voice recognition completed voice information data and the consistent user's voice information data stored in the storage module 300 to search for the same consistent voice information data as the voice recognition completed voice information data. If not, the storage module 300 calls the text information data matched in the order of the voice recognition completed voice information data and the voice similarity to be displayed on the display screen of the display module 200 in the form of a list. A function of controlling the operation of the display module 200 may be performed.

In addition, the control module 400 selects any one of the text information data from the text information data matched in the order of the highest voice similarity to the voice information data that has been recognized by the user by using the user input module 700 . and control the operation of the display module 200 so that the selected text information data is displayed on the display screen of the display module 200, and at the same time, the selected text through a preset text-to-speech (TTS) technique It is possible to perform a function of controlling the operation of the voice output module 600 so that the information data is converted into voice and outputted to the outside.

In addition, the control module 400 may control all functions related to supplementary and alternative communication customized for each individual to be performed through the execution of a previously installed application related to supplementary and alternative communication for each individual.

In addition, the control module 400 receives the user's unique identification information (eg, ID and password, etc.) for authentication of the corresponding user when the individual customized complementary and alternative communication related application is executed and based on this It is possible to control so that all functions related to supplementary and alternative communication tailored to each individual are performed after the user's authentication is completed by comparing it with the user's unique identification information set in advance.

In addition, the control module 400 matches the character information data displayed on the display screen of the display module 200 together with the preset user's unique identification information and the consistent user's voice information data corresponding to each other through wired/wireless communication. A function of controlling the operation of the communication module 800 to be transmitted to an external terminal and/or server 20 may be performed.

In addition, the control module 400 compares the voice information data of the voice recognition completed with the consistent user's voice information data stored in the storage module 300 to search for the same consistent voice information data as the voice information data of the voice recognition completed. If not, a function of transmitting the voice information data on which the voice recognition has been completed to the external terminal and/or the server 20 through the communication module 800 may be performed.

Various embodiments described herein may be implemented in a computer-readable recording medium using, for example, software, hardware, or a combination thereof.

According to the hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs). , processors, controllers, micro-controllers, microprocessors, and may be implemented using at least one of an electrical unit for performing a function. In some cases, such embodiments may be implemented by the control module 400 .

According to the software implementation, embodiments such as a procedure or function may be implemented together with a separate software module for performing at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. In addition, the software code may be stored in the storage module 300 and executed by the control module 400 .

In addition, the power supply module 500 includes each of the above-described modules, that is, the voice acquisition module 100 , the display module 200 , the storage module 300 , the control module 400 , the voice output module 600 , and the user input. It performs a function of supplying power required for the operation of the module 700 and/or the communication module 800, etc., and preferably implemented with a conventional portable battery, but is not limited thereto, and continuous power supply For supply, commercial alternating current (AC) power (eg, AC 220V or 380V, etc.) may be implemented to be converted into direct current (DC) and/or alternating current (AC) power.

In addition, the power supply module 500 may include a power management unit (not shown) that protects components from external power shock and outputs a constant voltage. The power management unit may include an ESD (Electro Static Damage) protector, a power detector, a rectifier, and a power breaker.

Here, the ESD protector is configured to protect electronic components from static electricity or sudden power shock. The power detector is configured to send a cut-off signal to the power breaker when a voltage outside the allowable voltage range is introduced, and to transmit a step-up or step-down signal to the rectifier according to a voltage change within the allowable voltage range. The rectifier is configured to perform a step-up or step-down rectification operation according to the signal of the power sensor so that a constant voltage is supplied by minimizing the fluctuation of the input voltage. The power breaker is configured to cut off the power supplied from the battery according to the cut-off signal transmitted from the power detector.

Additionally, the voice output module 600 is a module for outputting voice information data, and performs a function of outputting a voice signal so that a person can aurally recognize the voice information data.

Such a voice output module 600 is preferably implemented as a conventional speaker (Speaker), but is not limited thereto, and for example, it may be implemented as a connection jack including a voice amplification circuit so that a person can listen through earphones or headphones. may be

The user input module 700 performs a function of outputting a specific input signal by a user's manipulation.

That is, the user input module 700 includes at least one key button to output a specific key input signal. It performs a function of outputting a key input signal corresponding to the key to the control module 400 .

The user input module 700 is preferably implemented as a key button, but is not limited thereto, and may include, for example, a keyboard, a mouse, and the like, and in some cases, a remote control.

The communication module 800 transmits text information data corresponding to and matching with the user's voice information data obtained from the voice acquisition module 100 for each user to an external terminal and/or server 20 through the communication network 10. perform the function

At this time, the communication network 10 is a communication network that is a high-speed backbone network of a large-scale communication network capable of large-capacity, long-distance voice and data services. It may be a next-generation wireless communication network including WiBro (Wireless Broadband Internet, Wibro) and Wimax (World Interoperability for Microwave Access, Wimax).

The Internet includes the TCP/IP protocol and various services existing in its upper layers, namely HTTP (Hyper Text Transfer Protocol), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), SMTP (Simple Mail Transfer Protocol), It means a worldwide open computer network structure that provides SNMP (Simple Network Management Protocol), NFS (Network File Service), NIS (Network Information Service), etc., and the communication module 800 is an external terminal and/or server 20 ) to provide an environment that allows access to Meanwhile, the Internet may be a wired or wireless Internet, or may be a core network integrated with a wired public network, a wireless mobile communication network, or the portable Internet.

If the communication network 10 is a mobile communication network, it may be a synchronous mobile communication network or an asynchronous mobile communication network. As an embodiment of the asynchronous mobile communication network, a wideband code division multiple access (WCDMA) type communication network may be used. In this case, although not shown in the drawings, the mobile communication network may include, for example, a Radio Network Controller (RNC). Meanwhile, although the WCDMA network is taken as an example, it may be a next-generation communication network such as a cellular-based 3G network, an LTE network, a 4G network, a 5G network, or other IP-based IP networks. The communication network 10 serves to mutually transmit signals and data between the communication module 800 and an external terminal and/or server 20 .

Moreover, the external terminal and/or server 20 corresponds to and matches the character information data displayed on the display screen of the display module 200 together with the unique identification information of each user through the communication module 800, consistent users It is possible to perform a function of storing and managing the user's voice information data that is consistent with corresponding and matched text information data for each user based on the received voice information data of the database (DB), stored and managed.

In addition, the external terminal and/or the server 200 receives the voice information data through the communication module 800, and based on the received voice information data, the voice recognition is completed through a separate voice information DB (not shown). It is possible to perform a function of searching for the same consistent voice information data as the data, calling the text information data matched with the searched identical consistent voice information data, and transmitting the same to the communication module 800 .

On the other hand, the external terminal and/or the server 20 includes various mobile terminals communicating through the wireless Internet or the portable Internet, and in addition, a Palm PC, a smart phone, a smart pad, and a user interface for accessing the communication module 800, such as a smart note, a mobile play-station, a DMB (Digital Multimedia Broadcasting) phone with a communication function, a tablet PC, and an iPad. It may comprehensively mean all wired and wireless home appliances/communication devices having.

In particular, when the external terminal and/or server 20 is implemented as a normal smart phone, the smart phone downloads various application programs desired by the user, unlike a general mobile phone (aka feature phone). It is a phone based on an open operating system that can be received, used, and deleted freely. It supports not only commonly used functions such as voice/video calls and Internet data communication, but also all mobile phones or voice calls with mobile office functions. However, it is preferable to understand it as a communication device including any Internet phone or tablet PC that can access the Internet.

As such, since the smart phone uses an open operating system, a user can install and manage various application programs arbitrarily, unlike a mobile phone having a closed operating system.

On the other hand, the personalized complementary and alternative communication device using voice recognition and artificial intelligence learning according to an embodiment of the present invention described above, for example, a smart phone (Smart phone), a smart pad (Smart Pad), and / or smart It may be implemented as at least one mobile communication terminal among smart notes, but is not limited thereto, and all wired/wireless communication functions such as mobile play-stations, tablet PCs, and iPads It may be implemented as /wireless home appliance/communication device.

Hereinafter, a personalized complementary and alternative communication method using speech recognition and artificial intelligence learning according to an embodiment of the present invention will be described in detail.

2 and 3 are overall flowcharts for explaining a supplementary and alternative communication method customized for each individual using speech recognition and learning of artificial intelligence according to an embodiment of the present invention.

1 to 3, the control module 400 is the subject of the personalized complementary and alternative communication method using voice recognition and artificial intelligence learning according to an embodiment of the present invention, unless otherwise specified. do.

First, at least one text information data is displayed on the display screen through the display module 200 (S100). In this case, it is preferable that the at least one text information data includes, for example, at least one of a word and/or a sentence.

Thereafter, the user's voice information data corresponding to the text information data displayed in step S100 is repeatedly acquired a plurality of times through the voice acquisition module 100 (S200).

Then, it is checked whether the user's voice information data repeatedly acquired a plurality of times in step S200 is consistent by using a preset artificial intelligence learning mode through the control module 400 (S300).

In this case, the artificial intelligence learning mode preset in step S300 is, for example, at least one artificial intelligence of a neural network, a support vector machine (SVM), a multi layer perception (MLP), and/or deep learning. It is desirable to apply an (AI)-based machine learning method.

Next, through the control module 400, the text information data displayed in step S100 and the user's voice information data consistent in step S300 are matched to each other and stored in the storage module 300 (S400).

Thereafter, voice recognition is performed on the user's voice information data obtained from the voice acquisition module 100 using a preset voice recognition technique through the control module 400 (S500).

Then, through the control module 400, the voice information data of the user whose voice recognition is completed in step S500 and the consistent user's voice information data stored in the step S400 are compared and analyzed, and the voice of the user whose voice recognition is completed in step S500 is compared and analyzed. The same consistent voice information data as the information data is searched (S600).

Next, the storage module 300 calls the text information data matched with the same consistent voice information data searched for in step S600 through the control module 400, and displays it on the display screen of the display module 200 ( S700).

Additionally, after the step S700, the storage module 300 calls the text information data matched with the same consistent voice information data searched for in the step S600 through the control module 400, and automatically converts a preset text-to-speech ( After converting the called text information data into voice using a Text to Speech (TTS) technique, the converted voice is output through the voice output module 600 (S800).

Furthermore, although not shown in the drawings, a first step of repeatedly acquiring the user's voice information data a plurality of times through the voice acquisition module 100 before and/or after the step S400, and the control module A second step of checking whether the user's voice information data repeatedly acquired a plurality of times in the first step using a preset speech recognition technique through 400 is consistent, and the user through the control module 400 A third step of receiving text information data corresponding to the consistent user voice information data confirmed in the second step according to a specific input signal output from the input module 700; A storage module by matching the consistent user's voice information data identified in the second step with the text information data provided in the third step based on the consistent user's voice information data provided in step 3 and corresponding text information data A fourth step of storing in 300 may be further included.

In addition, although not shown in the drawing, after the step S600, the voice information data of the voice recognition completed in the step S500 and the consistent user's voice information data stored in the step S400 are compared and analyzed through the control module 400, and the When the same consistent voice information data as the voice information data of the voice recognition completed in step S500 is not retrieved, the text information data matched in the order of the highest voice similarity with the voice information data of the voice recognition completed in the step S500 is stored in the storage module 300 ) and displaying it on the display screen of the display module 200 in the form of a list.

In addition, although not shown in the drawing, the text information that is matched in the order of the highest voice similarity with the voice information data whose voice recognition is completed in step S500 by the corresponding user using the user input module 700 through the control module 400 . After selecting any one of the text information data, the selected text information data is displayed on the display screen of the display module 200 and at the same time, the selected text to speech (TTS) technique is used to display the selected text information. After converting the text information data into voice, the method may further include outputting the converted voice through the voice output module 600 .

On the other hand, the additionally performed steps including the steps S100 to S800 are preferably performed through the execution of a pre-installed personalized supplementary and alternative communication-related application according to the control of the control module 400 .

In addition, when the individual customized complementary and alternative communication related application is executed through the control module 400 , the user unique identification set based on the user unique identification information for authentication of the corresponding user input through the user input module 700 is executed After completion of authentication of the corresponding user by comparison with the information, additionally performed steps including steps S100 to S800 may be performed.

In addition, although not shown in the drawing, the character information data displayed in step S100 together with the user's unique identification information preset through the control module 400 before and/or after step S400 and consistent user in step S300 The method may further include the step of matching each other so that the voice information data of the voice information is transmitted to the external terminal and/or the server 20 using the communication module 800 .

In addition, although not shown in the drawing, the text information data displayed in step S100 is matched and matched with the user's unique identification information transmitted from the communication module 800 through the external terminal and/or server 20 It may further include the step of storing and managing the consistent user's voice information data corresponding to and matched with the text information data for each user based on the consistent user's voice information data in step S300 into a database (DB), and storing and managing the same.

In addition, although not shown in the drawing, after the step S600, the voice information data of the voice recognition completed in the step S500 and the consistent user's voice information data stored in the step S400 are compared and analyzed through the control module 400, and the When it is not possible to search for the same consistent voice information data as the voice information data of the voice recognition completed in step S500, the voice information data of the voice recognition completed in the step S500 is transferred to an external terminal and/or server 20 through the communication module 800 in step S500. It may further include the step of transmitting to.

In addition, although not shown in the drawing, a separate voice information DB (not shown) based on the voice information data of which voice recognition is completed in step S500 transmitted from the communication module 800 through an external terminal and/or server 20 After searching for the same consistent voice information data as the voice information data for which voice recognition has been completed in step S500 through 800) may further include the step of transmitting.

On the other hand, the personalized complementary and alternative communication method using speech recognition and artificial intelligence learning according to an embodiment of the present invention can also be implemented as a computer-readable code on a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored.

For example, computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, hard disk, floppy disk, removable storage device, and non-volatile memory (Flash Memory). , and optical data storage devices.

In addition, the computer-readable recording medium may be distributed in a computer system connected through a computer communication network, and stored and executed as readable code in a distributed manner.

Although the above-described preferred embodiment of the personalized complementary and alternative communication device and method using voice recognition and artificial intelligence learning according to the present invention has been described, the present invention is not limited thereto, and the claims and detailed description of the invention It is possible to carry out various modifications within the scope of the description and the accompanying drawings, and this also belongs to the present invention.

100: voice acquisition module,
200: display module,
300: storage module,
400: control module,
500: power supply module,
600: voice output module,
700: user input module,
800: communication module

Claims (27)

a voice acquisition module for acquiring the user's voice information data;
a display module for displaying at least one text information data on a display screen;
a storage module for storing the user's voice information data obtained from the voice acquisition module and text information data corresponding to the database (DB); and
The operation of the display module is controlled so that at least one text information data is displayed on the display screen through a preset artificial intelligence learning mode, and the user's voice information data corresponding to the text information data displayed on the display screen is displayed a plurality of times. Controls the operation of the voice acquisition module to be repeatedly acquired, and after confirming whether the user's voice information data repeatedly acquired a plurality of times from the voice acquisition module is consistent with the text information data displayed on the display screen A control module for controlling consistent user voice information data to be matched with each other and stored in the storage module is included,
The control module receives the user's voice information data obtained from the voice acquisition module and performs voice recognition on the acquired user's voice information data using a preset voice recognition technique, After comparing and analyzing the voice information data and the consistent user's voice information data stored in the storage module to search for the same consistent voice information data as the voice information data of the user who has completed the voice recognition, the same consistent voice information data and Controls the operation of the display module so that the matching text information data is called and displayed on the display screen, and the voice recognition is performed by comparing the voice information data of the voice recognition completed with the consistent voice information data of the user stored in the storage module. Learning of voice recognition and artificial intelligence, characterized in that when the same consistent voice information data as the completed voice information data is not retrieved, the voice recognition completed voice information data is transmitted to an external terminal or server through a separate communication module Personalized complementary and alternative communication device using
According to claim 1,
A voice output module for outputting voice information data is further included,
The control module calls the text information data matched with the searched identical and consistent voice information data, and converts the called text information data into voice through a preset text-to-speech (TTS) technique. to control the operation of the voice output module so that it is output to the outside.
According to claim 1,
The preset artificial intelligence learning mode is an artificial intelligence (AI)-based machine learning method of at least one of a neural network, a support vector machine (SVM), a multi layer perception (MLP), or deep learning. Personalized complementary and alternative communication device using speech recognition and artificial intelligence learning, characterized in that it applies.
According to claim 1,
A user input module for outputting a specific input signal by a user's operation is further included,
The control module controls the operation of the voice acquisition module so that the voice information data of the corresponding user is repeatedly acquired a plurality of times through a preset user voice input mode, and the voice is acquired using the preset voice recognition technique After confirming whether the user's voice information data repeatedly acquired a plurality of times from the module is consistent, the user's voice information data and text information data corresponding to the consistent user's voice information data are provided by a specific input signal output from the user input module. Based on this, the consistent user's voice information data and the text information data provided from the user input module are matched with each other and are controlled to be stored in the storage module. Personalized supplementation using voice recognition and artificial intelligence learning Alternative communication devices.
According to claim 1,
The control module compares and analyzes the voice information data of the voice recognition completed with the consistent user's voice information data stored in the storage module, and when it fails to search for the same consistent voice information data as the voice information data on which the voice recognition has been completed, the Speech recognition and artificial intelligence, characterized in that the operation of the display module is controlled to be displayed on the display screen in the form of a list by calling the text information data matched in the order of high voice similarity with the voice information data of which voice recognition has been completed. Personalized complementary and alternative communication device using intelligence learning.
6. The method of claim 5,
The control module, by using a separate user input module, selects any one of the text information data from among the text information data matched in the order of the highest voice similarity to the voice information data that has been recognized by the user by the corresponding user, The operation of the display module is controlled so that the text information data is displayed on the display screen, and at the same time, the selected text information data is converted into voice through a preset text-to-speech (TTS) technique and output to the outside. An individual-customized complementary and alternative communication device using voice recognition and artificial intelligence learning, characterized in that it controls the operation of a separate voice output module.
According to claim 1,
The control module, through the execution of a pre-installed individual customized supplementary and alternative communication-related application, to control so that all functions related to individual customized supplementary and alternative communication are performed. communication device.
8. The method of claim 7,
The control module receives the user's unique identification information for authentication of the corresponding user when executing the personalized complementary and alternative communication-related application, and compares it with preset user identification information based on the user's unique identification information after the user's authentication is completed. A personalized complementary and alternative communication device using voice recognition and artificial intelligence learning, characterized in that it controls all functions related to customized complementary and alternative communication to be performed.
According to claim 1,
A communication module for transmitting text information data corresponding to and matching with the user's voice information data obtained from the voice acquisition module for each user to the outside is further included,
The control module is configured to match the user's voice information data corresponding to the text information data displayed on the display screen together with the preset user's unique identification information with each other and transmit it to an external terminal or server through wired/wireless communication. An individual-customized complementary and alternative communication device using voice recognition and artificial intelligence learning, characterized in that it controls the operation of the communication module.
10. The method of claim 9,
The external terminal or server receives, through the communication module, consistent user's voice information data corresponding to and matched with the text information data displayed on the display screen together with the unique identification information of each user, and based on this, each user Personalized complementary and alternative communication device using voice recognition and artificial intelligence learning, characterized in that it stores and manages the consistent user's voice information data corresponding to and matched with text information data into a database (DB).
delete According to claim 1,
The external terminal or server receives the voice information data through the communication module and searches for the same consistent voice information data as the voice information data through a separate voice information DB based on the received voice information data. , A personalized complementary and alternative communication device using voice recognition and artificial intelligence learning, characterized in that the searched text information data matched with the same consistent voice information data is called and transmitted to the communication module.
According to claim 1,
The at least one text information data is a personalized complementary and alternative communication device using speech recognition and artificial intelligence learning, characterized in that it consists of at least one of a word or a sentence.
A method for personalized complementary and alternative communication using a device including a voice acquisition module, a display module, a storage module, and a control module, the method comprising:
(a) displaying at least one text information data on a display screen through the display module;
(b) repeatedly acquiring the user's voice information data corresponding to the text information data displayed in step (a) a plurality of times through the voice acquisition module;
(c) checking whether the user's voice information data repeatedly acquired a plurality of times in step (b) is consistent by using a preset artificial intelligence learning mode through the control module;
(d) matching the text information data displayed in step (a) with the voice information data of the user consistent in step (c) through the control module to correspond to each other and storing the same in the storage module;
(e) performing voice recognition on the user's voice information data obtained from the voice acquisition module using a preset voice recognition technique through the control module;
(f) through the control module, the voice information data of the user whose voice recognition has been completed in step (e) and the consistent user's voice information data stored in the step (d) are compared and analyzed, and the voice recognition is completed in step (e) retrieving the same consistent voice information data as the user's voice information data; and
(g) calling the text information data matched with the same consistent voice information data retrieved in step (f) through the control module and displaying it on the display screen of the display module,
After the step (f), the voice information data of the voice recognition completed in the step (e) and the consistent user's voice information data stored in the step (d) are compared and analyzed through the control module in the step (e). When it is not possible to search for the same consistent voice information data as the voice information data on which the voice recognition has been completed, the method further includes the step of transmitting the voice information data of the voice recognition completed in step (e) to an external terminal or server through a separate communication module. Personalized complementary and alternative communication method using speech recognition and artificial intelligence learning, characterized in that
15. The method of claim 14,
After step (g),
(h) calling the text information data matched with the same consistent voice information data retrieved in step (f) through the control module, and using a preset text-to-speech (TTS) technique After converting the called text information data into a voice, the converted voice is output through a separate voice output module. Personalized complementary and alternative communication using voice recognition and artificial intelligence learning, characterized in that it further comprises Way.
15. The method of claim 14,
The artificial intelligence learning mode preset in step (c) is at least one of a neural network, a support vector machine (SVM), a multi layer perception (MLP), or deep learning (AI) A personalized complementary and alternative communication method using speech recognition and artificial intelligence learning, characterized by applying a machine learning method based on it.
15. The method of claim 14,
Before step (a) or after step (d),
a first step of repeatedly acquiring the user's voice information data a plurality of times through the voice acquisition module;
a second step of checking whether the user's voice information data repeatedly acquired a plurality of times in the first step using a preset voice recognition technique through the control module is consistent;
a third step of receiving, through the control module, text information data corresponding to the consistent user's voice information data confirmed in the second step according to a specific input signal output from a separate user input module; and
On the basis of the text information data corresponding to the consistent user's voice information data provided in the third step through the control module, the consistent user's voice information data confirmed in the second step and the text information provided in the third step Personalized complementary and alternative communication method using voice recognition and learning of artificial intelligence, characterized in that it further comprises a fourth step of matching data with each other and storing the data in the storage module.
15. The method of claim 14,
After the step (f), the voice information data of the voice recognition completed in step (e) and the consistent user's voice information data stored in the step (d) are compared and analyzed through the control module in the step (e). When it is not possible to search for the same consistent voice information data as the voice information data of the voice recognition completed, in the step (e), the text information data matched with the voice information data of the voice recognition completed and the voice information data in the order of high voice similarity are called and listed. ) in the form of displaying on the display screen of the display module, personalized complementary and alternative communication method using voice recognition and learning of artificial intelligence.
19. The method of claim 18,
Using a separate user input module through the control module, any one of the text information data matched with the voice information data that has been voice recognized in step (e) in the order of high voice similarity by the corresponding user After selection, the selected text information data is displayed on the display screen of the display module and, at the same time, the selected text information data is converted into voice using a preset text-to-speech (TTS) technique, and the Personalized complementary and alternative communication method using speech recognition and learning of artificial intelligence, characterized in that it further comprises the step of outputting the converted voice through a separate voice output module.
15. The method of claim 14,
The steps (a) to (g) are individually customized using voice recognition and artificial intelligence learning, characterized in that it is performed through the execution of a pre-installed personalized supplementary and alternative communication related application according to the control of the control module. Complementary and Alternative Communication Methods.
21. The method of claim 20,
When the individual customized complementary and alternative communication-related application is executed through the control module, the corresponding user-specific identification information is compared with preset user-specific identification information for authentication of the user input through a separate user input module. A personalized complementary and alternative communication method using voice recognition and learning of artificial intelligence, characterized in that the steps (a) to (g) are performed after the user's authentication is completed.
15. The method of claim 14,
Before or after step (d), the text information data displayed in step (a) together with the user's unique identification information preset through the control module and the consistent user's voice information data in step (c) correspond to each other Matching each other as much as possible and transmitting it to an external terminal or server using a separate communication module.
23. The method of claim 22,
Consistent user's voice information in step (c) corresponding to and matching with the text information data displayed in step (a) together with the user's unique identification information transmitted from the communication module through the external terminal or server Using the learning of voice recognition and artificial intelligence, characterized in that it further comprises the step of storing and managing the consistent user's voice information data corresponding to and matched with the text information data for each user based on the data into a database (DB) Complementary and alternative communication methods tailored to each individual.
delete 15. The method of claim 14,
On the basis of the voice information data of the voice recognition completed in step (e) transmitted from the communication module through the external terminal or server, the same as the voice information data of the voice recognition completed in step (e) through a separate voice information DB Speech recognition and artificial intelligence learning, characterized in that after searching for consistent voice information data, calling the text information data matched with the searched identical consistent voice information data and transmitting it to the communication module Personalized complementary and alternative communication method using
15. The method of claim 14,
In the step (a), the at least one text information data, a personalized complementary and alternative communication method using speech recognition and learning of artificial intelligence, characterized in that at least one of a word or a sentence.
A computer-readable recording medium in which a program capable of executing the method of any one of claims 14 to 23, 25, and 26 by a computer is recorded.

Images