KR102156570B1 - Smart hearing device for distinguishing non-natural language or natural language and the method thereof, and artificial intelligence hearing system - Google Patents

Abstract

The present invention relates to a smart hearing device, which provides control parameters and feedback for natural language or non-natural language determined by analyzing sound data. The smart hearing device includes: a receiving unit for receiving sound data of a voice signal and a noise signal from a first microphone and a second microphone formed on one side; a determination unit which compares the digital flow of the sound data and a previously stored graph pattern to determine a natural or non-natural language for the sound data; a processing unit which matches similar data for the determined natural language or non-natural language based on a database including a natural language area and an unnatural language area; and a providing unit which sets a control parameter in a specified natural or non-natural language according to the matched similar data and provides a user with a converted sound in one side.

Description

Smart hearing device, artificial intelligence hearing system, and method for distinguishing natural or non-natural languages {SMART HEARING DEVICE FOR DISTINGUISHING NON-NATURAL LANGUAGE OR NATURAL LANGUAGE AND THE METHOD THEREOF, AND ARTIFICIAL INTELLIGENCE HEARING SYSTEM}

The present invention relates to a smart hearing device, an artificial intelligence hearing system, and a method for distinguishing natural or non-natural languages, and more particularly, a technology for providing control parameters and feedback for natural or non-natural languages determined by analyzing sound data It is about.

In recent years, due to the leap forward in medical engineering technology, patients who have not received great help from wearing hearing aids in the past have been able to achieve excellent hearing improvement by selecting and wearing suitable hearing aids.

Hearing aids are high-tech medical devices that are always attached to the body and used among medical devices. Hearing aids should be continuously managed according to changes in hearing, and A/S should be received for parts damaged by moisture and foreign substances in the ear. Therefore, hearing aids are considered to be one of the most important technologies among medical technologies.

Conventional hearing aids were in the form of trumpet-type sound collectors, but now they are usually used in the form of electric hearing aids that help amplify sound. In addition, hearing aids have a bone diagram that is mounted on the mastoid, but they are usually airway-type structures, and they receive sound waves through a microphone and convert them into electric vibrations, and expand this to convert sound waves back into earphones so that they are heard by the ears.

Recently, research on a more powerful hearing aid processor is being conducted. The hearing aid processor is equipped with a processing speed and memory that is more than twice as fast as that of the existing processor, and is composed of chips and parts made of small pieces with advanced nanotechnology.

However, since the existing hearing aid technology is set based on the hearing data of the hearing impaired person (hereinafter referred to as “user”), there is a limitation in that data on the user's real-time ambient noise cannot be applied.

In addition, the existing hearing aid technology simply guesses the human voice through the sound range classification and separates the external noise from the human voice, but this has poor performance and cannot distinguish between natural and unnatural languages. In addition, the range of the human voice range is quite wide, about 500Hz to 5000Hz, but the existing hearing aid technology does not implement the voice enhancement function and noise reduction function well because the specific range of the voice is recognized as external noise. There were limitations.

An object of the present invention is to provide more effective hearing aid performance with different parameters and feedback by discriminating a user's voice and noise through distinction between natural and non-natural languages.

In addition, an object of the present invention is to provide a three-dimensional recognition of environmental changes and noise changes according to the sound direction of the left and right using a first smart hearing device and a second smart hearing device worn on the right and left side of the user. do.

The smart hearing device according to an embodiment of the present invention includes a receiver configured to receive sound data of a voice signal and a noise signal from a first microphone and a second microphone formed on one side, and compare the digital flow of the sound data with a previously stored graph pattern. Based on a determination unit that determines a natural or unnatural language for sound data, a database including a natural language region and an unnatural language region, a processing unit that matches similar data for the determined natural language or non-natural language, and the matched similar data And a providing unit configured to provide a user with a sound of one side converted by setting a control parameter in the specified natural language or non-natural language.

In addition, the smart hearing device according to an embodiment of the present invention may further include a feedback transmitting unit that provides feedback in response to the specified natural language or non-natural language according to the matched similar data.

The receiver may detect the voice signal and the noise signal from the first microphone and the second microphone formed at different positions, convert the voice signal and the noise signal to digital to receive the sound data.

The determination unit compares the digital flow of the digitally converted sound data with a graph pattern of a volume graph for a previously stored time and a volume graph for a frequency, and may determine a natural language or an unnatural language representing a specific pattern from the values of the graph. have.

Based on the database, the processor matches similar data of words and sentences similar to the sound data determined in natural language, or similar environmental noise, noise (noise) and similarity to the sound data determined in non-natural language. Data can be matched.

The providing unit sets at least one of amplification value change according to environmental change, volume control, and frequency control based on a natural language or non-natural language specified according to the user's hearing data and the matched similar data to customize the user. It can provide the sound of one side.

The providing unit sets the control parameter in the sound data of the digital signal received from the first microphone and the second microphone to adjust at least one or more of amplification value change, volume control, and frequency control, and the adjusted By converting a digital signal for a signal into an analog signal, a sound of one side may be provided to the user.

When the similar data of similar words and sentences are matched with the sound data determined in natural or non-natural language through the processing unit, the feedback transmitting unit may provide feedback of words and sentences in response to the similar data.

When the similar data of similar words and sentences with respect to the sound data are matched based on artificial intelligence (AI), the feedback transmitting unit provides feedback of words and sentences in response to a question caused by the user's voice. It can be extracted and delivered as a voice message.

The artificial intelligence hearing system according to an embodiment of the present invention determines a natural or non-natural language for the first sound data including a voice signal and a noise signal received from a first microphone and a second microphone formed on one side, and the determined natural language Alternatively, a first smart hearing device that provides sound from one side by setting a first control parameter due to matching similar data with an unnatural language, a third microphone formed on the other side, and a voice signal and a noise signal received from the fourth microphone roll. A second smart hearing device that determines a natural or unnatural language for the included second sound data, sets a second control parameter due to matching of the similar data with the determined natural or non-natural language to provide the sound of the other party, and And a mobile device that controls the first smart hearing device and the second smart hearing device.

Each of the first smart hearing device and the second smart hearing device includes the first microphone and the third microphone positioned near the user's mouth, and the second microphone positioned at a distance from the user's mouth And the fourth microphone.

Each of the first smart hearing device and the second smart hearing device can provide a user with a customized sound of one side and a sound of the other side, respectively, so that environmental changes and noise changes according to the sound direction of the left and right are three-dimensionally recognized. have.

Each of the first smart hearing device and the second smart hearing device may set the first control parameter and the second parameter of different parameter values based on left hearing data and right hearing data of the user.

The mobile device may control any one or more of power on/off, signal collection and control parameter setting of each of the first smart hearing device and the second smart hearing device according to a user's selection input. have.

In the operating method of a smart hearing device for discriminating natural or unnatural language according to an embodiment of the present invention, receiving sound data of a voice signal and a noise signal from a first microphone and a second microphone formed at one side, the sound data Determining a natural language or non-natural language for the sound data by comparing the digital flow of and a previously stored graph pattern, based on a database including a natural language region and a non-natural language region, similar data for the determined natural language or non-natural language And setting a control parameter to a specified natural language or non-natural language according to the matched similar data, and providing a converted sound to a user.

In addition, the method of operating the smart hearing device according to an embodiment of the present invention may further include providing feedback in response to the specified natural language or non-natural language according to the matched similar data.

According to an embodiment of the present invention, it is possible to provide more effective hearing aid performance through different parameters and feedback by discriminating the user's voice and noise through the classification of natural and non-natural languages.

In addition, according to an embodiment of the present invention, customized hearing aid in environmental changes and noise changes according to the sound direction of the left and right by using the first smart hearing device and the second smart hearing device worn on the right and left side of the user, respectively. The service can be provided and the convenience of using the hearing aid can be improved accordingly.

1 is a block diagram showing a detailed configuration of a smart hearing device according to an embodiment of the present invention.
2 is a flowchart illustrating a method of operating a smart hearing device according to an embodiment of the present invention.
3A and 3B are diagrams for explaining an example of providing feedback by classifying natural or non-natural languages according to an embodiment of the present invention.
4A and 4B illustrate an embodiment of a smart hearing device according to an embodiment of the present invention.
5, 6A, and 6B illustrate examples of application of a smart hearing device according to an embodiment of the present invention.
7 illustrates the configuration of an artificial intelligence hearing system according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the embodiments. In addition, the same reference numerals shown in each drawing denote the same member.

In addition, terms used in the present specification are terms used to properly express preferred embodiments of the present invention, which may vary depending on the intention of viewers or operators, or customs in the field to which the present invention belongs. Accordingly, definitions of these terms should be made based on the contents throughout the present specification.

The present invention relates to a smart hearing device, an artificial intelligence hearing system, and a method for classifying natural or non-natural languages, by analyzing sound data of a voice signal and a noise signal received from a smart hearing device worn on a user's ear, The gist is to determine non-natural language and provide control parameter setting and feedback according to matching similar data according to the determination result.

In this case, the smart hearing device according to an embodiment of the present invention is a hearing aid that provides amplified sound so that a user with low hearing ability can hear it.

Hereinafter, according to the determined natural or non-natural language, at least one of the amplification value, volume, and frequency of the signal is controlled, and feedback is provided, thereby improving the convenience of the customized hearing service to the user. A smart hearing device, an artificial intelligence hearing system, and a method for classifying natural or non-natural languages according to an embodiment will be described in detail with reference to FIGS. 1 to 7.

1 is a block diagram showing a detailed configuration of a smart hearing device according to an embodiment of the present invention.

Referring to FIG. 1, a smart hearing device according to an embodiment of the present invention analyzes sound data and provides control parameters and feedback for a natural or non-natural language determined.

The smart hearing device 100 according to an embodiment of the present invention to be described below is worn on the left or right ear of the user, and may be a first smart hearing device or a second smart hearing device, and the first smart hearing device and the The second smart hearing device is characterized in that it has the same detailed configuration as shown in FIG. 1 and described below.

To this end, the smart hearing device 100 according to an embodiment of the present invention includes a receiving unit 110, a determining unit 120, a processing unit 130, and a providing unit 140. In addition, the smart hearing device 100 according to an embodiment of the present invention may further include a feedback transmission unit 150 and a database 160.

The receiver 110 receives sound data of a voice signal and a noise signal from the first microphone 101 and the second microphone 102 formed at one side.

The receiver 110 may detect a voice signal and a noise signal from the first microphone 101 and the second microphone 102 formed at different locations, and convert the voice signal and noise signal digitally to receive sound data. .

In this case, the first microphone 101 may receive a user's voice signal, and the second microphone 102 may receive a noise signal around the user. More specifically, since the first microphone 101 and the second microphone 102 are located at different distances based on the user's mouth, the first microphone 101 is located in the vicinity of the user's mouth to focus on the user's voice. And the second microphone 102 is located as far as possible from the user's mouth compared to the first microphone 101 to receive a noise signal focusing on ambient noise.

The first microphone 101 and the second microphone 102 according to the embodiment of the present invention are disposed at different positions in the smart hearing device 100, but the tools of the first microphone 101 and the second microphone 102 The direction in which the (or hole) faces is characterized in that it senses a uniform voice signal and a noise signal, and is the same for proper noise reduction. Here, the appropriate noise may mean noise and numerical values other than the voice signal and noise signal collected at the location of the microphone.

Accordingly, the receiving unit 110 may receive the voice signal and noise signal sensed by the first microphone 101 and the second microphone 102, convert it to an electrical signal, that is, digitally, and receive it as sound data.

The determination unit 120 compares the digital flow of sound data with a previously stored graph pattern to determine a natural or non-natural language for the sound data.

The determination unit 120 compares the digital flow of digitally converted sound data based on artificial intelligence (AI) and a graph pattern of a volume graph for a previously stored time and a volume graph for a frequency, and values of the graph Natural language or unnatural language representing a specific pattern can be judged.

The natural language according to the embodiment of the present invention represents a word or sentence such as a conversation between people and a user's question, and the unnatural language is a noise such as a loud sound of a construction site, a car sound, a sound temporarily generated from an object, and unfamiliar to people. Represents a sound (or a new sound). Each natural language and non-natural language represent different specific patterns in a time-volume graph and a frequency-volume graph, and a specific pattern according to the graph may be stored and maintained in the database 160. For example, the user's conversation has a higher and lower level of sound compared to the noise generated at the construction site, and persistence occurs. Accordingly, there is a time for a sentence according to a user's conversation-a volume graph and a frequency-a specific pattern in the volume graph, and a time for the noise generated at the construction site-a volume graph and frequency-a specific pattern in the volume graph I can.

The determination unit 120 may determine a natural language or an unnatural language representing a specific pattern by comparing the digital flow of sound data converted to digital with graph patterns for natural and non-natural languages stored and maintained in the database 160. For example, the determination unit 120 may compare and analyze digitally converted sound data to determine natural language or non-natural language.

The processing unit 130 matches similar data for the determined natural language or non-natural language, based on the database 160 including the natural language region and the non-natural language region.

When the sound data is determined in natural or non-natural language by the determination unit 120, the processing unit 130 matches similar data of similar words and sentences with sound data determined in natural language based on the database 160, or Similar environmental noise, noise (noise), and similar data of a new sound may be matched with sound data determined in an unnatural language. More specifically, the database 160 may include a natural language region including words or sentences related to natural language and a non-natural language region including information related to noise and unfamiliar sound, and the sound data is stored by the determination unit 120. When it is determined in natural language, the processor 130 may match similar data of words or sentences similar to sound data in the natural language area based on the database 160.

For example, when the loud sound of the construction site or the car sound is determined by the determination unit 120 in an unnatural language, the processing unit 130 may match similar data with the noise of the construction site or the car noise. In addition, when an unnatural sound is determined by the determination unit 120 in an unnatural language, the processing unit 130 may match similar data with a new sound or a new noise.

As another example, when the voices of the user and the neighbors are determined in natural language by the determination unit 120, the processing unit 130 may match similar data such as “Goodbye” and “It's been a long time” based on the voices. In addition, when the user's question is determined in natural language by the determination unit 120, the processing unit 130 “how is the weather today?” through a related search of the database 160. Or, you can match similar data such as "hungry".

The providing unit 140 sets a control parameter in a specified natural language or non-natural language according to the matched similar data, and provides the user with the converted sound.

In this case, the smart hearing device 100 according to an embodiment of the present invention may basically include hearing data (Personal Hearing Profile) of a user using a hearing aid. For example, the providing unit 140 may include a user's hearing data including a volume and a frequency preferred by the user, an amplification value such that a sense of difference is not felt, a volume, and a frequency range. Depending on the embodiment, the above-described data may be received by the mobile device 500 or the external server 600 interworking with the smart hearing device 100 according to the embodiment of the present invention, and the device 500 or an external server It may be stored and maintained at 600.

However, the hearing data is not limited to items or numerical values such as amplification value, volume, and frequency. For example, the hearing data is nonlinear compression information that amplifies small sounds large and reduces loud sounds small, directional information that accurately detects the direction in which the sound is heard, and amplifies the received sound by a microphone so that it can be heard without other noise. The user's preference and numerical value for at least one or more of information for helping feedback and noise reduction information for reducing noise may be further included.

The providing unit 140 sets at least one of amplification value change, volume control, and frequency control according to environmental changes based on a specified natural language or non-natural language according to similar data matched with the user's hearing data. The sound of one side can be customized. In this case, the providing unit 140 is characterized by applying different control parameter values to natural and non-natural languages. For example, even if the same control parameter for increasing the volume is applied, the providing unit 140 may further increase the control parameter value applied to the natural language rather than the non-natural language.

More specifically, the providing unit 140 sets a control parameter in sound data of a digital signal received from the first microphone 101 and the second microphone 102 to change at least one of amplification value change, volume control, and frequency control. One or more balances may be adjusted, and a digital signal for the adjusted signal may be converted into an analog signal to provide a user with a sound of one side.

For example, at least one or more of an amplification value, volume, and frequency according to sound data received from the first microphone 101 and the second microphone 102 may be preset by the user or out of a preferred reference range. . This may be caused by any one of a factor according to a change in the environment where the user is located, a factor according to a change in the user's voice, a factor of a mechanical error, and a height factor due to a natural or unnatural language. Accordingly, the providing unit 140 may adjust the balance of at least one of the amplification values, the volume, and the frequency of the voice signal and the noise signal based on the similar data for the natural or non-natural language matched by the processing unit 130, A digital signal according to the adjusted balance can be converted into an analog signal (sound energy) and provided to the user as sound.

For example, when similar data is matched with noise of a construction site or vehicle noise by the processing unit 130, the providing unit 140 may adjust a balance for lowering the volume of the noise and provide it to the user.

As another example, when the voices of the user and the surrounding people are matched with similar data such as "Goodbye" or "It's been a while" by the processing unit 130, the providing unit 140 may provide an amplification value of the matched similar data, By adjusting the balance for increasing the volume and frequency, it is possible to provide information to be more accurately transmitted to the user.

That is, the smart hearing device 100 according to the embodiment of the present invention analyzes sound data including the voice signal and noise signal received from the first microphone 101 and the second microphone 102 according to a change in the user's environment. Hearing aid that is optimized for changing situations even if the user does not separately adjust the volume or frequency by setting control parameters for sound data by judging natural or non-natural languages and matching similar data for natural or non-natural languages Service can be provided, and convenience of hearing aid can be improved.

The feedback transmission unit 150 of the smart hearing device 100 according to an embodiment of the present invention may provide feedback in response to a specified natural language according to matched similar data.

When the similar data of similar words and sentences are matched with sound data determined in natural or non-natural language through the processing unit 130, the feedback transmission unit 150 may provide feedback of words and sentences in response to the similar data. have.

When the similar data of similar words and sentences with respect to sound data are matched based on artificial intelligence (AI), the feedback transmission unit 150 provides feedback of words and sentences in response to a question from the user's voice. It can be extracted and delivered as a voice.

For example, when the loud sound or car sound of the construction site is determined by the determination unit 120 in an unnatural language, and the processing unit 130 matches the noise of the construction site or similar data of the car noise, it corresponds to the non-natural language. The sound data can be balanced by a control parameter. Thereafter, the feedback transmission unit 150 may provide feedback on sound data corresponding to an unnatural language to the user. For example, when similar data is matched due to vehicle noise, the feedback transmission unit 150 may provide a voice message notification such as “be careful” to the user. As another example, when similar data is matched with a new noise unfamiliar to the user, the feedback transmission unit 150 may provide a voice message notification such as “This is a new sound” or “This is a new environment” to the user.

As another example, a user's question is determined in natural language by the determination unit 120, and “how is the weather today?” by the processing unit 130 Alternatively, when matching with similar data such as “hungry”, sound data corresponding to natural language may be balanced by a control parameter. Thereafter, the feedback delivery unit 150 may provide feedback on sound data corresponding to natural language to the user. For example, when similar data of “how is the weather today?” are matched, the feedback delivery unit 150 may provide a voice message notification such as “the weather is sunny” to the user. As another example, when similar data such as “hungry” are matched, the feedback delivery unit 150 may provide a voice message notification such as “the nearby restaurant has OOO” to the user.

The smart hearing device 100 according to an embodiment of the present invention transmits information related to sound data, similar data, control parameters, and feedback to a mobile device or an external server, and receives a user's input from the mobile device (not shown). ) May be further included.

For example, the communication unit is a mobile device possessed by a user through any one of Bluetooth, Wi-Fi (Wireless Fidelity), Zigbee, and BLE (Bluetooth Low Energy) wireless communication module. , Similar data, control parameters, and information related to feedback may be transmitted, and a control command for controlling power on/off or control parameters of the smart hearing device according to the user's selection input may be received from the mobile device. .

Depending on the embodiment, the communication unit of the smart hearing device 100 according to the embodiment of the present invention includes not only a short-range wireless communication module, but also a cellular telephone network, a wireless local area network (LAN), and/or a metropolitan area network (MAN). Wireless networks, intranets, and/or networks such as the Internet, also referred to as the World Wide Web (WWW), and other devices may communicate by wireless communication.

These wireless communications include Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), and Bluetooth. Bluetooth), Wi-Fi (Wireless Fidelity) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and/or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, Long Term Evolution (LTE) , Zigbee, Z-wave, BLE (Bluetooth Low Energy), Beacon, IMAP (Internet Message Access Protocol) and/or POP (Post Office Protocol) protocol for email, eXtensible Messaging and Presence (XMPP) Protocol), SIMPLE (Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions), IMPS (Instant Messaging and Presence Service), SMS (Short Message Service), instant messaging such as LoRa, or communication protocols not developed at the time of filing of this application. Can include. However, a plurality of communication standards, protocols, and technologies that are not limited to these may be used for the wireless communication described above.

2 is a flowchart illustrating a method of operating a smart hearing device according to an embodiment of the present invention.

The operation method of FIG. 2 is performed by the smart hearing device 100 according to the embodiment of the present invention of FIG. 1.

Referring to FIG. 2, in step 210, sound data of a voice signal and a noise signal are received from a first microphone and a second microphone formed at one side.

In step 210, a voice signal and a noise signal are detected from a first microphone and a second microphone formed at different locations, and in step 220, the voice signal and the noise signal are digitally converted and received as sound data.

In step 230, a natural or unnatural language for the sound data is determined by comparing the digital flow of sound data with a previously stored graph pattern.

Step 230 compares the digital flow of digitally converted sound data based on artificial intelligence (AI) with the graph pattern of the volume graph for the previously stored time and the volume graph for the frequency, and a specific pattern from the values of the graph. It is possible to judge a natural or unnatural language representing

In step 230, the sound data is divided into natural language or non-natural language. Steps 241 and 251 match similar data for the determined natural language or non-natural language based on a database including a natural language region and an unnatural language region. For example, in step 241, when it is determined by step 230 that a loud sound or a car sound is unnatural, similar data may be matched with the noise of the construction site or the car noise. In addition, in step 251, when the voices of the user and the surrounding people are determined in natural language by step 230, similar data such as "hello" and "it's been a long time" based on the voice may be matched, and in step 230, the user's question If it is judged as a natural language, the database is searched for "how is the weather today?" Or, you can match similar data such as "hungry".

Thereafter, in steps 242 and 252, a control parameter is set in the specified natural language or non-natural language according to the matched similar data, and the converted sound of one side is provided to the user.

Steps 242 and 252 are based on a natural or non-natural language specified according to similar data matched with the user's hearing data, and set at least one of amplification value change, volume control, and frequency control according to environmental changes, The sound of one side can be customized.

For example, if similar data is matched with noise of a construction site or automobile noise in step 241, step 242 may adjust a balance for lowering the volume of noise and provide it to the user. In addition, when the voices of the user and the surrounding people are matched with similar data such as "Goodbye" or "It's been a while" in step 251, step 252 is a balance that increases the amplification value, volume, and frequency of the matched similar data rather than the ambient noise. It can be provided so that information is delivered to the user more accurately by adjusting.

In this case, steps 242 and 252 are characterized in that different control parameter values are applied to natural or non-natural languages. In more detail, even if the same control parameter for increasing the volume is applied, the control parameter applied to the natural language may be higher than the control parameter applied to the non-natural language.

Thereafter, in steps 243 and 253, feedback may be provided in response to a specified natural language or non-natural language according to the matched similar data.

Steps 243 and 253 are based on artificial intelligence (AI), when similar data of similar words and sentences with respect to sound data are matched, extracting word and sentence feedback as a response to the user's question, I can deliver.

For example, if similar data is matched by car noise, step 243 may provide a voice message notification such as “Please be careful” to the user, and if similar data is matched with a new noise unfamiliar to the user, Voice message notifications such as "This is a new sound" or "This is a new environment" can be provided to the user. In addition, when similar data of “how is the weather today?” are matched, step 253 may provide a voice message notification such as “weather is sunny” to the user, and similar data such as “hungry” are matched. In this case, a voice message notification such as “the nearby restaurant has OOO” can be provided to the user.

In the non-natural language areas of steps 241, 242, and 243, the operation method of the smart hearing device according to an embodiment of the present invention sets a control parameter for noise and provides a sound suitable to the user, but threatening sound or new sound You can provide feedback with voice message notifications, such as "be careful" or "this is a new sound."

Similarly, in the natural language domain of steps 251, 252, and 253, the operation method of the smart hearing device according to an embodiment of the present invention sets a control parameter for the conversation sound between users and provides the sound suitable to the user. For questions, corresponding answers can be provided as feedback. In this case, the method of operating the smart hearing device according to the embodiment of the present invention may extract an appropriate answer to a user's question in a natural language based on artificial intelligence (AI).

3A and 3B are diagrams for explaining an example of providing feedback by classifying natural or non-natural languages according to an embodiment of the present invention.

In more detail, FIG. 3A is a diagram illustrating an example of determining a natural language according to an embodiment of the present invention and providing feedback, and FIG. 3B is a diagram illustrating a non-natural language according to an embodiment of the present invention and providing feedback. It is shown to illustrate an example.

Referring to FIG. 3A, the smart hearing device according to an embodiment of the present invention may detect sound data of “hello” from a first microphone and a second microphone formed at one side. At this time, the sound data of “Hello” represents a specific pattern in the time-volume graph and frequency-volume graph as shown in FIG. 3A, and the smart hearing device according to an embodiment of the present invention provides a digital flow and Sound data can be determined in natural language by comparing graph patterns previously stored in the database.

After determining the natural language, the smart hearing device according to the embodiment of the present invention matches similar data for the natural language. For example, the smart hearing device according to an embodiment of the present invention attempts to match similar data close to the sensed “hello”, and “hello” among data such as “hello”, “hello”, and “hello”. Similar data of'Hello' similar to the sound data of may be matched.

According to the matching of similar data, the smart hearing device according to an embodiment of the present invention sets at least one or more control parameters of amplification value change, volume control, and frequency control in natural language, and in response to the sound data of “Hello”. Feedback can be extracted and provided to the user through sound. For example, the smart hearing device according to an embodiment of the present invention provides voice message notifications such as "Yes, hello", "It's been a while", and "What can I do for you?" as feedback on sound data of "Hello". Can be provided to the user.

Referring to FIG. 3B, the smart hearing device according to an exemplary embodiment of the present invention may detect sound data of a “colog call” from a first microphone and a second microphone formed at one side. At this time, the sound data of the “coloc call” represents a specific pattern in the time-volume graph and frequency-volume graph as shown in FIG. 3B, and the smart hearing device according to the embodiment of the present invention is a digital flow for sound data. The sound data can be determined in an unnatural language by comparing the graph pattern previously stored in the database. 3A and 3B, it can be seen that natural language and non-natural language represent different graph patterns in the time-volume graph and frequency-volume graph, and the smart hearing device according to the embodiment of the present invention is such a graph It is possible to distinguish and judge natural or non-natural languages by the difference in patterns.

After determining the non-natural language, the smart hearing device according to the embodiment of the present invention matches similar data for the non-natural language. For example, the smart hearing device according to an embodiment of the present invention attempts to match similar data to the sensed “colok colok”, and the sound of “coloco coloc” among data such as “call”, “cool” and “takk” It is possible to match similar data of'coloc coloc' similar to data.

According to the matching of similar data, the smart hearing device according to an embodiment of the present invention sets at least one or more control parameters of amplification value change, volume control, and frequency control in an unnatural language, and Feedback can be extracted as a response and provided to the user through sound. For example, in the smart hearing device according to an embodiment of the present invention, "I will inform you of the nearby pharmacy", "I think you have a cold", "Drink a hot tea", and "I will inform you of the nearby pharmacy" with feedback on the sound data of the "Kolok Kolok" You can provide a voice message notification to the user, such as "Are you okay?"

4A and 4B illustrate an embodiment of a smart hearing device according to an embodiment of the present invention.

4A and 4B, the smart hearing device according to the embodiment of the present invention may be a pair of two to be worn on each of the user's left and right ears.

The first smart hearing device 300 and the second smart hearing device 400 to be described below include the same functions as the smart hearing device according to the embodiment of the present invention described with reference to FIGS. 1 to 3B.

4A, the first smart hearing device 300 according to the embodiment of the present invention includes a first microphone 301, a second microphone 302, and an on/off switch 303, and a second smart hearing device 300 The hearing device 400 includes a third microphone 401, a fourth microphone 402 and an on/off switch 403. Although the first smart hearing device 300 is worn on the left ear of the user, and the second smart hearing device 400 is worn on the right ear of the user, the location and shape of the smart hearing device are not limited thereto.

At this time, the first microphone 301 and the third microphone 401 may be positioned adjacent to the user's mouth in order to receive a voice signal focused on the user's voice, and the on/off switches 303 and 403 It is located at the lower side and is characterized in that it is relatively close to the user's mouth compared to the second microphone 302 and the fourth microphone 402.

In addition, the second microphone 302 and the fourth microphone 402 are located as far from the user's mouth as possible in order to receive a noise signal focusing on ambient noise according to the user's position, and the on/off switches 303 and 403 It is located above and is characterized in that it is located relatively far from the user's mouth compared to the first microphone 301 and the third microphone 401.

Further, the first microphone 301 and the second microphone 302, and the third microphone 401 and the fourth microphone 402 is characterized in that the direction of the tool (or hole) is the same, which is a uniform voice It is to collect signals and noise signals, respectively, and to remove appropriate noise accordingly.

4A, the first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention have two microphones 301, 302 and 401, 402 having different positions. Including, by software setting the first microphone 301 and the third microphone 401 as main, and applying the second microphone 302 and the fourth microphone 402 as secondary input sources, different voice signals And a noise signal can be uniformly collected.

At this time, the first microphone 301 of the first smart hearing device 300 and the third microphone 401 of the second smart hearing device 400 are paired with each other, and the second microphone 401 of the first smart hearing device 300 The microphone 302 and the fourth microphone 402 of the second smart hearing device 400 are paired with each other, and a microphone paired with any one microphone may also be automatically set according to a setting applied to any one microphone. I can.

For example, when the volume of the first microphone 301 is increased to a predetermined value by the control parameter, the volume of the paired third microphone 401 can also be automatically adjusted to a predetermined value. . As another example, when the power of the second microphone 302 of the first smart hearing device 300 is turned on, the power of the fourth microphone 402 of the paired second smart hearing device 400 is also automatically turned on. can be (on).

4A, the first smart hearing device 300 and the second smart hearing device 400 include on/off switches 303 and 403, and the on/off switches 303 and 403 are the first smart hearing devices. Powers of the hearing device 300 and the second smart hearing device 400 are turned on or off. For example, when the user touches, pushes, or presses the switch-type on/off switches 303 and 403 for a long time, the first smart hearing device 300 and the second smart hearing device 400 are turned on or off ( off). At this time, when at least one of the first smart hearing device 300 and the second smart hearing device 400 is turned on or off, the other paired smart hearing device is also turned on in the same manner. It can be (on) or off (off).

Referring to FIG. 4B, the first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention include charging modules 305 and 405 and speakers 304 and 404.

The first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention are rechargeable devices, and may include charging terminals 305 and 405.

For example, the first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention include a rechargeable Li-ion Polymer Battery and a battery meter of a mobile device. ) May be included, and the battery may be charged through the charging modules 305 and 405.

In addition, the first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention provide sound converted from a digital signal to an analog signal (sound energy) through the speakers 304 and 404. can do.

For example, the first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention include a first microphone 301, a second microphone 302, a third microphone 401, Set a first control parameter and a second control parameter according to information related to the analysis result in the voice signal and noise signal collected from the fourth microphone 402, and balance at least one of amplification value change, volume control, and frequency control The adjusted digital signal may be converted into an analog signal and provided as sound to a user through the speakers 304 and 404.

5, 6A, and 6B illustrate examples of application of a smart hearing device according to an embodiment of the present invention.

More specifically, FIG. 5 shows an example of a user wearing a smart hearing device according to an embodiment of the present invention as viewed from the top, and FIG. 6A is a first smart hearing device according to an embodiment of the present invention. An example of a user viewed from the left is shown, and FIG. 6B illustrates an example of a user wearing a second smart hearing device according to an embodiment of the present invention viewed from the right.

Referring to FIG. 5, the user 10 wears the first smart hearing device 300 on the left ear and the second smart hearing device 400 on the right ear. By wearing both the first smart hearing device 300 and the second smart hearing device 400, the user 10 can perceive environmental changes and noise changes according to the sound direction of the left and right in a more three-dimensional manner, and can be customized. Hearing aid services are available.

6A, the first smart hearing device 300 according to an embodiment of the present invention is mounted on the left ear of the user 10, and the first microphone 301 and the second microphone 302 are based on the user's mouth. It can be located at different distances.

For example, the first microphone 301 is positioned closer to the user's mouth than the second microphone 302 and may receive a voice signal focused on the user's voice. On the other hand, the second microphone 302 is positioned as far from the user's mouth as possible compared to the first microphone 301, and may receive a noise signal mainly for ambient noise according to the user's location.

In this case, as shown in FIG. 6A, it can be confirmed that the first microphone 301 and the second microphone 302 are located near or far from the user's mouth based on the on/off switch 303.

The first microphone 301 and the second microphone 302 are included in different locations in the first smart hearing device 300 according to the embodiment of the present invention, but the first microphone 301 and the second microphone 302 The direction in which the tool (or hole) faces is characterized in that it collects a uniform voice signal and noise signal, and accordingly is the same for proper noise reduction.

6B, a second smart hearing device 400 according to an embodiment of the present invention is mounted on the right ear of the user 10, and the third microphone 401 and the fourth microphone 402 are based on the user's mouth. It can be located at different distances.

For example, the third microphone 401 is positioned closer to the user's mouth than the fourth microphone 402 and may receive a voice signal focused on the user's voice. On the other hand, the fourth microphone 402 is located as far from the user's mouth as possible compared to the third microphone 401, and may receive a noise signal focusing on ambient noise according to the user's location.

In this case, as shown in FIG. 6B, it can be confirmed that the third microphone 401 and the fourth microphone 402 are located near or far from the user's mouth based on the on/off switch 403.

The third microphone 401 and the fourth microphone 402 are included in different positions in the second smart hearing device 400 according to the embodiment of the present invention, but the third microphone 401 and the fourth microphone 402 The direction in which the tool (or hole) faces is characterized in that it collects a uniform voice signal and noise signal, and accordingly is the same for proper noise reduction.

7 illustrates the configuration of an artificial intelligence hearing system according to an embodiment of the present invention.

Referring to FIG. 7, the artificial intelligence hearing system according to an embodiment of the present invention includes a first smart hearing device 300 formed on one side and a second smart hearing device 400 formed on the other side, and the first smart hearing device 300 and a mobile device 500 that controls the second smart hearing device 400. In addition, the artificial intelligence hearing system according to an embodiment of the present invention may further include an external server 600.

The first smart hearing device 300 determines a natural language or an unnatural language for the first sound data including a voice signal and a noise signal received from a first microphone and a second microphone formed on one side, and the determined natural or unnatural language The sound of one side is provided by setting a first control parameter due to matching of similar data to. In addition, the second smart hearing device 400 determines a natural language or an unnatural language for the second sound data including a voice signal and a noise signal received from the third microphone and the fourth microphone formed on the other side, and the determined natural language Alternatively, the sound of the other side is provided by setting a second control parameter due to matching of the similar data with an unnatural language.

The first smart hearing device 300 and the second smart hearing device 400 include a first microphone and a third microphone positioned near the user's mouth, and a second microphone positioned at a distance from the user's mouth, and It may include a fourth microphone. In this case, the first microphone and the third microphone may collect a user's voice signal from one side and the other side, and the second microphone and the fourth microphone may collect a noise signal from one side and a noise signal from the other side.

More specifically, the first smart hearing device 300 may be mounted on the user's left ear, receives the user's voice signal from the left through the first microphone located near the user's mouth, and is based on the user's mouth. It is possible to receive a noise signal from the left through the second microphone located at a distance. In addition, the second smart hearing device 400 may be mounted on the user's right ear, receives the user's voice signal from the right through a third microphone located near the user's mouth, and provides a separation distance based on the user's mouth. It is possible to receive a noise signal from the right through the fourth microphone located at.

In this case, the first microphone of the first smart hearing device 300 and the third microphone of the second smart hearing device 400, and the second microphone and the second smart hearing device 400 of the first smart hearing device 300 Each of the fourth microphones of) is characterized in that they are paired with each other, and a microphone paired with any one microphone may also be automatically set according to a setting applied to any one microphone. For example, when the volume of the first microphone is adjusted by the first control parameter, the volume of the paired third microphone may also be automatically adjusted. As another example, when the power of the second microphone of the first smart hearing device 300 is turned on, the power of the fourth microphone of the paired second smart hearing device 400 may also be automatically turned on. .

Each of the first smart hearing device 300 and the second smart hearing device 400 may set a first control parameter and a second control parameter of different parameter values based on the user's left hearing data and right hearing data. .

As an example, each of the first smart hearing device 300 and the second smart hearing device 400 may include a personal hearing profile for left and right sides of a user who uses a hearing aid. For example, since the user's left and right hearing data may be different, each of the first smart hearing device 300 and the second smart hearing device 400 has a volume preferred by the user, a specific recognizable volume, and a specific frequency. Wow, it may include user-customized hearing data including an amplification value, a volume, and a frequency range so as not to feel a sense of difference. In this case, the user's hearing data may be stored and maintained in the mobile device 500 and the external server 600.

However, the hearing data is not limited to items or numerical values such as amplification value, volume, and frequency. For example, the hearing data is nonlinear compression information that amplifies small sounds large and reduces loud sounds small, directional information that accurately detects the direction in which the sound is heard, and amplifies the received sound by a microphone so that it can be heard without other noise. The user's preference and numerical values for at least one or more of information for helping feedback and noise reduction information for reducing noise may be further included.

The first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention are based on the user's hearing data and the result information on the similar data determined in natural or non-natural language. At least one of a first control parameter and a second parameter of an amplification value change according to a noise change, a volume control, and a frequency control may be set to provide customized hearing aid services on the left and right.

As an example, the first smart hearing device 300 determines natural or non-natural language by analyzing sound data of a digital signal received from a first microphone and a second microphone, and a noise signal, and matched according to the determination result. By setting the first control parameter in the similar data, the balance of at least one of the amplification value change, volume control, and frequency control is adjusted, and the digital signal for the adjusted signal and feedback is converted into an analog signal to the user as the sound on the left. I can deliver.

As another example, the second smart hearing device 400 determines natural language or non-natural language by analyzing sound data of a digital signal received from a third microphone and a fourth microphone, and a noise signal, and matched according to the determination result. By setting the second control parameter in the similar data, the balance of at least one of the amplification value change, volume control, and frequency control is adjusted, and the digital signal for the adjusted signal and feedback is converted into an analog signal to the user as the sound on the right. I can deliver.

For example, at least one or more of amplification value, volume, and frequency according to sound data received from a first microphone, a second microphone, a third microphone, and a fourth microphone is out of a preset or preferred reference range by the user. I can. This may be due to any one of a factor according to a change in the environment in which the user is located, a factor according to a change in the user's voice, a factor of a mechanical error, and a height factor due to a natural or unnatural language. Accordingly, the first smart hearing device 300 and the second smart hearing device 400 apply the first control parameter and the second parameter to the sound data to provide at least one of an amplification value, a volume, and a frequency for the sound data. The balance is adjusted, and a digital signal according to the adjusted balance can be converted into an analog signal (sound energy) and provided to the user as sound.

In other words, the first smart hearing device 300 and the second smart hearing device 400 of the artificial intelligence hearing system according to the embodiment of the present invention are used as the first microphone, the second microphone, and the second microphone according to changes in the environment and noise of the user. 3 By analyzing sound data including voice signals and noise signals received from the microphone and the fourth microphone to determine natural or non-natural language, and by setting the first control parameter and the second control parameter for matching similar data, the user It is possible to provide a hearing aid service optimized for changing situations without separately adjusting the volume or adjusting the frequency, and improving the convenience of using the hearing aid.

In other words, each of the first smart hearing device 300 and the second smart hearing device 400 of the artificial intelligence hearing system according to the embodiment of the present invention provides a user with a left sound and a right sound customized, It is possible to provide three-dimensional recognition of environmental changes and noise changes according to the sound direction of the left and right sides.

The mobile device 500 controls the first smart hearing device 300 and the second smart hearing device 400.

As shown in FIG. 7, the first smart hearing device 300, the second smart hearing device 400, and the mobile device 500 may transmit and receive data through Bluetooth communication, which is a short-range wireless communication module. For example, the mobile device 500 may determine sound data including a voice signal and a noise signal from the first smart hearing device 300 and the second smart hearing device 400 through Bluetooth communication, natural language or non-natural language. As a result of the determination, similar data and information related to feedback may be received.

Thereafter, the mobile device 500 transmits the information received from the first smart hearing device 300 and the second smart hearing device 400 to the external server 600 through wireless data communication of Ethernet/3G, 4G, and 5G. I can.

In this case, the mobile device 500 is a terminal device possessed by a user who uses the first smart hearing device 300 and the second smart hearing device 400, and includes a personal computer (PC), a laptop computer, and It may be at least one of a smart phone, a tablet, and a wearable computer, and services such as configuring a service screen, inputting data, transmitting and receiving data, and storing data under the control of a web/mobile site or a dedicated application. You can perform the entire operation. In addition, the mobile device 500 may refer to an application downloaded and installed in the mobile device.

Depending on the embodiment, the mobile device 500 may display a screen including a plurality of items located in each of a plurality of areas through a display unit (not shown), and may be haptic contact or tactile. Another screen including at least one item related to a function may be displayed based on a touch-sensitive surface, a sensor, or a set of sensors that accepts an input from a user based on a contact. In addition, the mobile device 500 may receive a user's selection input through an input unit (not shown) such as a keyboard, a touch display, a dial, a slider switch, a joystick, and a mouse, and includes an audio module, a speaker module, and a vibration module. Information related to a customized hearing aid service may be output through an output unit (not shown).

The mobile device 500 may be interlocked with each of the first smart hearing device 300 and the second smart hearing device 400 to provide a screen for testing the user's hearing and information related to various reports according thereto. . In this case, the report may be a history index or record for a customized hearing aid service over time.

In addition, the mobile device 500 includes sound data including voice signals and noise signals from the first smart hearing device 300 and the second smart hearing device 400, a determination result determined in natural or non-natural languages, similar data, and Information related to feedback can also be databaseized.

In addition, the mobile device 500 may turn on or off the power of each of the first smart hearing device 300 and the second smart hearing device 400 according to a user's selection input, and the first Numerical values such as an amplification value, volume, and frequency of the smart hearing device 300 and the second smart hearing device 400 may be manually controlled.

In addition, the mobile device 500 is paired with a serial number or device information assigned to each of the first smart hearing device 300 and the second smart hearing device 400, and the first smart hearing device 300 and the second Battery management, loss management, and failure management of the smart hearing device 400 may be performed.

The external server 600 numerically stores and stores user information, hearing data according to the user information, and an appropriate range of the user's preferred amplification value, volume, and frequency, and the first smart hearing device 300 corresponding to the user information , The second smart hearing device 400 and the mobile device 500 may be matched to form a database. For example, the external server 600 includes sound data, natural language or non-natural language by the first smart hearing device 300 and the second smart hearing device 400 received through the mobile device 500 at preset time intervals. As a result of the determination determined as, similar data and information related to feedback may be converted into a database.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose computers or special purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodyed in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description by those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

100, 300, 400: smart hearing device
301: first microphone
302: second microphone
303, 403: on/off switch
304, 404: speaker
305, 405: charging module
401: third microphone
402: fourth microphone
500: mobile device
600: external server

Claims (16)

A receiver configured to receive sound data of a voice signal and a noise signal from a first microphone and a second microphone formed at one side;
A determination unit for determining a natural or non-natural language for the sound data by comparing the digital flow of the sound data with a previously stored graph pattern;
A processing unit that matches similar data for the determined natural language or non-natural language based on a database including a natural language region and an unnatural language region; And
A providing unit that sets a control parameter in a specified natural language or non-natural language according to the matched similar data and provides the converted sound to the user
Including,
The first microphone
It is located near the user's mouth and collects the user's voice signal,
The second microphone
Located at a distance from the user's mouth to collect noise signals,
A third microphone for collecting the user's voice signal from a smart hearing device different from the first microphone
They are paired with each other, and the microphone paired with any one of the microphones is also automatically set according to the settings applied to any one of the microphones,
The second microphone and a fourth microphone collecting a noise signal from the other smart hearing device
The second microphone and the fourth microphone are paired with each other, and a microphone paired with any one of the second microphone and the fourth microphone is also automatically set according to a setting applied to one of the second microphone and the fourth microphone. Smart hearing device to distinguish. The method of claim 1,
Feedback delivery unit for providing feedback in response to the specified natural language or non-natural language according to the matched similar data
Smart hearing device for distinguishing a natural language or non-natural language further comprising a. The method of claim 1,
The receiver
The voice signal and the noise signal are sensed from the first microphone and the second microphone formed at different positions, and the voice signal and the noise signal are converted to digital and received as the sound data. Smart hearing device to distinguish. The method of claim 3,
The determination unit
A natural language or visa that compares the digital flow of the digitally converted sound data with a graph pattern of a volume graph for a pre-stored time and a volume graph for a frequency, and determines a natural or non-natural language representing a specific pattern from the values of the graph Smart hearing device that distinguishes salmon. The method of claim 1,
The processing unit
Based on the database, similar data of similar words and sentences are matched with the sound data determined in natural language, or similar environmental noise, noise (noise) and similar data of new sound are matched with the sound data determined in non-natural language. A smart hearing device that distinguishes between natural or non-natural languages. The method of claim 5,
The provision unit
Based on the user's hearing data and the matched similar data, at least one of the amplification value change according to the environment change, volume control, and frequency control is set based on the specified natural language or non-natural language. A smart hearing device that provides sound and distinguishes between natural or unnatural language. The method of claim 6,
The provision unit
By setting the control parameter in the sound data of the digital signal received from the first microphone and the second microphone to adjust at least one or more of amplification value change, volume control, and frequency control, the adjusted signal is A smart hearing device for distinguishing natural or non-natural languages, characterized in that converting a digital signal into an analog signal and providing a sound of one side to the user. The method of claim 2,
The feedback transmission unit
When the similar data of similar words and sentences are matched with the sound data determined in natural or non-natural languages through the processing unit, the natural language or non-natural language is distinguished by providing feedback of words and sentences in response to the similar data. Smart hearing device. The method of claim 8,
The feedback transmission unit
When the similar data of similar words and sentences with the sound data are matched based on artificial intelligence (AI), feedback of words and sentences is extracted in response to a question from the user's voice, and a voice message A smart hearing device that distinguishes between natural or non-natural languages, characterized in that it is transmitted to. A natural language or non-natural language is determined for the first sound data including a voice signal and a noise signal received from the first microphone and the second microphone formed on one side, and a reduction due to matching of similar data to the determined natural language or non-natural language 1 A first smart hearing device that sets a control parameter to provide a sound of one side;
A natural language or non-natural language is determined for the second sound data including a voice signal and a noise signal received from the third microphone and the fourth microphone formed on the other side, and the similar data is matched with the determined natural language or non-natural language. A second smart hearing device configured to set a second control parameter to provide sound from the other side; And
A mobile device that controls the first smart hearing device and the second smart hearing device
Including,
Each of the first smart hearing device and the second smart hearing device
The first microphone and the third microphone positioned near the user's mouth, and the second microphone and the fourth microphone positioned at a distance from the user's mouth,
The first microphone and the third microphone
One side and the other side collect the user's voice signal,
The second microphone and the fourth microphone
Collecting the noise signal of one side and the noise signal of the other side,
The first microphone and the third microphone, and each of the second microphone and the fourth microphone are paired with each other, and the microphone paired with the one microphone is also automatically set according to the setting applied to the one microphone. Hearing system. delete The method of claim 10,
Each of the first smart hearing device and the second smart hearing device
An artificial intelligence hearing system that provides a customized sound of one side and the sound of the other side to the user to perceive environmental changes and noise changes according to the sound direction of the left and right sides in three dimensions. The method of claim 10,
Each of the first smart hearing device and the second smart hearing device
The artificial intelligence hearing system, characterized in that the first control parameter and the second control parameter of different parameter values are set based on the user's left hearing data and right hearing data. The method of claim 10,
The mobile device is
An artificial intelligence hearing system that controls any one or more of power on/off, signal collection and control parameter setting of each of the first smart hearing device and the second smart hearing device according to a user's selection input . In the operating method of a smart hearing device for distinguishing natural or non-natural languages,
Receiving sound data of a voice signal and a noise signal from a first microphone and a second microphone formed at one side;
Comparing the digital flow of the sound data with a previously stored graph pattern to determine a natural language or an unnatural language for the sound data;
Matching similar data for the determined natural language or non-natural language based on a database including a natural language region and an unnatural language region; And
Setting a control parameter in a specified natural or non-natural language according to the matched similar data to provide a converted sound to a user
Including,
The first microphone
Located near the user's mouth to collect the user's voice signal,
The second microphone
Located at a distance from the user's mouth to collect noise signals,
A third microphone for collecting the user's voice signal from a smart hearing device different from the first microphone
They are paired with each other, and the microphone paired with any one of the microphones is also automatically set according to the settings applied to any one of the microphones,
The second microphone and a fourth microphone collecting a noise signal from the other smart hearing device
The second microphone and the fourth microphone are paired with each other, and a microphone paired with any one of the second microphone and the fourth microphone is also automatically set according to a setting applied to one of the second microphone and the fourth microphone. How to distinguish smart hearing devices. The method of claim 15,
Providing feedback in response to the specified natural language or non-natural language according to the matched similar data
A method of operating a smart hearing device for distinguishing natural or non-natural languages further comprising a.

Images