KR102355182B1 - Voice interacting method of hearing assisting apparatus using interactive process - Google Patents

Abstract

The present invention relates to a fitting method of a hearing aid device, and more particularly, to a hearing aid using an interactive process in which fitting for each hearing characteristic of a user is performed using an interactive process including a voice conversation with a user without direct consultation with a hearing aid expert. It relates to a fitting method of an auxiliary device.
According to the present invention, a method for fitting a hearing aid device using an interactive process includes: performing a hearing aid function according to a preset condition of the hearing aid device; and transmitting to the user, acquiring a response voice from the user, and maintaining or changing a setting condition according to at least one of a user voice characteristic or an auditory characteristic related to the acquired response voice characteristic.

Description

The method of voice conversation of a hearing aid using an interactive process

The present invention relates to a voice conversation method of a hearing aid device, and more particularly, to perform fitting according to a user's voice characteristics and/or auditory characteristics using an interactive process including voice conversation with a user without direct consultation with a hearing aid expert. It relates to a voice conversation method of a hearing aid using an interactive process.

In general, the human auditory organ is composed of an external ear, a middle ear, and an internal ear. At this time, anatomically examining the path through which sound can be heard in the human ear as described above. First, sound energy transmitted through a chain transmission process of air particles is collected by the outer ear from the skull, and then the sound collected in the outer ear is transmitted through the outer ear canal. It is transmitted to the eardrum and vibrates the eardrum. And the above vibration is again transmitted to the inner ear through the middle ear, and the sound energy transferred to the inner ear moves the lymphatic fluid in the cochlea called the cochlea, and thousands of microscopic hair cells in the cochlea detect the movement of the lymph fluid and generate electrical energy In this way, the converted electrical energy is transmitted to the brain through the auditory nerve, and humans perceive sound.

However, when the functions for transmitting sound or electrical energy from the outer ear to the auditory nerve as described above are not smooth, hearing is deteriorated or impaired hearing occurs, and in general, such a sound is converted to electrical energy in such a patient. Hearing aids that can compensate for this are essential. These hearing aids amplify or modify the sound in the range that normal people can hear, so that the hearing-impaired person can perceive sound to the same degree as normal people.

It is a device to make it possible. However, since the hearing aid is a very small device that is small enough to be worn inside or behind the user's ear during normal use, before using the hearing aid, consult a hearing aid fitting expert according to a prescription to amplify a frequency range that is difficult for the user to recognize. individually programmed by

For programming by a hearing aid fitting expert, there is a problem in that it is necessary to repeatedly make reservations, visits, and prescriptions for the expert, which causes inconvenience to the user.

The present invention enables fitting suitable for a user to be performed using an interactive process including voice conversation and text display with the user without direct consultation with a hearing aid expert, and improves the user's cognitive ability and socially An object of the present invention is to provide a voice conversation method of a hearing aid device using an interactive process capable of solving an isolation problem.

The present invention provides a method for voice conversation in a hearing aid device using an interactive process, comprising the steps of: performing a hearing aid function according to a preset condition of the hearing aid device; And, a second process of delivering the voice conversation content determined or selected in the first process to the user as sound or vibration, a third process of acquiring a response voice from the user, and recognizing the user's response voice in the third process, , performing an interactive process including a fourth process of determining or selecting new voice conversation contents corresponding to the recognized response contents or not related to the response contents.

In addition, the voice conversation method preferably includes the step of maintaining or varying the setting condition according to at least one of a user voice characteristic or an auditory characteristic related to the acquired characteristic of the user's response voice.

In addition, the step of transmitting the sound or vibration is preferably performed according to a preset cycle or performed in a specific mode among acoustic environment modes.

In addition, it is preferable that the characteristics of the response voice include at least one of the volume of the voice, the content of the voice, the accuracy of the voice content, and the response speed.

In addition, the performing step of the interactive process preferably includes the step of confirming the user's satisfaction by voice expression, and the step of maintaining or varying the setting condition according to the user's response voice in the confirmation step additionally.

The present invention uses an interactive process including voice dialogue and text display with the user without direct consultation with a hearing aid expert so that fitting suitable for the user is repeatedly and naturally performed through conversation with the user. It has the effect of improving the cognitive ability and solving the social isolation problem.

1 is a block diagram of a voice conversation system for performing a voice conversation method of a hearing aid device using an interactive process.

Hereinafter, the present invention will be described in detail through examples and drawings. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that various modifications, equivalents, and/or alternatives of the embodiments of the present invention are included. In connection with the description of the drawings, like reference numerals may be used for like components.

In this document, expressions such as "have", "may have", "includes", or "may include" indicate the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this document, expressions such as “A or B”, “at least one of A or/and B”, or “one or more of A or/and B” may include all possible combinations of the items listed together. . For example, "A or B", "at least one of A and B", or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

Expressions such as "first", "second", "first", or "second" used in this document may modify various elements, regardless of order and/or importance, and may modify one element to another. It is used only to distinguish it from the components, and does not limit the components. For example, the first user equipment and the second user equipment may represent different user equipment regardless of order or importance. For example, without departing from the scope of the rights described in this document, a first component may be referred to as a second component, and similarly, the second component may also be renamed as a first component.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component); When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element). On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

The expression "configured to (or configured to)" as used in this document, depending on the context, for example, "suitable for", "having the capacity to" It can be used interchangeably with "," "designed to", "adapted to", "made to", or "capable of". The term "configured (or set up to)" may not necessarily mean only "specifically designed to" in hardware. Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase "a processor configured (or configured to perform) A, B, and C" executes a dedicated processor (eg, an embedded processor), or one or more software programs stored in a memory device, to perform the corresponding operations. By doing so, it may refer to a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe specific embodiments, and may not be intended to limit the scope of other embodiments. The singular expression may include the plural expression unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meanings as commonly understood by one of ordinary skill in the art described in this document. Among the terms used in this document, terms defined in a general dictionary may be interpreted with the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, have an ideal or excessively formal meaning. not interpreted In some cases, even terms defined in this document cannot be construed to exclude embodiments of this document.

The interactive process is performed by driving an application installed in the hearing aid device 20 or the electronic communication device 40 in FIG. 1 , and periodically detects and monitors the user's voice and the external acoustic environment (or acoustic environment mode). It is a process of periodically and/or aperiodically performing a voice conversation function with a user. The interactive process is installed in the form of an application in the hearing aid 20 or the electronic communication device 40, and is performed by the data processor 11 of the hearing aid 20 or the data processor 31 of the electronic communication device 40 is executed Among the voice conversation functions performed by the interactive process, the user's voice characteristics and auditory characteristics are calculated or determined based on the volume of the user's voice, the content of the voice, the accuracy of the voice content, the response speed, etc., and the preset condition (eg, a fitting value) is modified or maintained. In the case of such a voice conversation function, it is like a voice chatbot, but the process of acquiring the user's voice during the voice conversation function, analyzing the acquired voice, and fitting the set condition is a fitting method using an interactive process. In the case of the fitting method using such an interactive process, it is not conducted through repeated and repeated visits and remote consultations to experts as in the prior art. This has the effect of improving the user's cognitive ability and solving the social isolation problem.

1 is a block diagram of a voice conversation system for performing a voice conversation method of a hearing aid device using an interactive process.

The voice conversation system is inserted or attached to the user's hearing organ (eg, ear, etc.) or attached to the user's skin. It is configured to include a hearing aid device 20 that transmits to the device, and an electronic communication device 40 that performs data communication with the hearing aid device 20 in a wired or wireless communication method, and the like.

The hearing aid 20 includes an input unit 1 that obtains an input from a user (eg, power on/off, communication connection, operation/stop of an interactive process, etc.) and applies it to the data processor 11 , and power A display unit 3 for displaying a state or an operating state (operation/stop, etc.) of an interactive process, a microphone 5 for obtaining a sound and generating an electric signal corresponding thereto and applying it to the data processor 11; The speaker 7 that receives an electrical signal from the processor 11 to generate sound or vibration and transmit it to the user, the communication unit 9 performing wired or wireless communication with the electronic communication device 40, and the above-described components control the sound, store the setting conditions for processing the sound, and process the electrical signal from the microphone 5 according to the set condition, and apply the processed electrical signal to the speaker 7 or receive from the communication unit 9 It includes a data processor 11 that applies the electrical signal to the speaker 7 to perform a hearing aid function, and transmits an electrical signal from the microphone 5 to the electronic communication device 40 through the communication unit 9 is composed by However, the power supply unit (not shown), the input unit 1, the display unit 3, the microphone 5, the speaker 7, and the communication unit 9, etc. are technically recognized by those of ordinary skill in the art to which the present invention belongs. As such, a detailed description thereof is omitted.

The data processor 11 includes a processor (eg, CPU, MICROPROCESSOR, MCU, etc.) that independently performs an interactive process or performs an interactive process through data communication with the electronic communication device 40 in addition to the above-described hearing aid function, and an interactive process storage space (for example, memory, etc.) for storing applications including is comprised of A voice conversation method using an interactive process performed by the data processor 11 and a fitting method during execution of the interactive process will be described in detail below.

The electronic communication device 40 corresponds to, for example, an electronic device such as a smartphone or tablet, and obtains an input from a user (eg, communication connection, operation/stop of an interactive process, etc.) to obtain a data processor ( 31), the display unit 23 for displaying the setting conditions, acoustic environment mode, etc. of the hearing aid device 20, and wired or wireless communication with the hearing aid device 20 or an external server, etc. It is configured to include a communication unit 29 that performs, and a data processor 31 that performs a voice conversation method using an interactive process by controlling the above-described components. However, the power supply unit (not shown), the input unit 21, the display unit 23, the communication unit 29, etc. are only techniques recognized naturally by those skilled in the art to which the present invention belongs, and the description thereof will be omitted.

The data processor 31 controls the communication unit 29 to communicate with the hearing aid 20 while performing an interactive process with a processor (eg, CPU, MICROPROCESSOR, MCU, etc.) and an application including an interactive process and a storage space (eg, memory, etc.) for storing setting conditions for processing sound, acoustic environment mode (eg, conversation mode, quiet place mode, TV watching mode, etc.), etc. . A voice conversation method using an interactive process performed by the data processor 31 and a fitting method during execution of the interactive process will be described in detail below.

In the present embodiment, the setting condition is configured to include fitting values corresponding to the characteristic of the user's auditory organ or the hearing symptom or the acoustic environment mode.

First, a case in which the hearing assistance device 20 independently performs an interactive process with respect to the electronic communication device 40 will be described. The hearing assistance device 20 performs the interactive processor while performing the above-described hearing assistance function.

The data processor 11 executes an application including an interactive process, and performs a hearing assistance function according to a preset or default stored setting condition. With this hearing aid function, the data processor 11 processes the sound applied from the microphone 5 using the setting conditions corresponding to the acoustic environment mode input by the input unit 1 or set by default to the speaker 7 . approve The data processor 11 performs an interactive process to periodically detect and monitor the user's voice and the external acoustic environment (or acoustic environment mode), calculate and store the user's voice, or determine and store the external acoustic environment do.

The data processor 11 performs an interactive process, such as a generating unit for generating voice conversation content, a voice generating unit for converting the generated voice conversation content into voice, and a voice recognition unit for recognizing a user's voice. .

The voice conversation content generating unit may start a conversation according to a preset period, or may start a voice conversation in a specific mode (eg, a conversation mode, a quiet environment mode, etc.) among acoustic environment modes. Voice conversation contents (eg, conversation contents about weather/movie/exercise, etc.) are, for example, included in the application in the form of a corpus, and the contents may be updated by communication with the electronic communication device 40 . . Also, the voice conversation content may include content corresponding to the acoustic environment mode (eg, content of a TV program in the TV viewing mode, and content of weather or exercise in the quiet environment mode). The voice conversation content generating unit determines or selects the voice conversation content and applies it to the voice generating unit.

In addition, the voice generating unit converts the voice conversation content applied from the voice conversation content generating unit into voice and applies it to the speaker 7 so that the sound is emitted.

Further, the voice recognition unit recognizes the voice from the microphone 5, and applies the recognized response content to the voice conversation content generating unit.

The voice conversation contents generating unit determines or selects new voice conversation contents corresponding to or related to the authorized response contents, or determines or selects new voice dialogue contents not related to the authorized response contents. The voice conversation content generating unit applies the new voice conversation content to the voice generating unit.

As described above, the data processor 11 may independently or actively perform an interactive process, which is a voice conversation function including generation of voice conversation contents, voice generation and sound emission, voice recognition, and the like, without a user's command or input. .

The data processor 11 reads and reproduces the pre-stored voice conversation content, applies it to the speaker 7 , and acquires an electrical signal including the user's response voice from the microphone 5 to determine the size of the user's voice, By calculating or judging user voice characteristics and/or auditory characteristics related to content (eg, 'I can't hear well, can you tell me again', etc.), accuracy or response speed of voice content, etc., the user voice characteristics and/or auditory characteristics Accordingly, a preset setting condition (eg, a fitting value) is corrected or maintained. In the case of voice volume, for example, if the current user's voice volume (or energy, etc.) is equal to or smaller than the average user's voice volume, the data processor 11 clearly recognizes the user's voice. (Hearing characteristic that the voice is well heard according to the current setting condition) and maintain the setting condition, otherwise the user did not clearly recognize the voice (hearing characteristic that the voice is not well heard according to the current setting condition) to change the setting conditions. In addition, in the accuracy of the voice content, the data processor 11 clearly recognizes the voice by the user if the voice content from the user through the above-described voice recognition unit is a response that matches the content of the voice transmitted through the speaker 7 . The setting condition is maintained by judging that it has been set, otherwise it is determined that the user did not clearly recognize the voice and the setting condition is changed. In addition, the data processor 11 calculates the user's response speed during the voice conversation, and compares the reference response speed or the previously calculated average response speed to determine the user's auditory characteristics to change or maintain the setting conditions. have.

The data processor 11 actively or independently performs the above-described interactive process, wherein during the performance of the interactive process, the transfer of the voice conversation, the acquisition of the user response voice and the user voice characteristics associated with the acquired response voice and/or the data processor 11 The process of calculating or judging auditory characteristics is repeatedly performed.

In addition, the data processor 11 may obtain a sound from the microphone 5 according to an interactive process and calculate the user's voice volume, calculate the average user's voice volume, or determine the acoustic environment mode.

In addition, the data processor 11 may check the user's satisfaction by voice expression through the speaker 7 according to an interactive process. For example, the data processor 11 transmits "Can you hear your voice well?" to the user through the speaker 7, and the user's response voice content from the microphone 5 is "You can hear the sound, but the speech is clear." In the case of "I don't do it", the data processor 11 changes the setting condition to increase the gain in the high frequency region of 2KHz or more, and again "Can you hear the voice well?" While repeatedly performing the above, etc., increasing or decreasing the overall gain of sound or voice, increasing or decreasing the compression point from the present, lowering or increasing the compression ratio, etc. It is also possible to determine the setting conditions.

Also, the data processor 11 may variably adjust the setting conditions for each acoustic environment mode by using the changed setting conditions.

By such an interactive process, the data processor 11 provides a voice conversation function, so that it is possible to prevent or treat a user's brain disease and cognitive decline through conversation and communication.

As described above, it has been described that the fitting method using the interactive processor is executed by the data processor 11 of the hearing aid 20 . Hereinafter, the electronic communication device 40 controls the hearing aid 20 to A fitting method using an interactive process is described.

The data processor 31 executes an application including an interactive process, and maintains a communication connection with the hearing aid device 20 through the communication unit 29 .

The data processor 31 performs the above-described interactive process to start a voice conversation, the voice conversation contents are stored in the data processor 31 , and the voice conversation contents are transmitted to the hearing aid device 20 through the communication unit 29 . do. The data processor 11 receives the voice conversation content through the communication unit 9 and applies it to the speaker 7 to emit sound. In addition, the data processor 11 transmits the user's voice from the microphone 5 to the electronic communication device 40 through the communication unit 9 . As described above, the data processor 31 calculates or determines the volume of the user's voice, the content of the voice, the accuracy of the voice content, the response speed, etc. according to the transmitted user response voice, that is, the voice characteristic related to the user's response voice. and/or determining an auditory characteristic to vary or maintain a set condition.

In addition, the data processor 31 controls the hearing assistance device 20 to check the user's satisfaction with the set condition, and maintains or changes the set condition as described above.

In addition, the data processor 31 transmits the changed setting condition to the hearing aid 20 through the communication unit 29 , and the data processor 11 receives and stores the changed setting condition through the communication unit 9 , , the sound or voice from the microphone 5 is processed according to the stored setting conditions, and the sound is emitted through the speaker 7 .

In addition, the data processor 31 displays the content of the setting condition through the display unit 23 so that the user can check it, and may change and store the setting condition by a variable input from the input unit 21 .

As described above, when the setting condition is changed, the data processor 11 performs a hearing aid function according to the changed setting condition, and repeatedly performs an interactive process to maintain or change the setting condition.

In the case of voice recognition and generation of conversation content by the data processor 11 and/or data processor 31 described above, the electronic communication device 40 is communicatively connected to a voice recognition server (not shown), and the voice recognition server The user's response voice may be authorized, the voice recognition server may recognize the response voice, generate conversation content corresponding to the recognized response content, and transmit it to the electronic communication device 40 . The voice recognition server stores more various voice conversation contents, and it is easy to update the voice conversation contents.

At least a portion of an apparatus (eg, a processor or functions thereof) or a method (eg, operations) according to various embodiments is stored in a computer-readable storage medium in the form of, for example, a program module It can be implemented with stored instructions. When the instruction is executed by a processor, the one or more processors may perform a function corresponding to the instruction. The computer-readable storage medium may be, for example, a memory.

Computer-readable recording media include hard disks, floppy disks, magnetic media (eg, magnetic tape), optical media (eg, CD-ROM, DVD (Digital Versatile Disc), magnetic- It may include an optical medium (eg, a floptical disk), a hardware device (eg, ROM, RAM, or flash memory, etc.), etc. In addition, program instructions include It may include not only machine code but also high-level language code that can be executed by a computer using an interpreter, etc. The above-described hardware device may be configured to operate as one or more software modules to perform operations of various embodiments, and the The same goes for the station.

The processor or functions by the processor according to various embodiments may include at least one or more of the aforementioned components, some may be omitted, or may further include additional other components. Operations performed by a module, a program module, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or in a heuristic method. Also, some operations may be executed in a different order, omitted, or other operations may be added.

As described above, the present invention is not limited to the specific preferred embodiments described above, and anyone with ordinary skill in the art to which the invention pertains can use various methods without departing from the gist of the present invention as claimed in the claims. It goes without saying that modifications are possible, and such modifications are intended to be within the scope of the claims.

20: hearing aid device 40: electronic communication device

Claims (5)

performing a hearing aid function according to a preset condition of a hearing aid device;
A first process of determining or selecting the content of a voice conversation while performing the hearing assistance function, a second process of delivering the content of the voice conversation determined or selected in the first process to the user as sound or vibration, and obtaining a response voice from the user An interactive process is performed including a third process of performing a third process, and a fourth process of recognizing the user's response voice in the third process, and determining or selecting a new voice conversation content corresponding to the recognized response content or not related to the response content comprising the steps of
The voice conversation method includes maintaining or varying a setting condition according to a user voice characteristic and an auditory characteristic related to the acquired characteristic of the user's response voice,
A voice conversation method of a hearing aid device using an interactive process, characterized in that the characteristics of the response voice include the volume of the voice, the content of the voice, the accuracy of the voice content, and the response speed. The method of claim 1,
The setting condition includes a fitting value. A voice conversation method of a hearing aid using an interactive process. The method of claim 1,
The step of transmitting the sound or vibration is performed according to a preset cycle or is performed in a specific mode among acoustic environment modes. delete The method of claim 1,
The performing step of the interactive process further includes the step of confirming the user's satisfaction by voice expression, and the step of maintaining or varying the setting condition according to the user's response voice in the confirmation step. How to use a hearing aid for voice communication.

Images