KR101450014B1 - Smart user aid devices using bluetooth communication - Google Patents

Abstract

The present invention relates to smart user aid technology generally using Bluetooth communications. More particularly, the present invention provides a smart user aid device with the optimum solution enabling a person with a hearing loss to enjoy the entertainment content (for example, game and music) of the smart terminal with high quality. For this, the quality of the life of the person with the hearing loss is improved by safely enjoying the entertainment outdoors by automatically fitting the frequency feature of the smart user aid device with an application of the smart terminal according to the intention of the user and the hearing properties of both ears by providing left and right stereo audios, and mounting four directional microphones in front, rear, left, and right sides.

Description

[0001] Smart user aid devices using bluetooth communication [

The present invention relates generally to smart user assistive technology using Bluetooth communication. More specifically, the present invention relates to a smart phone that provides an optimal solution for enabling a person with a difficulty in hearing (hereinafter referred to as a " hard-of-hearing ") to enjoy entertainment contents Provide an assistive device. To this end, the present invention provides a left / right stereo audio system, in which the frequency characteristics of a smart user assist device are automatically fitted to the application of a smart terminal in accordance with the user's intention to listen to both of the ears and user's intention, It improves the quality of life of the hard-of-hearing by allowing the entertainer to enjoy the entertainment safely from the outside.

In recent years, smart terminals such as smart phones, smart pads, and smart TVs have actively spread. These smart terminals are equipped with smart operating systems such as Android, iOS, and Tezen, and can be installed freely according to their needs. In addition, it has a wireless network, so digital data can be exchanged with an external server, and it is possible to install a large number of applications or to embed multimedia data with mass storage.

Because of these functions, smart terminals are being used for various purposes such as photo shooting, movie shooting, music playback, video playback, game play, and broadcast reception. Accordingly, in recent years, people who are using smart terminals indoors and outdoors are often witnessed, and many of them are enjoying their own entertainment while enjoying earphones or headphones.

On the other hand, there are many people with hearing loss due to aging and noise in the working environment. They are living with inconvenience and in severe cases hearing aids such as hearing aids are used to compensate for the decreased hearing. In general, when the hearing loss is reduced, the hearing ability is not lowered uniformly over the whole of the audio frequency band, but the sound is not heard only in some frequency bands. At this time, it is general that frequency bands that can not be heard like this are different for each person depending on their circumstances.

Hearing aids were produced after a long period of delicate work by the engineer (Chung-kwan) working in the store for individual hearing difficulties. As a result of this process, the hearing aid is basically made to be custom-made, and the price of the hearing aid is as high as several million won. Despite this high price, conventional hearing aids do not adapt to the ever-changing surroundings (eg, noise) and user intent (eg, conversation, listening to music, etc.). Conventional hearing aids are merely a matter of enabling the daily life to be as simple as recognizing the surrounding situation or communicating with the surrounding people without any problem.

Conventionally, even if a hearing aid is deteriorated, it is only a level that makes daily life unhappy even if a hearing aid is used, and it is practically impossible to enjoy entertainment such as music and games, and thus the life satisfaction is rapidly deteriorated. Accordingly, the audio processing policy is adaptively changed according to the individual's auditory characteristics, the surrounding environment, and the user's intention, thereby enhancing the life satisfaction by making it possible to enjoy both daily life and entertainment even in the case of the ordinary person or the hearing impaired The development of a technology that can do this is desired.

1. Korean Patent Application No. 10-2004-0052145 "Amount of ear-type digital hearing aid improved directivity"

2. Korean Patent Application No. 10-2008-0008467, entitled " Method for setting channels of a multi-channel digital hearing aid in accordance with Korean frequency characteristics, and a multi-channel digital hearing aid using the same "

3. Korean Patent Application No. 10-2008-0093077 "RF channel automatic setting hearing system and method"

4. Korean Patent Application No. 10-2009-0129690 "Radio-integrated hearing aid"

5. Korean Patent Application No. 10-2010-0017413 "Bluetooth Headset Combined with Hearing Aid and Control Method Thereof"

6. Korean Patent Application No. 10-2011-0007115 entitled "

7. Korean Patent Application No. 10-2011-0092759 "Hearing aid, hearing aid electronic system and hearing aid method"

8. Korean Patent Application No. 10-2012-0080088 "Hearing Aid Insertion-Type Hearing Aid Having a Hazardous Element Detection Function and Hazardous Element Detection System Using It"

9. Korean Patent Application No. 10-2012-0082438 "Cordless ear-type hearing aid system equipped with remote control function and control method thereof"

It is an object of the present invention to provide a smart user assistive technology that generally uses Bluetooth communication. In particular, it is an object of the present invention to provide an optimal solution that enables auditory difficulties to enjoy entertainment contents (e.g., games, music) of a smart terminal with high quality. In other words, stereo audio is provided, but the application of the smart terminal is adapted to the frequency characteristics of the smart user assist device by the application of the smart terminal according to the audible characteristics of both ears and the user's intention and the entertainment of the outside is safely enjoyed by attaching the four- It is an object of the present invention to improve the quality of life of a hearing-impaired person.

According to an aspect of the present invention, there is provided a smart user interface device for interfacing with an external smart terminal (e.g., a smart phone) via Bluetooth and for processing a stereo audio signal according to a device operation mode including an entertainment mode and a hearing aid mode, An auxiliary device, comprising: a Bluetooth communication unit (180) for wirelessly receiving stereophonic entertainment audio from an external smart terminal; A set of stereo microphones 160 arranged in correspondence with the left and right ears of the user in the smart user auxiliary device to collect ambient audio in a stereotype; A fitting data storage unit 140 for storing fitting data having amplification profiles for respective right and left ears according to frequency bands for each device operation mode of the smart user auxiliary device; A stereo DSP unit 130 for non-uniformly amplifying the entertainment audio provided from the Bluetooth communication unit and the peripheral audio provided from the stereo microphone set for each frequency band according to the fitting data for each device operation mode stored in the fitting data storage unit; A stereo output unit 120 for stereo outputting the audio signal to the user as a result of amplification of the stereo DSP; And a device control unit (110) that controls overall functions of the smart user auxiliary device in response to the device operation mode of the smart user assistive device.

The stereo microphone set 160 includes a left microphone corresponding to a left ear of a user and a right microphone corresponding to a user's right ear as a set of directional stereo microphones. The stereo DSP unit 130 receives both the audio signal of the left microphone and the left audio of the BT in the entertainment mode and outputs the audio signal of the left microphone in the hearing mode, A left DSP unit 131 for receiving only the audio signal of the right microphone and the right audio of the right microphone in the entertainment mode, Mode, And a write DSP unit 132 for receiving only the audio signal of the microphone and amplifying the received audio signal according to the operation mode of the device in accordance with the right fitting data of the fitting data. The stereo output unit 120 includes a left DSP And a left output unit 121 and a right output unit 122 for outputting sound of each of the audio signals generated by the input / output unit 131 and the write DSP unit 132 in a stereotyped manner.

In the present invention, the left microphone comprises L1 microphones 161 and L2 microphones 162 respectively corresponding to two directions of the left ear, and the light microphones are constituted by R1 microphones 163 corresponding to the two directions of the right ear, And the left DSP unit 131 receives the audio signal of the L1 microphone through the first input port at all times and the audio signal of the L2 microphone and the BT left audio of the Bluetooth communication unit through the second input port The write DSP unit 132 always receives the audio signal of the R1 microphone through the first input port and transmits the audio signal of the R2 microphone and the audio signal of the Bluetooth communication unit through the second input port, And the BT right audio is selectively received according to the device operation mode.

Also, the smart user assistance device according to the present invention provides a test sound for each of a plurality of frequency bands divided from an audible frequency band for each of the user's ears, and analyzes the corresponding user response input, An ear modeling means 220 for generating a band-specific acoustic characteristic model, a standard amplification profile for each frequency band for compensating the acoustic characteristic model for each of the user's ears, and reflecting the frequency characteristics of the target sound according to the operation mode of the device (230) for generating fitting data for each operation mode from the standard amplification profile, and fitting data transmitting means (260) for providing the fitting data for each operation mode to the fitting data storage unit via Bluetooth communication. A computer-readable computer on which the program is recorded A recording medium; And an operation mode setting unit 150 for setting a device operation mode of the smart user auxiliary device in response to a user operation. The stereo DSP unit 130 is connected to the device operation mode set by the operation mode setting unit It is preferable that the corresponding fitting data is acquired from the fitting data storage unit and the audio signal inputted to the stereo DSP unit is non-uniformly amplified for each frequency band based on the fitting data obtained above.

In the present invention, the stereo fitting means 230 further generates fitting data for each place from the standard amplification profile reflecting the frequency characteristics of noise according to the place, and the fitting data transmitting means 260 transmits fitting data for each place via Bluetooth communication The operation mode setting unit 150 further sets a location option of the smart user auxiliary device in response to a user operation, and the stereo DSP unit 130 sets a location option set by the operation mode setting unit It is preferable that the corresponding fitting data is acquired from the fitting data storage unit and the audio signal input to the stereo DSP unit is non-uniformly amplified for each frequency band based on the obtained fitting data.

The smart user auxiliary device according to the present invention further includes a communication microphone 170 for acquiring a voice signal of a user for making a call in a hands-free mode in which a telephone communication is established by wireless connection with an external smart terminal through a Bluetooth communication unit And the Bluetooth communication unit 180 wirelessly transmits a voice signal of the user provided from the communication microphone to the smart terminal and outputs the other party voice signal wirelessly received from the smart terminal to the BT left audio and BT right audio , The left DSP unit 131 receives both the audio signal of the L1 microphone and the BT left audio of the Bluetooth communication unit, acquires the fitting data corresponding to the hands-free mode from the fitting data storage unit, and based on the acquired fitting data, The audio signal and the BT left audio are non-uniformly amplified for each frequency band, The audio DSP unit 132 receives both the audio signal of the R1 microphone and the BT right audio of the Bluetooth communication unit and acquires the fitting data corresponding to the handsfree mode from the fitting data storage unit, Signal and the BT right audio are non-uniformly amplified for each frequency band.

According to the present invention, the hearing-impaired person also has an advantage of being able to enjoy entertainment such as games and music without any problem as in the case of ordinary people by interworking with a smart terminal through Bluetooth.

Particularly, according to the present invention, an application installed in a smart terminal to suit the purpose of use of the audio signal processing characteristic of the smart user assistive device and the ambient noise environment at that time is not limited to performing auto fitting, It is possible to overcome the difficulty of hearing loss as much as possible in enjoying entertainment.

In addition, considering that the hearing-impaired person has a relatively low hearing ability, the directional microphones are installed in both ears, so that the surroundings can be grasped three-dimensionally, thereby reducing the risk of accidents. This function is provided in the present invention in consideration of the risk of an accident that an audiophile can not enjoy the entertainment outside and recognize the surrounding situation in a timely manner.

1 is a block diagram showing an internal configuration of a smart user auxiliary apparatus according to the present invention;
2 is a block diagram illustrating a functional configuration of a smart terminal in which a smart application for interworking with a smart user assistant device is implemented in the present invention;
3 is a diagram illustrating an internal connection configuration of an audio signal as a whole in a smart user auxiliary device according to the present invention;
4 is a diagram showing an audio signal connection when the smart user auxiliary device of the present invention operates as a hearing aid;
5 is a diagram illustrating an audio signal connection when the smart user auxiliary device of the present invention operates as an entertainment assist device;
6 shows an audio signal connection when the smart user auxiliary device of the present invention operates as a hands-free device.

Hereinafter, the present invention will be described in detail with reference to the drawings.

1 is a block diagram showing an internal configuration of a smart user assistance apparatus 100 according to the present invention. In the present invention, the smart user auxiliary device 100 is linked to an external smart terminal 200 through Bluetooth and processes stereo audio according to various device operation modes (e.g., entertainment mode, loudspeaker mode, hands-free mode) .

The smart user auxiliary device 100 that performs this operation includes a device control unit 110, a stereo output unit 120, a stereo DSP unit 130, a fitting data storage unit 140, an operation mode setting unit 150, A stereo microphone set 160, a communication microphone 170, and a Bluetooth communication unit 180. The individual functional elements of the smart user assist device 100 will now be described.

First, the device control unit 110 generally controls the functions of the smart user assist device 100 in response to the device operation mode.

The stereo output unit 120 provides the audio output of the smart user assistive device 100 to the user as stereo audio. In particular, as described later, the stereo DSP unit 130 outputs a stereo signal to the user as a result of amplifying the audio signal based on the fitting data for the user.

The stereo DSP unit 130 non-uniformly amplifies the audio signal to be heard by the user according to the fitting data optimized for the user, for each frequency band. The fitting data may be stored in the fitting data storage unit 140 according to the operation mode and the location of the device. The stereo DSP unit 130 controls the operation of the device, which is set by the operation mode setting unit 150, From the fitting data storage unit 140, fitting data for compensating the hearing characteristic of the user corresponding to the mode and the place option (i.e., the hearing loss characteristic indicated by the frequency band).

First, in the hearing mode in which the user is in daily life (for example, conversation, discussion, walking, shopping, etc.), the stereo DSP unit 130 non-uniformly amplifies the peripheral audio provided from the stereo microphone set according to the fitting data.

Next, in the entertainment mode in which the user enjoys music or games, the stereo DSP unit 130 non-uniformly amplifies the entertainment audio provided from the Bluetooth communication unit 180 and the surrounding audio provided from the stereo microphone set according to the fitting data. While the user is enjoying the entertainment, the noise is not interrupted but is heard together because the ability of the hearing impaired to detect the danger is lowered.

Also, in a hands-free mode in which a user performs a Bluetooth hands-free call through the smart terminal 200, the stereo DSP unit 130 converts the other party's voice signal provided from the Bluetooth communication unit 180 and peripheral audio provided from the stereo microphone set into fitting data Non-uniformly amplified according to. Even in the hands-free mode, it is desirable to let the ambient noise be heard together without blocking.

In the present invention, a stereo sound signal is processed, thereby enhancing the satisfaction of entertainment appreciation. In addition, when performing a hearing aid function or a handsfree function, it is possible to listen to the voice of the surrounding people and the voice of the communication partner in both ears, Is possible.

The fitting data storage unit 140 stores the fitting data having the amplification profiles of the frequency bands of the user's right and left ears according to the device operation mode and location of the smart user accessory device 100. In the present invention, the smart terminal 200 linked through Bluetooth generates device operation mode and fitting data for each user through a dedicated application (fitting management program). The fitting data storage unit 140 of the smart user auxiliary device 100 receives the fitting data from the smart terminal 200 through the Bluetooth communication.

The operation mode setting unit 150 sets the device operation mode of the smart user auxiliary device 100 in response to a user operation. That is, the operation mode setting unit 150 interprets the context in which the user operates the smart user assistive device 100 to determine which device operation mode should be operated, such as the hearing aid mode, the entertainment mode, and the hands-free mode.

In addition, the operation mode setting unit 150 may set a location option of the smart accessory device 100 in response to a user operation. That is, the operation mode setting unit 150 can set a place option corresponding to a user operation such as a menu selection of a UI, for example, at a certain place (for example, a cafe, a conference room, So that the stereo DSP unit 130 can refer to it because the frequency characteristics of noise are different.

The directional stereo microphone set 160 is disposed in correspondence with the left and right ears of the user in the smart user assistive device 100 to collect ambient audio as a stereotype. Directional stereo microphone set, and includes a left microphone corresponding to the left ear of the user and a right microphone corresponding to the right ear of the user. Preferably, the microphone is provided in four directions so that the user can perceive the surroundings in three dimensions corresponding to the front and rear of the left ear and the front and back of the right ear.

The communication microphone 170 is a microphone module for acquiring a voice signal of a user for making a call when performing a handsfree mode in which a phone call is established by wireless connection with an external smart terminal 200 through a Bluetooth communication unit 180. [

The Bluetooth communication unit 180 wirelessly connects the smart user assist device 100 with the external smart terminal 200. [ In particular, the Bluetooth communication unit 180 wirelessly receives stereophonic entertainment audio from the external smart terminal 200 in an entertainment mode, and outputs stereo signals to BT left audio and BT right audio. Also, in the hands-free mode, the user's voice signal provided from the communication microphone 170 is wirelessly transmitted to the smart terminal 200, while the other party's voice signal (generally monaural sound) is wirelessly received from the smart terminal 200 BT left audio and BT right audio.

2 is a block diagram illustrating a functional configuration of a smart terminal 200 in which a smart application for operating in cooperation with the smart user assistant device 100 is implemented in the present invention. In the present invention, the smart terminal 200 is provided with a dedicated application for managing the smart user assistive device 100, in particular, a fitting management program (application) for generating fitting data for the user. When such an application is executed, the smart terminal 200 has a functional configuration as shown in FIG.

The smart terminal 200 according to the present invention includes a terminal control unit 210, an ear modeling unit 220, a stereo fitting unit 230, a fitting data DB unit 240, a fitting data reference unit 250, An entertainment reproduction unit 270, a voice call control unit 280, and a Bluetooth communication unit 290. The Bluetooth communication unit 290, The individual functional elements constituting the smart terminal 200 will now be described.

First, the terminal control unit 210 controls the functions of the smart terminal 200 as a whole.

The modeling unit 220 then provides a test sound for each of a plurality of frequency bands divided from the audible frequency band for each of the user's ears and analyzes corresponding user response inputs, . For example, the audio frequency band is divided into 16 channels, and a test sound is provided to the user through a speaker or a headphone of the smart terminal 200 for each channel, and a response input of the user is provided to, for example, a UI screen. The modeling unit 220 then analyzes the response input to generate auditory characteristic models for each of the ears according to frequency bands.

The stereo fitting unit 230 generates a standard amplification profile for each frequency band for compensating the auditory characteristic model for each of the user's ears. That is, a standard amplification profile is firstly generated as an amplification pattern for compensating for the degree of deterioration of auditory ability in each frequency band.

Then, preferably, the stereo fitting unit 230 generates fitting data for each operation mode from the standard amplification profile, reflecting the frequency characteristics of the target sound according to the device operation mode. The frequency characteristics of the target sound differ depending on the hearing aid mode, the entertainment mode, and the hands-free mode. In the hands-free mode, the target sound is set at about 8000 Hz or less where the human voice is concentrated. In the entertainment mode, it is preferable to set the target sound to a high frequency band, for example, 20000 Hz. In hearing mode, you can set the target sound at about 10000 Hz or less. In addition, it is desirable to further generate fitting data for each place from the standard amplification profile reflecting the frequency characteristics of the noise according to the place.

The fitting data DB unit 240 manages fitting data provided for each user, device operation mode, and place.

The fitting data reference unit 250 stores the reference fitting data prepared by a manufacturer in advance based on a person who is ordinary to each device operation mode and place. A user who has not particularly deteriorated the hearing can obtain satisfactory satisfaction even if the reference fitting data is utilized without having to generate the fitting data optimum to the user.

The fitting data transmission unit 260 transmits the device operation mode and fitting data generated by the stereo fitting unit 230 to the smart user auxiliary device 100 via the Bluetooth communication and stores the fitting data in the fitting data storage unit 140 do.

The entertainment reproducing unit 270 reproduces multimedia contents such as games and music, and is for executing the entertainment mode.

The voice call control unit 280 is a module for implementing a voice call as in a smart phone, for example, to execute a hands-free mode.

The Bluetooth communication unit 290 wirelessly connects the smart terminal 200 with the smart user assistive device 100.

3 is a diagram illustrating an internal connection configuration of an audio signal in the smart user auxiliary apparatus 100 according to the present invention in its entirety.

First, the directional stereo microphone set 160 is disposed in correspondence with the left and right ears of the user to collect ambient audio as a stereotype. The left microphone is composed of an L1 microphone 161 and an L2 microphone 162 corresponding to two directions (e.g., front and rear) of the left ear to generate an L1 hearing aid and an L2 hearing aid . The light microphone is composed of an R1 microphone 163 and an R2 microphone 164 corresponding to two directions (e.g., front and rear) of the right ear to generate an R1? Hearing aid and an R2? Hearing aid .

The call microphone 170 is a microphone module for acquiring a voice signal of a user for making a telephone call in the hands-free mode. 3, the communication microphone 170 is composed of an L3 microphone 171 and an R3 microphone 173 as stereotypes to generate an L3 call signal and an R3 phone call But the present invention is not limited thereto.

The Bluetooth communication unit 180 receives stereophonic entertainment audio from the smart terminal 200 in the entertainment mode and wirelessly receives the other party's voice signal from the smart terminal 200 in the hands-free mode to generate BT left audio BT audio L and BT right L And outputs it as audio (BT audio R). Also, in the hands-free mode, the voice signal (L3? Phone call, R3? Phone call) of the user provided from the communication microphone 170 is wirelessly transmitted to the smart terminal 200.

The stereo DSP units 131 and 132 receive the hearing aid signals L1, L2, R1 and R2 of the directional stereo microphone sets 161 to 164 and the Bluetooth audio signals BT audio L and BT audio R, Amplifies them according to the fitting data, and outputs them to the stereo output units 121 and 122. The stereo DSP unit 130 includes a left DSP unit 131 and a write DSP unit 132 for stereo signal processing.

First, the left DSP unit 131 receives both the left microphone audio signal L1 and the left BT audio sound L in the entertainment mode and the audio signals L1 and L2 And amplifies the received audio signal according to the device operation mode according to the left fitting data of the fitting data to output the left audio signal Audio L. [ The left DSP unit 131 always receives the audio signal of the L1 microphone 161 through the first input port and transmits the audio signal of the L2 microphone 162 and the BT left audio BT audio L) according to the device operation mode.

Similarly, in the entertainment mode, the write DSP unit 132 receives both the audio signal R1 of the write microphone 163 and the BT right audio BT audio R, and in the hearing mode, the audio signal of the write microphones 163 and 164 R1 and R2, and outputs the right audio signal Audio R by amplifying the received audio signal according to the device operation mode according to the right fitting data of the fitting data. The write DSP unit 132 always receives the audio signal R1 of the R1 microphone 163 through the first input port and the audio signal R2 of the R2 microphone 164 via the second input port and the audio signal R2 of the Bluetooth communication unit 180 ) Of the BT right audio (BT audio R) according to the device operation mode.

In the hands-free mode, the BT left / right audio (BT audio L, BT audio R) output from the Bluetooth communication unit 180 is the voice signal of the other party. The left DSP unit 131 and the write DSP unit 132 receive both the audio signals L1 and R1 of the L1 / R1 microphones 161 and 163 and the voice signal of the other party via the Bluetooth communication unit 180, The right fitting data is referenced and amplified.

The left output unit 121 and the right output unit 122 respectively output the audio signals Audio L and Audio R generated by the left DSP unit 131 and the write DSP unit 132 in a stereo type.

FIGS. 4 to 6 are diagrams illustrating audio signal connections of the smart user assist device 100 of the present invention in a hearing mode, an entertainment mode, and a hands-free mode. 4 illustrates an audio signal connection when the smart user assist device 100 of the present invention operates as a hearing aid device and FIG. 5 illustrates a case where the smart user assist device 100 of the present invention is connected to an entertainment assist device 6 shows an audio signal connection when the smart user auxiliary device 100 of the present invention operates as a hands-free device.

4 is a diagram showing an audio signal connection when the smart user auxiliary apparatus 100 of the present invention operates as a hearing aid (hearing mode). When operating in the hearing aid mode, the L1-hearing aid, L2-hearing aid, R1-hearing aid, and R2-assist signal generated by the directional stereo microphone set 160 R 2? Hearing aid to the left DSP unit 131 and the write DSP unit 132, performs amplification processing according to the fitting data, and outputs the stereo sound to the user.

5 is a diagram showing audio signal connection when the smart user auxiliary apparatus 100 of the present invention operates as an entertainment assistant device (entertainment mode). When operating in the entertainment mode, high quality stereo audio such as games and music generated by the smart terminal 200 is transmitted to the BT left audio (BT audio L) and the BT right audio (BT audio R) through the Bluetooth communication unit 180 .

The left DSP unit 131 receives both the audio signal L1 of the L1 microphone 161 and the BT audio L of the Bluetooth communication unit 180 and amplifies it according to the fitting data (left fitting data) And outputs it as an audio signal (Audio L). The write DSP unit 132 receives both the audio signal R1 of the R1 microphone 163 and the BT right audio BT of the Bluetooth communication unit 180 and amplifies it according to the fitting data (right fitting data) And outputs it as a rear right audio signal (Audio R). This allows users to enjoy high-quality entertainment sound while not missing the surrounding situation.

Meanwhile, the smart user auxiliary device 100 of the present invention is preferably implemented as a neckband type suitable for providing stereo sound. The smart user auxiliary device 100 may include a left output unit 121 and a right output unit 122 Stereo sound is closely communicated to the user. 5, the left DSP unit 131 and the write DSP unit 132 may separately generate fitting data (left fittings, left-hand fittings, etc.) corresponding to the auditory characteristics of both ears, Data, right fitting data) to minimize the effect of hearing loss.

6 is a diagram showing an audio signal connection when the smart user auxiliary apparatus 100 of the present invention operates as a hands-free device (hands-free mode). In the hands-free mode, the BT left / right audio (BT audio L, BT audio R) output from the Bluetooth communication unit 180 is the voice signal of the other party. The left DSP unit 131 and the write DSP unit 132 receive both the audio signals L1 and R1 of the L1 / R1 microphones 161 and 163 and the voice signal of the other party via the Bluetooth communication unit 180, By referring to the fitting data on the right side and amplifying it, the user can recognize the note from the voice of the other party.

In addition, the L3 microphone 171 and the R3 microphone 173 acquire a voice signal of a user when performing a telephone conversation as a communication microphone 170. [ The voice signals L3 and R3 of the user are wirelessly transmitted to the smart terminal 200 through the Bluetooth communication unit 180. [

The present invention can also be embodied in the form of computer readable code on a computer readable recording medium. At this time, the computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage, and the like, and may be implemented in the form of a carrier wave . The computer-readable recording medium can also be stored and executed by a computer-readable code in a distributed manner on a networked computer system. And functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

As described above, the embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms have been used, they have been used only in a general sense to easily describe the technical contents of the present invention and to facilitate understanding of the invention. And is not intended to limit the scope of the invention. It will be apparent to those skilled in the art that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein.

100: Smart User Assistant
110:
120: Stereo output section
121, 122: Left, right output section
130: a stereo signal processing unit (DSP unit)
131, 132: left, right DSP unit
140: Fitting data storage unit
150: Operation mode setting section
160: Directional stereo microphone set
161, 162: Left microphones (L1, L2)
163, 164: Light microphones (R1, R2)
170: Microphone for call
171, 172: L3 for communication, R3 microphone
180: Bluetooth communication unit
200: smart terminal
210:
220:
230: Stereo fitting part
240: Fitting data database part (DB part)
250: Fitting data reference part
260: Fitting data transmission unit
270: Entertainment playback section
280: voice call control unit
290: Bluetooth communication section

Claims (6)

A smart user assistance device that interlocks with an external smart terminal via Bluetooth and processes stereo audio according to a device operation mode including an entertainment mode and a hearing mode to provide the signal to a user,
A Bluetooth communication unit 180 for wirelessly receiving stereophonic entertainment audio from an external smart terminal;
A stereo microphone set 160 arranged in correspondence with the left and right ears of the user in the smart user auxiliary device to collect surround audio in a stereotype;
A fitting data storage unit 140 for storing fitting data having amplification profiles for respective right and left ears according to frequency bands for each device operation mode of the smart user auxiliary device;
A stereo DSP unit 130 for non-uniformly amplifying the entertainment audio provided from the Bluetooth communication unit and the peripheral audio provided from the stereo microphone set by frequency band according to fitting data for each device operation mode stored in the fitting data storage unit;
A stereo output unit 120 for stereo outputting an audio signal resulting from amplification of the stereo DSP to a user;
A device control unit (110) for generally controlling the function of the smart user auxiliary device in response to the device operation mode of the smart user auxiliary device;
And,
The stereo microphone set 160 includes a left microphone corresponding to a user's left ear and a right microphone corresponding to a user's right ear as a directional stereo microphone set,
The Bluetooth communication unit 180 wirelessly receives entertainment audio from an external smart terminal and outputs a stereo signal to BT left audio and BT right audio,
The stereo DSP unit 130 receives both the audio signal of the left microphone and the left audio of the BT in the entertainment mode and only the audio signal of the left microphone in the hearing mode and outputs the received audio signal A left DSP unit 131 for amplifying audio data corresponding to left fitting data among the fitting data, an audio signal of the right microphone and a right audio of BT in an entertainment mode, and receives only an audio signal of the right microphone in a hearing mode And a write DSP unit (132) for amplifying the received audio signal according to a device operation mode in accordance with right fitting data among the fitting data,
The stereo output unit 120 includes a left output unit 121 and a right output unit 122 for outputting sound in a stereotyped manner to each of the audio signals generated by the left DSP unit 131 and the right DSP unit 132, The smart user assist device using Bluetooth.
delete The method according to claim 1,
The left microphone includes L1 microphones 161 and L2 microphones 162 corresponding to two directions of the left ear,
The light microphones are configured to include R1 microphones 163 and R2 microphones 164 respectively corresponding to two directions of the right ear,
The left DSP unit 131 always receives the audio signal of the L1 microphone through the first input port and transmits either the audio signal of the L2 microphone and the BT left audio of the Bluetooth communication unit through the second input port to the device operation Selectively receive according to the mode,
The write DSP unit 132 receives the audio signal of the R1 microphone through the first input port and transmits the audio signal of the R2 microphone and the BT right audio of the Bluetooth communication unit through the second input port to the device operation Mode, and is configured to selectively receive data according to the mode.
The method of claim 3,
Ear modeling means for generating a perceptually perceptible auditory feature model for each of the ears by providing a test sound for each of a plurality of frequency bands divided from the audible frequency band for each of the user's ears and analyzing the corresponding user response input And generating a standard amplification profile for each frequency band based on the frequency characteristics of the target sound according to the operation mode of the device, (260) for providing fitting data for each of the operating modes to the fitting data storage unit via Bluetooth communication, and a fitting management program media;
An operation mode setting unit 150 for setting a device operation mode of the smart user auxiliary device in response to a user operation;
Further comprising:
The stereo DSP unit 130 obtains fitting data corresponding to the device operation mode set by the operation mode setting unit from the fitting data storage unit and outputs an audio signal input to the stereo DSP unit based on the fitting data And the non-uniform amplification is performed for each frequency band.
The method of claim 4,
The stereo fitting means 230 further generates fitting data for each place from the standard amplification profile by reflecting the frequency characteristic of noise according to the place,
The fitting data transmission unit 260 further provides the fitting data by Bluetooth communication to the fitting data storage unit,
The operation mode setting unit 150 further sets a location option of the smart user auxiliary device in response to a user operation,
The stereo DSP unit 130 obtains fitting data corresponding to a place option set by the operation mode setting unit from the fitting data storage unit and outputs an audio signal inputted to the stereo DSP unit on the basis of the frequency data And the non-uniform amplification is performed for each band.
The method of claim 5,
A communication microphone (170) for acquiring a voice signal of a user for a call in a hands-free mode for making a telephone conversation by wireless connection with an external smart terminal through the Bluetooth communication unit;
Further comprising:
The Bluetooth communication unit 180 wirelessly transmits a voice signal of a user provided from the communication microphone to the smart terminal and outputs a counterpart voice signal wirelessly received from the smart terminal to the BT left audio and BT right audio,
The left DSP unit 131 receives both the audio signal of the L1 microphone and the BT left audio of the Bluetooth communication unit and obtains the fitting data corresponding to the handsfree mode from the fitting data storage unit, The audio signal of the L1 microphone and the left audio of the BT are nonuniformly amplified for each frequency band,
The write DSP unit 132 receives both the audio signal of the R1 microphone and the BT right audio of the Bluetooth communication unit and acquires fitting data corresponding to the hands-free mode from the fitting data storage unit, and based on the obtained fitting data Wherein the audio signal of the R1 microphone and the right audio of the BT are non-uniformly amplified by frequency bands.

Images