KR101250929B1 - A two ear-piece insertable hearing aid having a vibration pattern alert function - Google Patents

Abstract

PURPOSE: A binaural insertion type hearing aid is provided to allow hearing impaired people to previously cope with a dangerous factor by recognizing an external dangerous factor in the left, right and rear sides through a camera and providing information relating to the external dangerous factor to hearing impaired people. CONSTITUTION: A pair of hearing aid modules process detected sound from outside and output into acoustic. A curved frame(120) connects a pair of the hearing aid modules. A pair of camera modules(130a,130b) are arranged in the outer side of the pair of hearing aid modules and acquire a first external video from the side. An auxiliary camera module is arranged in the center of the curved frame and acquires a second external video. A biometric data measurement module(150) is arranged in the inner side of a kernel inserted into an external auditory canal of the pair of hearing aid modules and acquires biometric data of a wearer. A control module(160) receives the biometric data and collects a biometric state of the wearer.

Description

A TWO EAR-PIECE INSERTABLE HEARING AID HAVING A VIBRATION PATTERN ALERT FUNCTION}

The present invention relates to a double-ear insert hearing aid having a vibration pattern notification function, and more particularly, to a double-ear insert hearing aid having a vibration pattern notification function capable of detecting a danger to the external environment and a risk inside the wearer.

In general, an impaired hearing patient refers to a person who does not hear sound well enough to understand and talk in everyday life, that is, a person with hearing impairment, and has a hearing loss of 27 dB to 40 dB depending on the degree of hearing loss. Loss is classified as mild hearing loss, 41 dB to 55 dB hearing loss is moderate hearing loss, 56 dB to 70 dB hearing loss is moderate hearing loss and 70 dB to 90 dB hearing loss is classified as high hearing loss, hearing loss above 91 dB It is classified as a sea bass.

Hearing aids that can compensate for sound or electrical energy in hearing-impaired people who have deteriorated or lost hearing due to inability to transmit sound or electrical energy from the human ear to the auditory nerve It is essential.

Generally, a hearing aid includes a transmitter and a receiver for collecting a speaker sound and a surrounding sound and outputting it as an electric signal, an amplifier for receiving and amplifying a signal output from the transmitter, an amplifier for amplifying the amplified signal to a sound wave, And a battery for supplying electric power to the receiver and the amplifier. Types of hearing aids include box-type, ear-shaped, earring-type, and eyeglass-type. Recently, the ear-type has been further developed into a tympanic membrane that can be inserted deeply into the ear.

Korean Patent Publication No. 10-2009-0008612

On the other hand, a user wearing a hearing aid can compensate for some of his lost hearing by artificially reproducing the lost sound by an electric method, but it is difficult to recover to a normal hearing level. In particular, in the case of highly deaf or deaf, it is difficult to detect sounds from non-adjacent sound sources.

In addition to visual information, auditory information plays an important role in recognizing the situation. Especially, when a dangerous situation occurs in an area where the field of vision is not secured, the auditory elements occurring in that area are critical to recognize the danger situation and evacuate in advance. Can play a role.

For example, if a vehicle rushes suddenly while walking a patient with high hearing loss, it is difficult to recognize it in advance, and serious injury may occur if the noise generated from the rushing vehicle is not properly recognized. In particular, when the vehicle rushes from the left or right side or the rear of the hearing impaired person, the vision may not be secured, and thus may be in great danger.

As described above, the hearing impaired person may be frequently in a difficult or dangerous situation in acquiring and responding to auditory information about the external environment compared to the general public, and therefore, it is urgent to respond to the problem.

The present invention is to solve the above problems, the technical problem to be achieved of the present invention is to recognize the risk factor in advance and provide the information about this to the hearing impaired hearing aids so that the hearing loss can respond in advance and the risk using the same It is to provide an element detection system.

Another problem to be solved by the present invention is to provide a hearing aid and a risk detection system using the same, which detects an external danger in real time but delivers an appropriate stimulus to the hearing loss in consideration of the internal danger.

Another problem to be solved by the present invention, in consideration of the internal risk factors of the wearer, if a dangerous situation is found, hearing aids and risk factors using the same to propagate the dangerous situation to others outside the wearer is detected To provide a system.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, a double-ear insert hearing aid having a risk factor detection function according to an embodiment of the present invention, a neckband type of a double-ear insert hearing aid, a pair for outputting a sound signal processing the sound sensed from the outside A hearing aid module, a curved frame connecting the pair of hearing aid modules, a pair of camera modules respectively disposed on an outer surface of the pair of hearing aid modules to obtain a first external image of the side, and the curved frame of the An auxiliary camera module disposed at a central portion to acquire a second external image at a rear side, a biometric data measurement module arranged at an inner side of a kernel inserted into the ear canal of the pair of hearing aid modules, and obtaining biometric data of a wearer; and the biometric data Receiving a control module for collecting the biological state of the wearer, the control module periodically And transmits the living body data to an external management server, and the control module receives the first external image and the second external image and analyzes them to detect external risk factors in the unviewed area when the wearer walks. The control module transmits a notification sound to the pair of hearing aid modules when the external hazard is detected.

A double-ear insert hearing aid having a risk factor detection function according to another embodiment of the present invention is a neckband-type double-ear insert hearing aid, a pair of hearing aid modules for outputting a sound by processing the sound sensed from the outside, a pair of A curved frame connecting the hearing aid module, a pair of camera modules respectively disposed on the outer surfaces of the pair of hearing aid modules to acquire a first external image on the side, and a second external image disposed at the center of the curved frame A secondary camera module for acquiring the biometric data measurement module disposed inside the kernel inserted into the ear canal of the pair of hearing aid modules to acquire the biometric data of the wearer, and provided in the pair of hearing aid modules according to a predetermined vibration control signal; It includes a vibration motor for forming a vibration pattern, and a control module for receiving the biometric data to collect the biological state of the wearer, The control module periodically transmits the biometric data to an external management server, the control module receives the first external image and the second external image and analyzes it to detect external risk factors in the unviewed area of the wearer, and the control module When an external hazard is detected, a vibration pattern is transmitted to the vibration motor to generate a vibration pattern.

A risk factor detection system using a hearing aid according to an embodiment of the present invention, including a communication module, a hearing aid device capable of wirelessly communicating with the outside, an intermediate server for wirelessly communicating with the hearing aid device to transmit and receive predetermined data, and the And a plurality of distributed guidance servers connected to a mediation server, wherein the hearing aid device includes a camera module for acquiring an external image, and the hearing aid device transmits the image data obtained from the camera module to the mediation server. The server transmits the image data to one of the plurality of distributed guide servers, and one of the plurality of distributed guide servers analyzes the image data to detect an external risk factor of the unviewed area of the wearer wearing the hearing aid device. When the external hazard is detected, the first hearing aid device is used. It sends the treble.

Specific details of other embodiments are included in the detailed description and the drawings.

According to the two-ear inserted hearing aid having a risk factor detection function of the present invention as described above and a risk factor detection system using the same, there are one or more of the following effects.

That is, by installing a camera module on the outside of the hearing aid, and through the camera module in advance to recognize the external risk factors of the wearer's left and right sides and to provide the information to the hearing impaired person, the hearing impaired person can respond in advance. It can prevent human injury that may occur.

In addition, the detection of external risk factors in real time, but considering the internal risk factors derived by analyzing the biometric data, if the breathing of the hearing impaired, body temperature or pulse, such as unstable state, the possibility of noticing the alarm sound notification Considering this, it is possible to amplify the pattern period and size of the beeping sound in consideration of this, and to deliver an appropriate stimulus to the hearing impaired person.

In addition, considering the internal risk factors of the wearer obtained by analyzing the biometric data, if a dangerous situation is found, the external speaker and / or the LED module may be activated to spread the dangerous situation to others so that the wearer may be promptly taken. Can be.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description of the claims.

1 is a view showing a state wearing a double-ear insert hearing aid having a risk factor detection function according to an embodiment of the present invention.
FIG. 2 is a view showing a state shown from the opposite side of FIG. 1.
Figure 3 is a perspective view showing the structure of both ears inserted hearing aid having a risk factor detection function according to an embodiment of the present invention.
4 is a diagram illustrating an image acquisition structure of left, right, and rear cameras of the hearing aid of FIG. 3.
FIG. 5 is an exemplary diagram illustrating an external risk factor detected from the image data of FIG. 4.
6 is a block diagram schematically illustrating a principle of operation of the hearing aid of FIG. 3.
7 to 9 are views showing the operating principle when the external risk factors are detected in the images of the left and right and rear cameras of the hearing aid of FIG. 3, respectively.
FIG. 10 is a flowchart illustrating a notification sound pattern when an external hazard is detected in images of the left and right and rear cameras of the hearing aid of FIG. 3.
FIG. 11 is an exemplary diagram illustrating a pattern period and a size change of the biometric data of the hearing aid of FIG. 3 and the corresponding notification sound.
FIG. 12 is a flowchart sequentially showing the operating principle of the hearing aid of FIG. 3.
FIG. 13 is a flowchart sequentially illustrating a principle of determining an external risk factor of the hearing aid of FIG. 3.
14 is a block diagram schematically illustrating a risk factor detection system using a hearing aid according to an exemplary embodiment of the present invention.
FIG. 15 is a flowchart sequentially showing the operating principle of the system of FIG. 14.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. &Quot; and / or "include each and every combination of one or more of the mentioned items. ≪ RTI ID = 0.0 >

Hereinafter, with reference to FIGS. 1 to 3, a double-ear inserted hearing aid having a risk detecting function according to an embodiment of the present invention will be described. 1 is a view showing a state wearing a double-ear insert hearing aid having a risk factor detection function according to an embodiment of the present invention, Figure 2 is a view showing the appearance shown on the opposite side of Figure 1, Figure 3 is the present invention Perspective view showing the structure of the ear-insertable hearing aid having a risk factor detection function according to an embodiment of.

1 to 3, a double-ear insert hearing aid having a risk factor detection function according to an embodiment of the present invention is a neckband type double-ear insert hearing aid, which outputs a sound by processing a signal sensed from the outside A pair of hearing aid modules 110, a curved frame 120 connecting the pair of hearing aid modules 110, the first external image of the side is disposed on the outer surface of the pair of hearing aid modules 110, respectively A pair of camera modules 130a and 130b for acquiring a second camera module 140 and a second hearing aid module 110 for acquiring a second external image at a rear side of the curved frame 120. A biometric data measurement module 150 disposed inside the kernel inserted into the ear canal of the ear canal to acquire biometric data of the wearer, and a control module 160 receiving the biometric data to collect the biometric state of the wearer. The control module 160 periodically transmits the biometric data to an external management server, and the control module 160 receives the first external image and the second external image and analyzes the same. When the user detects an external hazard in the unviewed area when walking, the control module 160 transmits a notification sound to the pair of hearing aid modules 110 when the external hazard is detected.

1 and 2, a state in which both ears inserted hearing aids 100 usable to both ears of a wearer is worn is shown. The pair of hearing aid modules 110 are connected by a curved frame 120, the curved frame 120 is a streamlined shape according to the shape of the laryngeal head, for example, in the form of a wrap around the head of the wearer, for example. Can be determined. One side of the hearing aid module 110 may be further provided with an annular mounting portion (not shown) that can be mounted to the rear of the outer ear, but is not limited thereto. The hearing aid module 110 may be in the form of a kernel that is inserted into the ear canal without a mounting portion at a predetermined negative pressure and provides a fixable force.

The material and shape of the curved frame 120 is not limited, and in some embodiments, the curved frame 120 may be replaced with a sheath coated connection line. In this case, the hearing aid module 110 may be synchronized with the transmission and reception of the predetermined data by the coating coated connection line.

As described above, the pair of hearing aid modules 110 is a configuration of an existing hearing aid, for example, a speaker and a surrounding sound to collect and output as an electrical signal, the rectifier and amplifying the signal output from the transmitter An amplifier, a handset for converting the signal amplified by the amplifier into sound waves and delivered to the ear of the hearing impaired, may include a battery for supplying power to the handset and the amplifier.

For example, the talker plays a role of extracting a voice generated in a predetermined area, and various kinds of microphone devices for receiving voice data may be used. For example, depending on the microphone structure, a dynamic microphone, a condenser microphone, and a ribbon microphone may be used, and depending on the directivity, a directional microphone, a non-directional microphone, a uni-directional microphone, and a superdirectional microphone (Super cardioid, Hyper cardioid) and the like can all be used.

The handset 112b may have the same configuration as a known speaker or earphone, and delivers voice data to the wearer.

On the outer surface of the hearing aid module 110, a camera module 130 for acquiring an external image in a direction perpendicular to the plane of the hearing aid module 110 is provided. The camera module 130 is provided on the outer surface of the hearing aid module 110, and acquires an image of the outside of the left or right side of the wearer and converts it to image data.

The camera module 130 may include a camera unit for photographing an actual image, an encoder unit connected to the camera unit to generate analog or digital image data, and a transmission / reception unit for transmitting image data to the control module 160. .

The camera module 130 may include all of the known structures, for example, a photographing apparatus composed of one or more optical lenses, and includes all of the known camera units composed of semiconductor devices, for example, imaging devices such as CCDs. Can be.

The external image acquired from the camera module 130 is transmitted to the control module 160, and by analyzing the corresponding image, it is possible to determine whether an external hazard exists and whether the external hazard approaches the wearer.

In addition, an auxiliary camera module 140 for acquiring an image of the rear of the wearer is added to the rear of the curved frame 120 that connects the pair of hearing aid modules 110, that is, the center of the curved frame 120. It is provided with.

In some other embodiments, the rear camera module 140 may be omitted.

The auxiliary camera module 140 may acquire an image of the rear, which is a blind spot of the wearer's view, and the rear image may also be transmitted to the control module 160 to determine whether an external hazard exists in the rear image. .

In the example shown in FIGS. 1 and 2, the size of the pair of hearing aid modules 110 is exaggerated for convenience of description, but the present invention is not limited thereto, and the hearing aid is inserted into the ear canal of the wearer. It may be downsized.

Referring to FIG. 3, the hearing aid 100 according to the present embodiment may further include a biometric data measuring module 150, a control module 160, an LED module 170, and a speaker module 180, which will be described later. As shown in FIG. 13, a communication module for wireless communication with an external server may be further included.

The biometric data measuring module 150 is disposed inside the kernel inserted into the ear canal of the pair of hearing aid modules 110 to acquire biometric data of the wearer. The biometric data measurement module 150 is inserted into the ear canal of the wearer and positioned inside the body. The biometric data measuring module 150 is illustrated as being provided only on one side 110b of the pair of hearing aid modules 110, but is not limited thereto. The pair of hearing aids may measure biometric data at both ears of the wearer. It may be provided on both sides of the module (110).

The biometric data measured from the biometric data measurement module 150 may be transmitted to the control module 160 to monitor a wearer's condition. Biometric data may include heart rate, pulse rate and body temperature.

To this end, the biometric data measurement module 150 may include a body temperature detector and a pulse detector (not shown). The body temperature detector can measure the body temperature by attaching a temperature sensor to the surface of the kernel inserted into the ear canal, and can measure the amount of change in the body temperature continuously. The pulse detection unit measures a change in the pulse rate of the wearer, and attaches a pressure sensor or an optical transmission / reception sensor to detect a pulse by measuring a pressure change in the ear canal, which is transmitted from the blood, or a transmitter or light that generates light. The presence or absence of a pulse can be detected by mounting a receiver to a hearing aid and measuring a change in the intensity of the detected light.

From the biometric data measured as described above, it is possible to determine whether the wearer is in a normal state or an abnormal state. For example, when the pulse and / or body temperature changes rapidly, the wearer's condition may be determined as an abnormal state. Such biometric data may be transmitted to an external management server (not shown) to manage it externally.

The control model 160 receives image data regarding the external image from the camera module 130 and the auxiliary camera module 140, and simultaneously receives the biometric data from the biometric data measurement module 150, and the image data and the biometric data. This can be monitored and analyzed to determine anomalies or risks.

The control module 160 may periodically transmit the biometric data to an external management server (not shown), and may display the first external image and the second external image acquired from the camera module 130 and the auxiliary camera module 140, respectively. It can be received and analyzed to detect external hazards in the wearer's field of view. In addition, the control module 160 may transmit a notification sound to the pair of hearing aid modules 110 when an external risk factor is detected.

The image analysis process and the bio data analysis process in the control module 160 will be described later.

The LED module 170 may be attached to the outside of the curved frame 120 to emit light by the signal of the control module 160. In some other embodiments, the LED module 170 may be replaced with another light source, for example a bulb type light source or another type of optical element.

When the control module 160 analyzes the biometric data of the wearer and determines that the dangerous state, the control module 160 may emit the LED module 170 so as to propagate the wearer's condition to others outside the wearer. The brightness may be controlled by varying the power supply according to the amount of ambient light.

The LED module 170 may repeat the flashing operation to propagate the emergency situation.

The speaker module 180 may be attached to the outside of the curved frame 120 to transmit a predetermined sound to the outside due to the control of the control module 160. Like the LED module 170, when the control module 160 analyzes the wearer's biometric data and determines that the wearer is in a dangerous state, the speaker module 180 may transmit a predetermined decibel or more through the speaker module 180 to propagate the wearer's condition to others outside the wearer. It can generate a warning sound to draw attention to others located nearby.

The communication module 190 may be a mobile communication module capable of wirelessly communicating with an external server or a predetermined device, and may transmit / receive predetermined data with the external. For example, the image data and / or biometric data obtained from the hearing aid 100 may be transmitted to an external server, and a voice message may be received from the outside and transmitted to the wearer. In some other embodiments, the communication module 190 may receive a control signal from an external server, and transmit it to the control module 160 to receive the control signal through the external control unit 112b of the hearing aid module 110. You can also control the volume of the remote control.

4 to 10, the operation principle and structure of a double-ear inserted hearing aid having a risk factor detection function according to an embodiment of the present invention will be described in detail. FIG. 4 is a view showing image acquisition structures of the left and right and rear cameras of the hearing aid of FIG. 3, FIG. 5 is an exemplary view showing an external risk factor detected from the image data of FIG. 4, and FIG. 6 is a view of the hearing aid of FIG. 3. 7 is a block diagram schematically illustrating an operation principle, and FIGS. 7 to 9 are views illustrating an operation principle when an external hazard is detected in the images of the left and right and rear cameras of the hearing aid of FIG. 3, and FIG. A flowchart illustrating a sound pattern when an external hazard is detected in the left and right images of the hearing aid and the rear camera, respectively.

Referring to FIG. 4, exemplary left, right, and rear image data are shown. The left and right image data 210 and 220 of the wearer may be acquired through the camera modules 130a and 130b, and the rear image data 230 of the wearer may be obtained through the auxiliary camera module 140.

The obtained image data 210, 220, 230 may have a predetermined frame, and may be temporarily or stored for a predetermined amount by a predetermined storage space or buffer. For example, the hearing aid according to the present embodiment may further include a storage device of a flash memory type (not shown), and image data obtained by a camera module or the like may be stored in the storage device.

The control unit 160 may extract metadata of the image data 210, 220, and 230 stored in the storage device and separately manage the metadata. The extracting of the metadata may be performed simultaneously with or at the same time when the image data 210, 220, 230 is stored in the storage device. The metadata may be an example of important information of the image data 210, 220, 230. For example, it may include one or more of size information, pixel coordinate information, and color information.

The control unit 160 may index and store information about pixels of the image data 210, 220, or 230 so that the image data 210, 220, 230 may be quickly searched.

 The stored image data 210, 220, and 230 may be stored for a predetermined period and later extracted and inquired. When the capacity is exhausted before the predetermined period, the stored image data 210, 220, and 230 may be automatically deleted from previous data in a first-in first-out manner. have.

Referring to FIG. 5, there is shown a case in which an external risk factor 300, for example, a vehicle rushing to a wearer exists in the image data 220 on the right side. As illustrated, the external hazard 300 may mean any object element that may pose a threat to the wearer's body, for example, a vehicle, a bicycle, a motorcycle or a person approaching the wearer at a predetermined speed or more. And the like.

Image data acquired by the camera module 130 and 140 are analyzed by the control module 160, and the control module 160 may be a variety of known object recognition algorithms, for example, an adaptive Gaussian Mixture Model. Based on the external risk factor 300 can be detected. In addition, a method of detecting an object under a controlled background image using a solid background image of a plane or using a predefined fixed background image, a simple extraction of moving object regions from image data, and a neural network using statistical cluster information An access technique may be used, but is not limited thereto.

For example, a background and foreground image are extracted from image data 210, 220, and 230 by applying a method of separating a moving object from a background through a background separation technique. The background can then be modeled to create an explicit background, and a binary foreground split mask can be used to separate the moving objects from the background. The moving object is extracted from the foreground image and the center point of the object is determined to determine the direction information. The direction can extract only the information of up, down, left and right, and recognizes the dangerous situation by judging the behavior information and the preset profiling information of the object.

In addition, the external risk factor 300 may be determined as an object approaching the wearer side more than a predetermined size and more than a predetermined speed among the recognized objects of the first external image and the second external image 210, 220, 230.

For example, the size of the object extracted from the image data 210, 220, 230 may be determined by a relative comparison with other objects, and in consideration of the magnification of the camera module or the auxiliary camera module 130, 140. Can determine the size of an object. For example, the external risk factor 300 may be determined by limiting to an object having an area of 50px or more on the image data 210, 220, and 230.

In addition, by tracking the movement path of the object for each frame to determine whether the object approaches the wearer side, if approaching to the wearer side, if the approaching the movement speed of the object approaching a predetermined speed or more to the external risk factor 300 You can decide.

For example, if an object has an area of 150px in a predetermined frame and the area occupied by pixels gradually increases as the frame elapses, it may be determined that the object approaches the wearer, and a predetermined moving speed is considered in consideration of the speed at which the area is increased. In the above case, the object may be determined as the external risk factor 300.

Referring to FIG. 6, image data obtained from the camera module 130 and / or the auxiliary camera module 140 is transmitted to the control module 160, and the biometric data measured from the biometric data measurement module 150 is also controlled. Sent to module 160. The control module 160 receiving such data determines whether an external hazard exists or an internal hazard exists according to the algorithm described above, and if there is a hazard, the control module 160 transmits the hazard to the pair of hearing aid modules 110. A notification sound may be transmitted.

7 to 9, a notification sound transmission pattern according to the position of the external risk factor 300 on the image data 210, 220, and 230 obtained from the camera module 130 and / or the auxiliary camera module 140. This is shown.

The camera modules 130 and 140 on the side where the external hazard 300 is detected in response to the position of the external hazard 300 so that the wearer can intuitively recognize in which position the external hazard 300 approaches. A notification sound or a warning sound may be output to the hearing aid module 110 in the direction.

For example, when the external risk factor 300 is detected on the right (wearer's left) image data 220 as shown in FIG. 7, the alarm sound is generated in the hearing aid module 110 on the left side of the wearer. Can be.

Similarly, when the external risk factor 300 is detected on the left (wearer's right) image data 210 as shown in FIG. 8, the hearing aid module 110 on the right side of the wearer may be set to generate a notification sound. have.

In addition, as shown in FIG. 9, when the external risk factor 300 is detected on the upper side (rear of the wearer) image data 230, both of the hearing aid module 110 of the wearer are set to generate a notification sound at the same time. Can be.

As such, when the alert sound occurs in the direction in which the external hazard 300 is detected, the wearer may intuitively switch to the direction in which the alert sound occurs or change the field of view to immediately respond to and respond to the external hazard 300. have.

In some other embodiments, if a danger is detected in two or more areas of the wearer's left, right, and rear, the alarm may be patterned to allow the wearer to recognize a hazard approaching in the plurality of orientations. .

For example, when the hazard approaches at the left and right side of the wearer at the same time, the sound is generated alternately in the left and right of the hearing aid module 110, which causes the wearer to at the same time the external hazard 300 on the left and right side. It can be recognized that is present.

Referring to FIG. 10, the notification sound for notifying the wearer, that is, the hearing impaired person, of the external hazard 300 may have a different pattern according to the position of the unviewed region where the external hazard 300 exists. As described above, in addition to the method of controlling the location of the notification sound in response to the detected direction, as shown in the figure, the location of the external hazard 300 may be notified to the wearer according to a predetermined notification sound pattern. For example, when the external risk factor 300 is detected on the left (right side of the wearer) image data 210, a notification sound may be reproduced according to the first pattern 410, and the other side image data 220, In operation 230, the notification sound may be reproduced according to the second pattern 420 and the third pattern 430, respectively. In this case, the hearing aid module 110 may be simultaneously reproduced at both sides.

In some other embodiments, pre-recorded warning phrases may be played back to intuitively recognize a dangerous direction for the hearing impaired person. For example, if an external hazard 300 is detected on the left side of the wearer, you can play a warning phrase such as "left" or "left is dangerous", and if it detects on the other side, play the corresponding warning phrase. It may be.

In addition, in some other embodiments, the hearing aid 100 may include an acceleration sensor and / or a tilt sensor (not shown), and may detect the wearer's gaze direction due to the acceleration or tilt. When the external module is detected by the control module 160, if the wearer changes the gaze direction in the direction in which the external hazard exists, for example, within a predetermined time interval of, for example, 0.1 second, As the detected state, the notification sound reproduction function may not be performed.

In addition, in some other embodiments, the hearing aid 100 further includes a vibration motor (not shown) on one side of the hearing aid module 110, and if a hazard occurs, vibration may occur to inform the wearer of an external hazard. have. As described above, by vibrating the motor in the direction in which the external hazard occurs, it can intuitively inform the wearer of the dangerous direction. That is, the vibration pattern generated from the vibration motor may be generated by the vibration motor provided in the hearing aid module in the direction of the camera module on the side where the external hazard is detected.

In particular, in the case of a user with a high hearing loss level, even when the alarm sound or the warning sound or the warning sentence is output to the maximum, there is a risk that such a danger may not be recognized when the noise level of the surrounding environment is high. Therefore, when the control module 160 recognizes an external danger element, instead of outputting sound through the hearing aid module 110, the vibration motor may be vibrated to propagate the danger by tactile sense instead of hearing.

That is, a hearing aid according to another embodiment of the present invention is a neckband-type bilateral insertion type hearing aid, which is a curve connecting a pair of hearing aid modules and a pair of hearing aid modules to signal-process and output the sound sensed from the outside. Frame, a pair of camera modules respectively disposed on the outer surface of the pair of hearing aid modules to acquire the first external image on the side, and an auxiliary camera module disposed on the center of the curved frame to acquire the second external image at the rear And a biological data measuring module disposed inside the kernel inserted into the ear canal of the pair of hearing aid modules to acquire the biometric data of the wearer, and provided on the pair of hearing aid modules to form a vibration pattern by a predetermined vibration control signal. And a control module for receiving the biometric data and collecting the biometric state of the wearer, wherein the control module periodically checks the biometric data. Is transmitted to the external management server, the control module receives the first external image and the second external image and analyzes it to detect the external risk factors of the wearer's field of view unobscured, and the control module vibrates when the external risk factors are detected. The vibration pattern is transmitted to the motor to generate a vibration pattern.

As in the previous embodiment, the biometric data includes pulse and body temperature, and the control module may determine the magnitude and repetition frequency of the vibration pattern in consideration of the biometric data upon detection of an external risk factor in the unviewed region. Under conditions where the temperature outside the normal temperature is above the first temperature or the pulse is above the first value from the normal pulse, when an external hazard is detected, the control module may increase the magnitude and repetition frequency of the vibration pattern to a predetermined range or more. have.

In addition, the hearing aid including the vibration motor according to the present embodiment as in the previous embodiment, a plurality of LED modules that are attached to the outside of the curved frame and emits light by the signal of the control module, and the outside of the curved frame It may further include a speaker module for transmitting a predetermined sound to the outside due to the control of the control module, the body temperature is more than the second temperature higher than the first temperature from the normal temperature or the pulse is higher than the first value in the normal pulse Under conditions outside the second value or more, when an external hazard is detected, the control module may transmit a notification sound to the speaker module to propagate the wearer's state to the outside and periodically blink the plurality of LED modules.

Hereinafter, with reference to FIGS. 11 to 13, a control method according to a risk detection function of a double-ear insert hearing aid having a risk detection function according to an embodiment of the present invention will be described. FIG. 11 is an exemplary diagram illustrating a pattern period and a size change of the biometric data of the hearing aid of FIG. 3 and a corresponding notification sound, FIG. 12 is a flowchart sequentially illustrating an operation principle of the hearing aid of FIG. 3, and FIG. 13 is FIG. Is a flow chart that sequentially illustrates the principles of determining the external risk factors of hearing aids.

Referring to FIG. 11, a change in the notification sound according to the biometric data is illustrated. As described above, the biometric data may include a pulse and body temperature, and when the control module 160 detects the external risk factor 300 in the unviewed region, in consideration of the biometric data, the size and repetition of the notification sound The frequency can be automatically determined differently.

For example, if the wearer's body temperature deviates from the normal body temperature by more than the first temperature or the pulse deviates from the normal pulse by the first value or more, and the external risk factor 300 is detected, the control module 160 sounds the alarm. The size and repetition frequency of can be increased to more than a predetermined range.

Referring to FIG. 11, in the upper graph, a pulse graph G1 and a temperature graph G2 versus time measured from the wearer are shown. In addition, the graph below shows the frequency and magnitude graphs G3 and G4 of the alarm sound generated when an external hazard is detected. In other words, it is assumed that the external risk factor 300 is detected in all sections of the graph and the notification sound is transmitted to the wearer.

In this wearer, the pulse and body temperature of the A section and the E section are normally maintained, and as a result, as shown in the graph below, the frequency and the magnitude of the notification sound are kept constant in the A section and the E section. That is, it is determined that the wearer's state is normal, and a notification sound is generated according to a normal pattern.

The B section and the D section are sections in which the pulse and body temperature temporarily increase, and as a result, as shown in the graph below, the frequency and the magnitude of the notification sound may increase in the corresponding section B and the section D. That is, it is determined that the wearer's state is an initial abnormal state, and in the case of the initial abnormal state, the possibility of not recognizing a notification sound indicating a danger occurrence is higher than that of the normal state, so that the notification sound is increased to improve the recognition rate. By controlling the frequency and size, the control module 160 may control this regardless of the wearer's selection.

In the case of the C section, the pulse and body temperature are greatly increased, and as a result, as shown in the graph below, the frequency and size of the notification sound may be maximized in the corresponding C section. That is, the wearer's state may be regarded as an abnormal state, and the size and frequency of the notification sound may be increased to the maximum in order to recognize the notification sound informing the occurrence of the danger.

In some other embodiments, in the case of the C section, the control of the LED module 170 and the speaker module 180 may be performed simultaneously with or separately from the notification sound control as described above. That is, under the condition that the body temperature deviates above the second temperature higher than the first temperature mentioned above from the normal temperature or the pulse deviates above the second value higher than the above-mentioned first value from the normal pulse, the external hazard 300 is detected. In this case, the control module 160 may transmit a notification sound to the speaker module 180 to propagate the wearer's state to the outside, and periodically flash the plurality of LED modules 170.

In other words, in the case of the C section, not only the notification sound is transmitted to the wearer to the maximum, but also when the external risk factor 300 is recognized according to the wearer's condition, it may not be able to respond due to unconsciousness. have.

Therefore, in preparation for such a case, in a state where the pulse and temperature values of the C section are set in advance, the LED module 170 and the speaker can be externally assisted by the wearer when the range is exceeded upward or downward. The module 180 may propagate the dangerous state to the outside.

In some other embodiments, in the case of the C section, only the configuration of propagating a danger state of the wearer to the outside may be performed, and may be controlled not to transmit a notification sound to the wearer in a state where danger recognition is difficult.

In addition, in some other embodiments, the hearing aid 100 may include an acceleration sensor and / or a tilt sensor (not shown), and may detect the wearer's gaze direction due to the acceleration or tilt. When the control server 160 detects an external hazard, the wearer's living body may be immediately responded to when the wearer switches the gaze direction toward the external hazard within a predetermined time interval of, for example, 0.1 second. Even in the case of an initial abnormal state or a dangerous state in consideration of the data, it may be determined that the wearer has already sensed a danger, and does not change the size and repetition period of the notification sound or play the notification sound. However, even in such a case, since the wearer is not in a normal state, a configuration for propagating a dangerous state to the outside through the LED module 170 and the speaker module 180 may be performed.

Referring to FIG. 12, an operation procedure of controlling the hearing aid 100 in response to a result of analyzing the biometric data according to the present embodiment is illustrated.

First, image data is collected from the camera modules 130 and 140 (S110), and biometric data is collected through the biometric data collection module 150 (S120). As described above, the collected data is transmitted to the control module 160 and the control module 160 analyzes the corresponding data.

The control module 160 analyzes the image data to determine whether an external hazard exists in the left and right or rear image data of the wearer (S130), and if it is determined that the external hazard exists, a sound notification to the hearing aid module 110. To generate (S140). If no external risk factors exist, the process returns to the image data collection step and continues collecting image data.

Separately, the control module 160 may check the wearer's body state by analyzing the biometric data. If the pulse rate and / or body temperature is a predetermined first range, for example, the pulse is 100 times per minute, the body temperature exceeds 38 ° C, or the pulse is 40 times per minute, and the body temperature is lower than 35 ° C. The state is determined to be an abnormal state, thereby increasing the size and repeat frequency of the notification sound of the hearing aid module 110 (S160). When there is no external risk factor at the time of biometric data analysis, the setting value of the notification sound may be changed to have a loudness and a repetition frequency of the sound changed by the setting value at the time of generating the notification sound. That is, the size and repetition frequency of the previously changed notification sound may be directly applied.

In addition, the analysis of the biological data indicates that the second range in which the pulse and / or body temperature is greater than the predetermined first range, for example, the pulse rate is 120 times per minute, the body temperature exceeds 40 ° C, or the pulse rate is 20 times per minute, and the body temperature is 33 ° C. In case of falling below, the wearer may be determined to be an emergency state of emergency, and the risk of the wearer may be propagated to the outside through the LED module 170 and the speaker module 180 (S180).

Referring to FIG. 13, the principle of determining the external hazard 300 is shown sequentially.

First, first and second external images are obtained from the camera modules 130 and 140 (S210). For example, the first external image may be an image in the left and right directions of the wearer, and the second external image may be an image of the rear of the wearer.

Subsequently, the external image is encoded to generate image data in the form of a digital media file (S220). For this purpose, a known video encoder may be used, and various known image or video file formats may be used.

The generated image data is analyzed to extract objects that meet specific conditions. As described above, a specific object is extracted using a known algorithm such as an adaptive Gaussian mixture model (S230).

It is determined whether the extracted object is greater than or equal to the predetermined size, and if it is greater than or equal to the predetermined size, it is determined whether the specific object approaches the wearer's side (S240). As a result of the determination, when the specific object approaches the wearer side, the approach speed of the specific object is extracted (S250). As a result of the approach speed extraction (S260), when the predetermined range, for example, 30 km per hour or more, the corresponding object may be determined as the external risk factor 300 (S270).

As such, according to the present embodiment, the camera modules 130 and 140 are installed outside the hearing aid 100, and the external hazards 300 on the left, right, and rear sides of the wearer are installed through the camera modules 130 and 140. By recognizing the information in advance and providing the information to the deaf person, the deaf person can respond in advance, thus preventing the loss of life.

In addition, in addition to detecting the external risk factors 300 in real time, considering the internal risk factors derived by analyzing the biometric data, if the breathing of the hearing impaired, body temperature or pulse, such as unstable state, recognize the alarm sound notification Because of the high possibility of not being able to do this, it is possible to amplify the pattern period and size of the notification sound to deliver appropriate stimuli to the hearing impaired.

In addition, considering the internal risk factors of the wearer obtained by analyzing the biometric data, if a dangerous situation is found, the external speaker and / or the LED module may be activated to spread the dangerous situation to others so that the wearer may be promptly taken. Can be.

Hereinafter, a risk factor detection system using a hearing aid according to an embodiment of the present invention will be described with reference to FIGS. 14 and 15. FIG. 14 is a block diagram schematically illustrating a risk factor detection system using a hearing aid according to an exemplary embodiment of the present invention, and FIG. 15 is a flowchart sequentially illustrating a principle of operation of the system of FIG. 14.

The risk factor detection system using a hearing aid according to an embodiment of the present invention includes a communication module 190 and wirelessly communicates with the hearing aid device 100 and the hearing aid device 100 that can communicate wirelessly with a predetermined data. Mediation server 500 for transmitting and receiving, and a plurality of distributed guidance server 600 connected to the mediation server 500, the hearing aid device 100 includes a camera module (130, 140) for acquiring an external image The hearing aid apparatus 100 transmits the image data obtained from the camera modules 130 and 140 to the mediation server 500, and the mediation server 500 transmits the image data to one of the plurality of distributed guide servers 600. One of the plurality of distributed guide servers 600 analyzes the image data and detects an external risk factor in the unviewed area of the wearer who wears the hearing aid device 100, and when the external risk factor is detected, the hearing aid device 100 is detected. First beep It transmits.

The hearing aid device 100 according to the present embodiment is the same as the hearing aid according to the above-described embodiment, except that it may selectively perform one of a remote mode and a local mode. That is, the hearing aid device 100 further includes a control module 160 for receiving image data. When the wearer sets the mode of the hearing aid device 100 to a local mode using a switch button (not shown) or the like. For example, the hearing aid may operate in the same manner as the hearing aid described above. That is, the control module 160 directly analyzes the image data to detect an external hazard in the unviewed area when the wearer walks, and transmits a first warning sound to the hearing aid module 110 of the hearing aid device when the external hazard is detected. Can be. At this time, the control module 160 may block the connection so that the image data is not transmitted to the intermediary server 500.

On the other hand, when the remote mode is selected, the risk factor detection and monitoring are not performed in the hearing aid device 100, and the corresponding analysis process may be performed by an external distributed guidance server 600.

Specifically, the hearing aid device 100 communicates with the intermediary server 500, and the intermediary server 500 maintains a state of being connected to the plurality of distributed guide servers 600. When the hearing aid device 100 is set to the remote mode and connected to the mediation server 500, the mediation server 500 may select one of the plurality of distributed guide servers 600 to connect with the hearing aid device 100. In this case, the mediation server 500 may receive resource information of the plurality of distributed guidance servers 600 in advance, and the mediation server 500 may distribute guidance to be connected to the hearing aid device 100 based on such resource information. The server 600 can be determined.

When the distributed guide server 600 is determined, the mediation server 500 may transmit the image data to the corresponding distributed guide server 600 having the most available resources among the distributed guide servers 600.

In some other embodiments, the image data or the biometric data may be directly transmitted between the hearing aid device 100 and the distributed guidance server 600 which have been mutually authenticated from the hearing aid device 100 without passing through the intermediary server 500. . In this case, the control signal according to the analysis result of the image data or the biometric data may also be directly transmitted from the distributed guidance server 600 to the hearing aid device 100.

The mediation server 500 and the distributed guidance server 600 according to the present embodiment receive a connection request signal from a client, that is, the hearing aid device 100 and allow the connection through a predetermined authentication process, similar to the configuration of a general server. Predetermined data can be transmitted and received.

The intermediary server 500 may be connected to the plurality of hearing aid devices 100, and performs a function of routing the hearing aid device 100 requesting a connection to the appropriate distributed guidance server 600.

The distributed guidance server 600 may be provided in plural, and may perform a function of distributing traffic so that an excessive load is not applied to the intermediate server 500.

The distributed guidance server 600 may be configured to perform a function of the hearing aid device 100 performing in the local mode. That is, the image data and the biometric data obtained from the hearing aid device 100 are directly received through the mediation server 500 or analyzed, and the external risk factors of the wearer who is wearing the hearing aid device 100 are not visible. When the external hazard is detected, the hearing aid device 100 may transmit a signal corresponding to the first warning sound, and the hearing aid device 100 may receive the signal to reproduce the first warning sound to the wearer.

The distributed guidance server 600 may employ a high-performance central processing unit and a high-capacity memory, etc., compared to the hearing aid device 100, and may significantly improve the processing speed and precision of image data and biometric data, compared to the local mode. .

As shown, the hearing aid device 100 may further include a biometric data measurement module 150 disposed inside the kernel inserted into the ear canal of the wearer to obtain biometric data of the wearer, and the hearing aid device 100 Receives the biometric data obtained from the biometric data measurement module 150 via the mediation server 500 or directly to the distributed guidance server 600, one of the plurality of distributed guidance server 600 (610, 620, 630 may monitor the wearer's internal risk factors by analyzing biometric data and transmit a second warning sound through the hearing aid device 100 when the internal risk factor is detected. That is, in addition to the external hazard, when the wearer's internal hazard, ie, the pulse, breathing or body temperature, as described above, is out of a predetermined range, a second warning sound different from the first beep that senses this as a hazard and informs the external hazard Can transmit

Such biometric data may include pulse and body temperature, and one of the plurality of distributed guidance servers 600 may determine the magnitude and repetition frequency of the first warning sound in consideration of the biometric data when detecting an external risk factor in the unviewed region. If the external risk is detected under the condition that the body temperature is above the first temperature from the normal temperature or the pulse is above the first value from the normal pulse, one of the plurality of distributed guide server 600, The loudness and repetition frequency of the alert sound can be increased beyond a predetermined range.

In addition, the hearing aid device 100 may further include a plurality of LED modules 170 for emitting light to the outside and a speaker module 180 for transmitting a predetermined sound to the outside, the body temperature is normal body temperature One of the plurality of distributed guide servers 600 when an external hazard is detected under a condition in which the temperature deviates by more than the second temperature higher than the first temperature or when the pulse deviates from the normal pulse by more than the second value higher than the first value. The electronic device may transmit the first warning sound to the speaker module to propagate the wearer's state to the outside, and may periodically blink the plurality of LED modules.

In particular, when the body temperature is out of the second temperature or the pulse is out of the second value, the size and repetition frequency of the first warning sound may be increased to the maximum range for the wearer of the hearing aid device 100 to recognize.

As described in the above embodiment, the first warning sound may have a different pattern according to the position of the unviewed area where the external hazard exists, and the first warning sound may be a camera module on the side where the external hazard is detected. Output to hearing aid module 110 in the direction.

In addition, the external risk factor may be determined as an object approaching the wearer's side by a predetermined size or more and a predetermined speed among the recognized objects of the first external image and the second external image obtained from the camera module and the auxiliary camera module 130 and 140. In this process, the plurality of distributed guidance servers 600 may detect the external risk factor based on the adaptive Gaussian Mixture Model.

Hereinafter, with reference to FIG. 15, the structure of the hearing aid apparatus 100, the intermediary server 500, and the distribution guide server 600 is demonstrated.

The mediation server 500 is connected to the plurality of distributed guide servers 600, and collects resource information for each server from the plurality of distributed guide servers 600 (S302). The intermediary server 500 may periodically collect such resource information, and may determine distribution of the hearing aid device 100 to be next accessed in consideration of remaining resources, for example, the share of the central processing unit or memory usage. have.

The hearing aid device 100 obtains image data and biometric data through a camera module and a biometric data measurement module in operation S304. Subsequently, the intermediate server 500 requests a distributed guide server connection (S306). The mediation server 500 that has received the request may perform a predetermined authentication process, and after performing the authentication process, the distributed guidance server to be allocated to the hearing aid device 100 is determined in consideration of the available resources (S308). The determined information about the distributed guide server is transmitted to the hearing aid device 100 (S310).

Subsequently, the hearing aid device 100 may attempt to connect to the distributed guidance server 600 (S312). The distributed guidance server 600 may perform a predetermined authentication process (S314), and the certified hearing aid device 100 may transmit the obtained image data and biometric data to the distributed guidance server 600 (S316). .

As described above, the distributed guidance server 600 receiving the data analyzes the image data and the biometric data and detects an external risk factor (S318 and S320). When an external hazard is detected, the first hearing sound is transmitted to the hearing aid device 100 (S322). When the internal risk factor is detected as a result of analyzing the biometric data (S324), the second warning sound is transmitted (S326).

Subsequently, when an external hazard exists, the magnitude and repeat frequency of the first warning sound may be increased according to the range of the internal hazard (S328). In addition, the electronic signal for driving the speaker module and the LED module may be transmitted to the hearing aid device 100 according to the range of the internal risk factors (S330).

If only internal hazards are present and no external hazards exist (i.e., analysis of biometric data shows that the pulse or temperature is outside of a certain range), the electric power for driving the speaker module and the LED module can be used to draw attention to outsiders. A typical signal may be transmitted to the hearing aid device 100.

As described above, according to the present embodiment, the hearing aid device 100 transmits and receives predetermined data with the mediation server 500 and the distributed guide server 600, and the hearing aid device 100 is controlled by the distributed guide server 600. Each configuration of the can be controlled, it can greatly improve the risk awareness of the wearer, it may be to spread the risk of the wearer to the outside to avoid the dangerous situation by the help of others.

It will be understood by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. The scope of the present invention is defined by the appended claims rather than the foregoing detailed description, and all changes or modifications derived from the meaning and scope of the claims and the equivalents thereof are included in the scope of the present invention Should be interpreted.

100: Hearing Aid Device
110: hearing aid module
120: curved frame
130: camera module
140: secondary camera module
150: biometric data measurement module
160: control module
170: LED module
180: speaker module
190: communication module

Claims (6)

Neckband type both ears insert hearing aid,
A pair of hearing aid module for outputting a sound by processing the signal sensed from the outside;
A curved frame connecting the pair of hearing aid modules;
A pair of camera modules disposed on outer surfaces of the pair of hearing aid modules, respectively, for acquiring a first external image of a side;
An auxiliary camera module disposed at a center portion of the curved frame to acquire a second external image behind the curved frame;
A biometric data measurement module disposed inside the kernel inserted into the ear canal of the pair of hearing aid modules to acquire biometric data of a wearer;
A vibration motor provided in the pair of hearing aid modules to form a vibration pattern by a predetermined vibration control signal; And
And a control module for receiving the biometric data and collecting the biometric state of the wearer.
The control module periodically transmits the biometric data to an external management server,
The control module receives the first external image and the second external image and analyzes the detected external hazards in the unviewed area of the wearer,
The control module generates the vibration pattern by transmitting the vibration control signal to the vibration motor when the external hazard is detected.
The biometric data includes pulse and body temperature,
The control module determines the magnitude and repetition frequency of the vibration pattern in consideration of the biometric data when detecting an external risk factor in the field of view unclear,
When the external risk factor is detected under the condition that the body temperature deviates from the normal temperature by more than the first temperature or the pulse deviates from the normal pulse by the first value or more, the control module determines the magnitude and repetition frequency of the vibration pattern. Both ears insert hearing aid with a vibration pattern notification function to increase beyond the range. delete The method of claim 1,
A plurality of LED modules attached to an outer side of the curved frame to emit light by signals of the control module; And
It is attached to the outside of the curved frame further comprises a speaker module for transmitting a predetermined sound to the outside due to the control of the control module,
The control module when the external hazard is detected under a condition that the body temperature deviates from the normal body temperature by more than the second temperature higher than the first temperature or the pulse deviates from the normal pulse by more than the second value higher than the first value. Is, both ears inserted hearing aid having a vibration pattern notification function to transmit a notification sound to the speaker module to propagate the state of the wearer to the outside, and periodically flashing the plurality of LED modules. The method of claim 1,
The vibration pattern is generated by the vibration motor provided in the hearing aid module in the direction of the camera module on the side where the external hazard is detected, both ears inserted hearing aid having a vibration pattern notification function. The method of claim 1,
The external risk factor is a double-ear insert hearing aid having a vibration pattern notification function, which is determined as the object approaching the wearer side by more than a predetermined size and a predetermined speed among the recognized objects of the first external image and the second external image. . The method of claim 1,
And the control module detects the external hazard based on an adaptive Gaussian Mixture Model.

Images