WO2017190803A1 - Ambient sound monitoring and visualizing system for hearing impaired persons - Google Patents

Ambient sound monitoring and visualizing system for hearing impaired persons Download PDF

Info

Publication number
WO2017190803A1
WO2017190803A1 PCT/EP2016/060204 EP2016060204W WO2017190803A1 WO 2017190803 A1 WO2017190803 A1 WO 2017190803A1 EP 2016060204 W EP2016060204 W EP 2016060204W WO 2017190803 A1 WO2017190803 A1 WO 2017190803A1
Authority
WO
WIPO (PCT)
Prior art keywords
display device
image display
notification system
sound
flashing visual
Prior art date
Application number
PCT/EP2016/060204
Other languages
French (fr)
Inventor
Emrah YAVUZ
Mustafa UGUZ
Osman SOLAKOGLU
Lutfiye DURAK ATA
Ahmet SERBES
Nurullah CALIK
Original Assignee
Arcelik Anonim Sirketi
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Arcelik Anonim Sirketi filed Critical Arcelik Anonim Sirketi
Priority to PCT/EP2016/060204 priority Critical patent/WO2017190803A1/en
Priority to TR2017/06630A priority patent/TR201706630A2/en
Publication of WO2017190803A1 publication Critical patent/WO2017190803A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00—Teaching, or communicating with, the blind, deaf or mute
    • G09B21/009—Teaching or communicating with deaf persons

Definitions

  • WO2011000113 discloses a system for notifying a deaf/hearing impaired user of multiple sounds produced in the ambient environment including non-verbal sounds and words spoken by the human voice, said system comprising a controller, at least one audio input device and at least one output device.
  • the controller provides for storing a plurality of target sounds.
  • the audio input device continuously captures in real-time sounds produced in the ambient environment and transmits in real-time the captured sounds to the controller.
  • the controller compares in real-time a given captured sound to each target sound in order to determine a match there between.
  • the controller signals the output device when a match is made.
  • the output device receives this signal from the controller that a given sound produced in the ambient environment has been captured and produces a notification signal thereby notifying in real-time the user of the given captured sound.
  • Primary object of the present invention is to provide an ambient sound notification system for monitoring ambient sounds and generating respective warning signals in association with predetermined sounds by means of an image display device such as a TV set.
  • the present invention proposes a notification system comprising an image display device and at least one remote audio input device in signal communication with said image display device.
  • the remotely captured real-time ambient sounds as received by the image display device are processed by an electronic control unit of said image display device and thereby associated, if any, with a certain preloaded sound pattern based on an internal database of the image display device.
  • the electronic control unit automatically wakes up the screen and generates a flashing visual image to be communicated to the user with hearing loss or partial inability to hear.
  • said flashing visual image is replaced by a stationary informational screen with a textual description of the urgent situation upon recognition of a certain hand clapping pattern or pressing of a dedicated button of the image display device or a remote control thereof.
  • Fig. 1 demonstrates a general flow diagram of an algorithm implemented in an audio input device according to the present invention.
  • the present invention relates to an image display device having an automatic user notification function.
  • the present invention proposes a system and method for monitoring ambient sounds and a means for visualizing processed ambient sounds for hearing impaired persons.
  • the image display device of the present invention comprises an electronic control unit in communication with a plurality of audio input devices typically in the form of microphone units in wireless communication with said image display device.
  • Each audio input device can be typically installed in a separate room in a house.
  • Each audio input device monitors and captures real-time ambient sounds and transmits the same to the image display device.
  • the electronic control unit compares received ambient sounds to a plurality of sound patterns in a database of preloaded sound patterns so that the ambient sound can be categorically defined.
  • the electronic control unit of the image display device performing analysis of the ambient sounds, generates on the screen of the image display device a black and white flashing visual image as an indication of the detected sound being classified as an urgent situation to be communicated to the user.
  • the captured signal is multiplied by window function as a common method to reduce the noise.
  • the audio input device applies linear predictive coding (LPC) to the output of a pre-filter.
  • LPC analysis is a known technique as taught in various references including the text L. Rabiner & B. Juang, Fundamentals of Speech Recognition.
  • the LPC analysis provides LPC coefficients as features that describe formants and a residual signal.
  • the image display device assumes the role of a central node receiving data from a plurality of peripheral nodes, i.e. audio input devices.
  • Each audio input device s readings being compared to a reference sound pattern, upon verification of a match with a preloaded sound signal pattern indicative of an urgent situation such as a fire alarm signal, the electronic control unit automatically wakes up the image display device if it is not already turned on, and generates on the screen a high-contrast preferably black and white flashing visual image.
  • the flashing visual screen is effective in visually notifying the emerging urgent situation in a flashing manner with a strong impact to the user having hearing impairment.
  • the flashing screen will be replaced by an informational screen indicating a textual description of the urgent event in response to recognition of a user's commands.
  • the informational screen is shown when a certain number of claps being made are recognized and therefore, the person with hearing impairment clapping his palms together (for instance twice) activating the informational screen is directly notified about the urgent need to address the situation.
  • the informational screen is not shown to the user upon waking up of the image display device in advance because the flashing screen is needed to first catch attention of the user and it is therefore ensured that the information is only displayed in the presence of the user because his presence is verified when the image display device recognizes a certain number of claps through a respective microphone unit, which can for instance be an additional built-in unit.
  • Said audio input devices as well as the image display device typically are peripheral and central network nodes with network interface cards to establish a network connection in a LAN.
  • the present invention proposes a notification system comprising an image display device and at least one remote audio input device in signal communication with said image display device and capable of capturing ambient sound in the vicinity thereof and transmitting the captured ambient sound to said image display device.
  • said at least one remote audio input device monitors and captures real-time ambient sounds and transmits the same to the image display device and an electronic control unit of said image display device compares received ambient sounds to a plurality of sound patterns in a database of preloaded sound patterns in the manner that the ambient sound is categorically definable.
  • an internal memory unit of the image display device stores a database of sound patterns to be associated with different predetermined situations.
  • said black and white flashing visual image is activated in a time-dependent manner during night time.
  • said flashing visual image is a high contrast color flashing visual image activated during daytime.
  • said flashing visual image is replaced by a stationary informational screen with a textual description of the urgent situation to be communicated in response to recognition of a user command by said image display device.
  • said stationary informational screen is activated upon recognition of a certain number of claps by means of an audio input device in the vicinity of the image display device.
  • said stationary informational screen is activated upon pressing of a dedicated button of the image display device or a remote control thereof.
  • said textual description of the urgent situation is communicated to a portable electronic device.
  • said portable electronic device is user-wearable device.
  • the automatic notification system is advantageous in that an urgent situation can be effectively communicated to the user by waking up the screen of the TV set and by verifying the user’s presence in the vicinity of the image display device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Selective Calling Equipment (AREA)
  • Details Of Television Systems (AREA)

Abstract

The present invention relates to a system and method for monitoring ambient sound and means for visualizing processed ambient sound in a visualized manner for hearing impaired persons. The present invention more particularly relates to a notification system comprising an image display device and at least one remote audio input device in signal communication with said image display device and capable of capturing ambient sound in the vicinity thereof and transmitting the captured ambient sound to said image display device.

Description

AMBIENT SOUND MONITORING AND VISUALIZING SYSTEM FOR HEARING IMPAIRED PERSONS
The present invention relates to a system and method for monitoring ambient sound and means for visualizing processed ambient sound in a visualized manner for hearing impaired persons.
Warning devices for the hearing impaired or deaf persons are intended to timely convey important information as to an urgent situation where persons with hearing impairment are present. Therefore, environment sensing systems together with a personal warning means wearable by a person with hearing impairment are known per se. Such a system and wearable unit is particularly effective in notifying the user of a sound event in the surrounding environment.
A further approach involves a system with a plurality of listening nodes in communication with a central station. The system monitors ambient sounds and when needed, converts the processed signal into textual information blocks by which an individual is alerted with an alerting signal when pre-recorded digital information matches the ambient digital information corresponding to the warning signal.
A prior art publication in the technical field of the present invention may be referred to as WO2011000113, which discloses a system for notifying a deaf/hearing impaired user of multiple sounds produced in the ambient environment including non-verbal sounds and words spoken by the human voice, said system comprising a controller, at least one audio input device and at least one output device. The controller provides for storing a plurality of target sounds. The audio input device continuously captures in real-time sounds produced in the ambient environment and transmits in real-time the captured sounds to the controller. The controller compares in real-time a given captured sound to each target sound in order to determine a match there between. The controller signals the output device when a match is made. The output device receives this signal from the controller that a given sound produced in the ambient environment has been captured and produces a notification signal thereby notifying in real-time the user of the given captured sound.
The present invention provides an ambient sound notification system automatically generating a visual notification signal by means of an image display device as defined by the characterizing features in Claim 1.
Primary object of the present invention is to provide an ambient sound notification system for monitoring ambient sounds and generating respective warning signals in association with predetermined sounds by means of an image display device such as a TV set.
The present invention proposes a notification system comprising an image display device and at least one remote audio input device in signal communication with said image display device. The remotely captured real-time ambient sounds as received by the image display device are processed by an electronic control unit of said image display device and thereby associated, if any, with a certain preloaded sound pattern based on an internal database of the image display device.
If the image display device’s screen is not already active, the electronic control unit automatically wakes up the screen and generates a flashing visual image to be communicated to the user with hearing loss or partial inability to hear.
In an additional variation of the present invention, said flashing visual image is replaced by a stationary informational screen with a textual description of the urgent situation upon recognition of a certain hand clapping pattern or pressing of a dedicated button of the image display device or a remote control thereof.
Accompanying drawings are given solely for the purpose of exemplifying a system and method by which an image display device is operable, whose advantages over prior art were outlined above and will be explained in brief hereinafter.
The drawings are not meant to delimit the scope of protection as identified in the claims nor should they be referred to alone in an effort to interpret the scope identified in said claims without recourse to the technical disclosure in the description of the present invention.
Fig. 1 demonstrates a general flow diagram of an algorithm implemented in an audio input device according to the present invention.
Fig. 2 demonstrates a general flow diagram of an algorithm implemented by an electronic control unit of an image display device in signal communication with said audio input device according to the present invention.
The present invention relates to an image display device having an automatic user notification function. The present invention proposes a system and method for monitoring ambient sounds and a means for visualizing processed ambient sounds for hearing impaired persons.
The image display device of the present invention comprises an electronic control unit in communication with a plurality of audio input devices typically in the form of microphone units in wireless communication with said image display device. Each audio input device can be typically installed in a separate room in a house. Each audio input device monitors and captures real-time ambient sounds and transmits the same to the image display device. The electronic control unit compares received ambient sounds to a plurality of sound patterns in a database of preloaded sound patterns so that the ambient sound can be categorically defined.
The electronic control unit of the image display device, performing analysis of the ambient sounds, generates on the screen of the image display device a black and white flashing visual image as an indication of the detected sound being classified as an urgent situation to be communicated to the user. Before audio input devices transmit captured ambient sounds, the captured signal is multiplied by window function as a common method to reduce the noise. The audio input device applies linear predictive coding (LPC) to the output of a pre-filter. LPC analysis is a known technique as taught in various references including the text L. Rabiner & B. Juang, Fundamentals of Speech Recognition. The LPC analysis provides LPC coefficients as features that describe formants and a residual signal.
According to the present invention, the image display device assumes the role of a central node receiving data from a plurality of peripheral nodes, i.e. audio input devices. Each audio input device’s readings being compared to a reference sound pattern, upon verification of a match with a preloaded sound signal pattern indicative of an urgent situation such as a fire alarm signal, the electronic control unit automatically wakes up the image display device if it is not already turned on, and generates on the screen a high-contrast preferably black and white flashing visual image. The flashing visual screen is effective in visually notifying the emerging urgent situation in a flashing manner with a strong impact to the user having hearing impairment.
In a variation of the invention, the flashing black and white visual image is used in a time-dependent manner as it will be more noticeable during night time. On the other hand, high contrast color images flashing in the same manner can be used during daytime.
Additionally, in a variation of the present invention, the flashing screen will be replaced by an informational screen indicating a textual description of the urgent event in response to recognition of a user's commands. According to the invention, the informational screen is shown when a certain number of claps being made are recognized and therefore, the person with hearing impairment clapping his palms together (for instance twice) activating the informational screen is directly notified about the urgent need to address the situation.
The informational screen is not shown to the user upon waking up of the image display device in advance because the flashing screen is needed to first catch attention of the user and it is therefore ensured that the information is only displayed in the presence of the user because his presence is verified when the image display device recognizes a certain number of claps through a respective microphone unit, which can for instance be an additional built-in unit.
Said audio input devices as well as the image display device typically are peripheral and central network nodes with network interface cards to establish a network connection in a LAN.
In a nutshell, the present invention proposes a notification system comprising an image display device and at least one remote audio input device in signal communication with said image display device and capable of capturing ambient sound in the vicinity thereof and transmitting the captured ambient sound to said image display device.
In one embodiment of the present invention, said at least one remote audio input device monitors and captures real-time ambient sounds and transmits the same to the image display device and an electronic control unit of said image display device compares received ambient sounds to a plurality of sound patterns in a database of preloaded sound patterns in the manner that the ambient sound is categorically definable.
In a further embodiment of the present invention, said electronic control unit automatically wakes up the image display device screen and generates on the screen a flashing visual image as an indication of a detected sound being classified as an urgent situation to be communicated.
In a further embodiment of the present invention, an internal memory unit of the image display device stores a database of sound patterns to be associated with different predetermined situations.
In a further embodiment of the present invention, said flashing visual image is a high contrast black and white flashing visual image.
In a further embodiment of the present invention, said black and white flashing visual image is activated in a time-dependent manner during night time.
In a further embodiment of the present invention, said flashing visual image is a high contrast color flashing visual image activated during daytime.
In a further embodiment of the present invention, said flashing visual image is replaced by a stationary informational screen with a textual description of the urgent situation to be communicated in response to recognition of a user command by said image display device.
In a further embodiment of the present invention, said stationary informational screen is activated upon recognition of a certain number of claps by means of an audio input device in the vicinity of the image display device.
In a further embodiment of the present invention, said stationary informational screen is activated upon pressing of a dedicated button of the image display device or a remote control thereof.
In a further embodiment of the present invention, said textual description of the urgent situation is communicated to a portable electronic device.
In a further embodiment of the present invention, said portable electronic device is user-wearable device.
The automatic notification system is advantageous in that an urgent situation can be effectively communicated to the user by waking up the screen of the TV set and by verifying the user’s presence in the vicinity of the image display device.

Claims (10)

  1. A notification system comprising an image display device and at least one remote audio input device in signal communication with said image display device and capable of capturing ambient sound in the vicinity thereof and transmitting the captured ambient sound to said image display device characterized in that;
    said at least one remote audio input device is capable of monitoring and capturing real-time ambient sounds and transmitting the same to the image display device and an electronic control unit of said image display device is capable of comparing received ambient sounds to a plurality of sound patterns in a database of preloaded sound patterns in the manner that the ambient sound is categorically definable,
    said electronic control unit automatically wakes up the image display device screen and generates on the screen a flashing visual image as an indication of a detected sound being classified as an urgent situation to be communicated.
  2. A notification system as in Claim 1, characterized in that an internal memory unit of the image display device stores a database of sound patterns to be associated with different predetermined situations.
  3. A notification system as in Claim 1 or 2, characterized in that said flashing visual image is a high contrast black and white flashing visual image.
  4. A notification system as in Claim 3, characterized in that said black and white flashing visual image is activated in a time-dependent manner during night time.
  5. A notification system as in Claim 1 or 2, characterized in that said flashing visual image is a high contrast color flashing visual image activated during daytime.
  6. A notification system as in Claim 3, 4 or 5, characterized in that said flashing visual image is replaced by a stationary informational screen with a textual description of the urgent situation to be communicated in response to recognition of a user command by said image display device.
  7. A notification system as in Claim 6, characterized in that said stationary informational screen is activated upon recognition of a certain number of claps by means of an audio input device in the vicinity of the image display device.
  8. A notification system as in Claim 6 or 7, characterized in that said stationary informational screen is activated upon pressing of a dedicated button of the image display device or a remote control thereof.
  9. A notification system as in Claim 1 or 2, characterized in that said textual description of the urgent situation is communicated to a portable electronic device.
  10. A notification system as in Claim 9, characterized in that said portable electronic device is user-wearable device.
PCT/EP2016/060204 2016-05-06 2016-05-06 Ambient sound monitoring and visualizing system for hearing impaired persons WO2017190803A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/EP2016/060204 WO2017190803A1 (en) 2016-05-06 2016-05-06 Ambient sound monitoring and visualizing system for hearing impaired persons
TR2017/06630A TR201706630A2 (en) 2016-05-06 2017-05-05 MEDIA AUDIO MONITORING AND VISUALIZATION SYSTEM FOR THE Hearing Impaired

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2016/060204 WO2017190803A1 (en) 2016-05-06 2016-05-06 Ambient sound monitoring and visualizing system for hearing impaired persons

Publications (1)

Publication Number Publication Date
WO2017190803A1 true WO2017190803A1 (en) 2017-11-09

Family

ID=55953155

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2016/060204 WO2017190803A1 (en) 2016-05-06 2016-05-06 Ambient sound monitoring and visualizing system for hearing impaired persons

Country Status (2)

Country Link
TR (1) TR201706630A2 (en)
WO (1) WO2017190803A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11722763B2 (en) 2021-08-06 2023-08-08 Motorola Solutions, Inc. System and method for audio tagging of an object of interest

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997008896A1 (en) * 1995-08-23 1997-03-06 Scientific-Atlanta, Inc. Open area security system
US20020198716A1 (en) * 2001-06-25 2002-12-26 Kurt Zimmerman System and method of improved communication
US20060017559A1 (en) * 2004-07-23 2006-01-26 Albert David E Enhanced fire, safety, security and health monitoring and alarm response method, system and device
WO2008067638A1 (en) * 2006-12-08 2008-06-12 Harmonya Technologies Inc. Environment sensing system for the hearing-impaired
WO2011000113A1 (en) 2009-06-30 2011-01-06 Harmonya Technologies Multiple sound and voice detector for hearing- impaired or deaf person
US20130132095A1 (en) * 2011-11-17 2013-05-23 Microsoft Corporation Audio pattern matching for device activation
EP2733698A1 (en) * 2012-11-14 2014-05-21 Deutsche Telekom AG Apparatus for communicating and/or for displaying information
FR3000270A1 (en) * 2012-12-21 2014-06-27 Finsecur Device for signaling alarm for providing safety to deaf people in common lobby of hotel against fire risk, has actuator providing visual and/or tactile alarm signal, and pairing unit matching transmitted identifier with stored identifier

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997008896A1 (en) * 1995-08-23 1997-03-06 Scientific-Atlanta, Inc. Open area security system
US20020198716A1 (en) * 2001-06-25 2002-12-26 Kurt Zimmerman System and method of improved communication
US20060017559A1 (en) * 2004-07-23 2006-01-26 Albert David E Enhanced fire, safety, security and health monitoring and alarm response method, system and device
WO2008067638A1 (en) * 2006-12-08 2008-06-12 Harmonya Technologies Inc. Environment sensing system for the hearing-impaired
WO2011000113A1 (en) 2009-06-30 2011-01-06 Harmonya Technologies Multiple sound and voice detector for hearing- impaired or deaf person
US20130132095A1 (en) * 2011-11-17 2013-05-23 Microsoft Corporation Audio pattern matching for device activation
EP2733698A1 (en) * 2012-11-14 2014-05-21 Deutsche Telekom AG Apparatus for communicating and/or for displaying information
FR3000270A1 (en) * 2012-12-21 2014-06-27 Finsecur Device for signaling alarm for providing safety to deaf people in common lobby of hotel against fire risk, has actuator providing visual and/or tactile alarm signal, and pairing unit matching transmitted identifier with stored identifier

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11722763B2 (en) 2021-08-06 2023-08-08 Motorola Solutions, Inc. System and method for audio tagging of an object of interest

Also Published As

Publication number Publication date
TR201706630A2 (en) 2017-11-21

Similar Documents

Publication Publication Date Title
US10171677B2 (en) Systems and methods for monitoring sound during an in-building emergency
US10506411B1 (en) Portable home and hotel security system
US10455342B2 (en) Sound event detecting apparatus and operation method thereof
ES2971050T3 (en) Notification method and device
JP2007228459A (en) Monitoring system
TW201342313A (en) Voice recognition nursing method and system thereof
KR101328865B1 (en) Wrist watch for deaf and its control method
CN109451385A (en) A kind of based reminding method and device based on when using earphone
KR102488741B1 (en) Emergency bell system with improved on-site situation identification
KR101342448B1 (en) Remote monitoring apparatus and the method thereof
CN106504750A (en) A kind of device and method thereof of utilization Application on Voiceprint Recognition quick detection scene of fire comburant
WO2011000113A1 (en) Multiple sound and voice detector for hearing- impaired or deaf person
JP6310243B2 (en) Automatic fire alarm equipment
WO2017190803A1 (en) Ambient sound monitoring and visualizing system for hearing impaired persons
KR101615824B1 (en) spontaneousness rescue apparatus for vehicle
EP2983148B1 (en) Voice-sensitive emergency alert system
JP2015111327A5 (en)
KR101709224B1 (en) Self security system for crime protection
TWI574232B (en) Security Host having Internet-of-Things Communication Function
KR101169316B1 (en) A security system coupled to controlling light emitting diode lamp
KR20200047083A (en) Emergency monitoring system for people with disabilities and the elderly according to voice recognition
WO2019034244A1 (en) Image display device provided with sound prioritization function
KR20180064762A (en) Scream recognition device located in indoor space
KR101153191B1 (en) System and method for preventing crime in parking lot using sound recognition
EP3489919B1 (en) Apparatus, method and computer program for indicating an emergency situation

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16721421

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 16721421

Country of ref document: EP

Kind code of ref document: A1