EP4646715A1

EP4646715A1 - Event-triggered audio-forwarding arrangement and wireless communication device

Info

Publication number: EP4646715A1
Application number: EP23836777.5A
Authority: EP
Inventors: Peter Deixler; Alexander Henricus Waltherus VAN EEUWIJK
Original assignee: Signify Holding BV
Current assignee: Signify Holding BV
Priority date: 2023-01-05
Filing date: 2023-12-20
Publication date: 2025-11-12
Also published as: CN120457471A; WO2024146799A1

Abstract

The invention is directed to an event-triggered audio-forwarding arrangement (500) comprising a voice-assistant device (502) configured to connect with a remote caregiver. The arrangement comprises a first wireless communication device (200) with a loudspeaker (202) arranged at a first location at a first distance from the microphone (504) of the voice-assistant device (502) a second wireless communication device (100) with a microphone (102) arranged at a second location at a second distance, and including an event-ascertaining unit (104) configured to ascertain an occurrence of a physical event related to a person (101) and an audio-data forwarding unit (106) configured to register a first audio signal (A1) and to provide first audio data (AD1) to the first wireless communication device, which is configured to play a second audio signal (A2) indicative of the first audio data (AD1) to be picked-up by the voices-assistance device, thereby increasing its effective spatial hearing-range.

Description

Event-triggered audio-forwarding arrangement and wireless communication device

FIELD OF THE INVENTION

The present invention is directed to a wireless communication device, to an event-triggered audio-forwarding arrangement, to a respective method for controlling operation thereof and to a computer program.

BACKGROUND OF THE INVENTION

Document US 2021/0160675 Al describes methods, systems and apparatus for smart speakerphone emergency monitoring, for instance for fall monitoring. The method includes detecting a sound at one or more multiple locations of a property, determining that the sound indicates an emergency, sending to a monitoring station, or call center, an indication of the emergency, receiving from the monitoring station a two-way voice call and broadcasting the two-way call to the one more multiple locations. Each speakerphone is configured to communicate with a hub device using digital enhanced cordless telecommunication (DECT) signals.

SUMMARY OF THE INVENTION

Falls, as a type of physical event, are one of the leading causes of injury and death among adults over 65, according to the Center for Disease Control and Prevention. The elderly’s inconsistent wearing of monitoring wearables asks for automated systems for fall prevention and fall detection. Leading loT sensing companies in the elderly care and falldetection domain concluded that their Al-driven fall detection services require a human in the loop. This is understandable given the may classes of falls, some of which are hard to detect even for advanced Al algorithms. Voice-assistant devices, such as for instance Amazon Echo devices, are becoming increasingly common in households, and leading players in the elderly care domain are introducing voice assistant speakers as part of their elderly care solutions, given the need for a human-in-the-loop decision making for fall prevention and/or fall detection. In the case of an emergency, such as a fall, a user can interact with the voiceassistant device to communicate with an remote caregiver, such as an external call center, a monitoring station or a family member or acquaintance of the user. A typical arrangement with just two voice-assistant devices per home will often not work, especially if the person has fallen at a location in the house or apartment which is outside a significant audio coverage or hearing range of the voice-assistant devices, i.e. that spatial range from where a sound of a given intensity can be registered with at least a predetermined quality by a microphone. However, including a respective voice-assistant device in more rooms (e.g. bedroom, kitchen, living-room, bathroom, etc.) drastically increases the cost of the elderly care solution.

It would be beneficial to increase the operational range of an event-monitoring arrangement connected to a remote caregiver such as an external call center or monitoring station with a reduced increase of the costs.

A first aspect of the present invention is formed by an event-triggered audioforwarding arrangement according to claim 1.

The event-triggered audio-forwarding arrangement, also referred to as the arrangement, comprises a voice-assistant device that includes a microphone and a loudspeaker and is configured to connect with a remote caregiver, which is external with respect to the arrangement. The remote caregiver may include, for instance, a call center or a monitoring station. The connection is achieved via a communication channel.

The arrangement further includes a first wireless communication device comprising a loudspeaker, the first wireless communication device arranged at a first distance from the microphone of the voice-assistant device.

The arrangement further includes one or more second wireless communication devices comprising a microphone, the second wireless communication device being arranged at a second distance from the microphone of the voice-assistant device.

The second wireless communication device comprises an event-ascertaining unit that is configured to ascertain an occurrence of a predefined physical event related to a person, and an audio-data forwarding unit that is configured to operate the microphone to register a first audio signal and to provide, upon ascertainment of the predefined physical event, first audio data indicative of the first audio signal to the first wireless communication device via an audio-data communication channel. In the arrangement of the first aspect of the invention, the first wireless communication device is configured, upon reception of the first audio data, to operate its loudspeaker to play a second audio signal indicative of the first audio data to be picked-up or registered by the microphone of the voice-assistant device. The voice-assistant device is also configured to provide second audio data indicative of the second audio signal to the external caregiver via the communication channel. The event-triggered audio forwarding arrangement of the first aspect of the present invention thus effectively extends an effective spatial hearing range or sound pick-up region of the voice-assistant device by enabling an audio-data communication channel between two wireless communication devices, where one of the communication devices, namely the first wireless communication devices is arranged at a first location separated from the microphone of the voice-assistant device by the first distance and effectively arranged within a sound pick-up region of the microphone of the voice-assistant device (e.g. in the same room, or at a relatively short distance), and the other communication device, namely the second wireless communication device, is arranged at a second location separated from the microphone of the voice-assistant device by the second distance and thus effectively arranged outside said sound pick-up region. The sound pick-up region can be defined as the spatial region or volume in which a sound or audio signal provided with a predetermined sound intensity (e.g. 60dB, or normal conversation level) can be captured or registered with at least a given signal-to-noise ratio or a predefined sound quality by the microphone of the voiceassistant device. The sound pick-up region of the voice-assistant device may be predetermined, e.g., a user may determine which wireless devices are within the sound pickup region of the voice assistant device, the wireless devices may determine whether they are within said sound pick up region by, for example, playing a test sound, the roles of the wireless devices, i.e., whether they are within the sound pick-up region may be stored in a memory, etc. The first distance and the second distance (or, in other words, the first and the second location) are such that a first sound intensity picked-up by the microphone of the voice-assistant device from a reference sound signal of a given intensity produced at the first distance (or first location) is higher than a second sound intensity picked-up by the microphone of the voice-assistant device from the reference sound signal of the same given intensity produced at the second distance (or second location). Typically the first distance is shorter than the second distance, so that the first wireless communication device is closer to the voice-assistant device than the second wireless communication device. However, and depending on the spatial orientation of the microphone and the loudspeaker and on the environment, e.g., the presence of objects such as walls, the second wireless communication device can be located in some cases closer to the voice-assistant device than the first communication device, i.e., the second distance is shorter than the first distance. The more relevant aspect is that a predetermined sound, referred to as reference sound, and having a predefined intensity, when provided at the location of the first wireless communication device, is registered or picked-up by the microphone of the voice-assistant device with a higher intensity or quality, or a lower signal-to-noise ration than when that same reference sound is provided at the location of the second wireless communication device.

The spatial extension of the sound pick-up region thus depends on the sound intensity, the sensitivity of the microphone and on the environment (e.g., presence of wall between the emitter of the sound and the microphone), Of course, the louder the sound, the larger the sound pick-up region. However, the sound pick-up region is preferably defined for sound intensities that are realistic in a domestic use, e.g. between 30 and 80 dB, and for a given environment (a home, a care facility, a warehouse, etc.).

The audio-data forwarding unit operates the microphone of the second wireless communication device for registering the sounds, referred to as audio signal, originated in its vicinity, which may include, for example, a person calling for help or a person moaning with pain. After the ascertainment of the event, which is referred to as a physical event of a person, and that may include, movement, in particular falls, gestures, vital signs such as breathing frequency or heart-beat, etc., by the event-ascertaining unit, an audiodata communication channel with the first wireless communication device is enabled or established, for providing the registered first audio signal, in the form of first audio data, to said first wireless communication device. The audio-data forwarding unit is thus configured to generate the audio data that is indicative of the registered audio signal. The first audio data is typically unprocessed in terms of speech-recognition by the second wireless communication device. In other words, unlike some voice-assistant devices, the second wireless communication device does not perform any speech recognition but registers and forwards the raw audio data or a compressed version of the raw audio data to the first wireless communication device. In turn, upon receiving the first audio data, the first wireless communication device is configured to operate its microphone and play the first audio data so that it is picked-up or registered by the microphone of the voice-assistant device. Thus, a second audio signal that is correlated to the first audio signal from the user, is picked up by the microphone of the voice-assistant device and provided, via the communication channel, to the remote caregiver. An operator of a call center or monitoring station, or a family member or supervisor, as possible examples of a remote caregiver, becomes then aware of what is happening in the vicinity of the second wireless communication device, which is not within the original sound pick-up region of the voice-assistant device. Thus, the event- triggered audio forwarding arrangement, besides needing a reduced number of voice-assistant devices, enhances the privacy of the environment since audio data is only forwarded after a predefined event has been detected. Further, not all wireless communication devices have to be directly connected to the remote caregiver. In addition, the wireless communication devices can utilize a very light weight microcontroller as they only need to capture the audio or replay the audio, but unlike voice assistant devices do not need to understand wake words or perform audio analytics.

In the frame of this description, audio signal refers to sound waves within a hearing frequency, either generated by the user in the case of the first audio signal, or generated by a loudspeaker in the case of the second audio signal. Audio data refers to a coded data signal indicative of an audio signal, which can be then transmitted over a wired or wireless communication link in accordance with a predetermined communication protocol.

In the following, embodiments of the event-triggered audio-forwarding arrangement of the first aspect of the invention will be described.

In a preferred embodiment, the event-triggered audio-forwarding arrangement is configured as a fall-triggered or gesture-triggered audio-forwarding arrangement, wherein the event-ascertaining unit is configured to ascertain event data indicative of a fall or a predetermined gesture of a person.

In a preferred embodiment, the event-ascertainment unit configured as a fall or gesture-ascertainment unit comprises, or receives fall or gesture data from, a radiofrequencybased fall or gesture-sensing arrangement that does not rely on an audio input to determine the occurrence of a fall or a gesture, and thus respects the privacy of the conversations inside the environment (e.g., the house, the apartment, etc.). The radiofrequency-based fall or gesture-sensing arrangement is configured to determine a fall or a gesture, or any other predetermined activity including vital signs, by using signal data indicative of one or more known signal parameters that are correlatable to the predefined event, such as for instance, a received signal strength indicator of a channel state information. A radiofrequency -based fall or gesture-sensing arrangement can be part of a broader activity sensing arrangement that is advantageously configured to detect not only occurrences of falls or gestures, but also presence, movement, gait, gestures, or vital signs such as heart beat or breathing frequency of a user within a given sensing volume.

Preferably, in an embodiment, wherein the audio data communication channel between the first wireless communication device and the second wireless communication device is based on a wireless communication protocol, in particular WiFi, or a power line communication protocol or a LiFi communication protocol.

In a preferred embodiment, the communication channel between the audioassistant device and the remote caregiver is a digital communication channel, where a digital signal comprising the audio data is transmitted via said communication channel. In an alternative embodiment, the communication channel is an analog communication channel.

In an embodiment, the second wireless communication device is configured, upon ascertaining the occurrence of a fall, to register the audio signal during time span of a predetermined duration and to provide the first audio data once the predetermined duration has lapsed. In this embodiment, the audio data is not provided in real time or quasi real time. Additionally, or alternatively, in another embodiment, the audio forwarding unit is configured to register the first audio signal and to provide upon ascertainment of the occurrence of the predefined physical event, to the first wireless communication device, audio data indicative of the audio signal registered during a first time-span previous to the ascertainment of the occurrence of the predefined physical event (e.g., fall, gesture, movement, vital sign indicative of a potential risk, etc.), fall event upon ascertainment of the fall event. For instance the microphone registers the audio in a continuous manner and, once the event, e.g., the fall or the predetermined gesture has been detected or data indicative thereof has been received, the audio-data forwarding unit provides audio data indicative of the audio signal registered during the predetermined time-span prior to the occurrence of the event. Thus, the registered sound is conveyed to the remote caregiver, where the situation can be better assessed by hearing what had happened prior to the occurrence of the event. Additionally, or alternatively, the audio forwarding unit is configured to register the first audio signal and to provide, upon ascertainment of the occurrence of the predefined physical event, to the first wireless communication device, audio data indicative of the audio signal registered during a second time-span after the ascertainment of the occurrence of the predefined physical event.

In an embodiment, the audio-data communication channel established between the first and the second wireless communication device can be a dedicated audio-data communication channel for the audio data or a shared communication channel where the transmission of audio data uses the same channel as other type of data provided by the wireless communication device.

Also preferably, in another embodiment the first, wireless communication device and the second wireless communication device are wirelessly controllable lighting devices of a wirelessly controllable lighting arrangement. In particular, the first and the second wireless communication devices are lighting devices or luminaires that can be controlled wirelessly, via a router, hub, bridge or any other suitable control device that provides wireless signals comprising instructions or status data for operating the lighting arrangement. A non-limiting example of such a lighting arrangement is the Philips Hue system or the Wavelinx system used in an elder care home. The first and/or second wireless communication devices can also be other devices of the lighting arrangement other than lighting devices, such as, for instance, associated sensors and/or switches.

The inventors have recognized that the density of wireless communication devices, e.g. WiFi-equiped devices, in a lighting arrangement is much typically much higher than the density of voice-assistance devices. Thus, an exemplary event-triggered audioforwarding arrangement comprising a wirelessly controllable lighting arrangement uses a first luminaire having a microphone and located proximate to the suspected fall location, which transmits an audio stream as audio data via WiFi radio signals to a second luminaire that includes a speaker and that is close to the voice-assistance device. This second luminaire plays the audio stream via its speaker, and the audio is captured by the voice-assistant device. Thus, the remote caregiver, for example a worker of an external call center or monitoring station, or a family member or acquaintance of the user will be aware of the situation of the person that has fallen as well is able to communicate with the person.

It is worth noting than adding a wireless communication device, such as a WiFi lighting device, the basic capabilities for recording and relaying an audio signal in the form of audio data between two devices and replaying it by the receiving device is significantly less costly and computation intensive than integrating a full voice-assistance functionality, including speech recognition, into each wireless communication device. The wireless communication devices of the arrangement, in particular the wirelessly controllable lighting devices do not need to comprehend the meaning of the audio message or even recognize individual words. In addition, the wireless communication device may have memory to buffer audio data. The stored audio data is for instance overwritten in the memory 20 seconds after no fall has been detected.

In an embodiment, the event-triggered audio-forwarding arrangement includes a fall-detection functionality, in particular a radiofrequency sensing functionality that uses signal parameters such as channel state information (CSI) or received signal strength indicator (RSSI) to infer location or activity of a subject within a sensing volume. For instance, an embodiment of the event-triggered audio-forwarding arrangement that is based on Zigbee sensing typically uses RSSI, since CSI data is not made available. An embodiment of the arrangement using a WiFi chip may use both CSI and RSSI data. The sensed activity may include a fall, which may be included into one of the following fall categories:

Trip and fall: this is a fast fall from the walking position to the ground. Fall when entering a chair and exiting a chair (static chair or wheelchair). Hence, this is a fast fall from the sitting position to the ground.

Soft fall: Person manages to grab furniture to (somewhat) slow the fall to the floor.

Brain stroke fall: Slow fall from standing position first to one knee and subsequently down to the ground.

Picking-something-up-from-the-floor fall: Very prolonged fall (up to two minutes!) occurring when elderly person tries to pick something up from the floor and does not have the power to get up again and eventually slowly falls to the ground.

Other types of activities that can be sensed will be described below.

Thus, in an embodiment, the fall ascertainment unit determines or detects whether a fall has occurred, using the fall-detection functionality, in particular the radiofrequency based fall detection functionality. In an alternative embodiment, the event- triggered audio forwarding arrangement receives a fall-indication signal from an external device, which is received by the event-ascertaining unit, and which is indicative of a fall of a person.

In another embodiment the event-ascertaining unit is further configured to ascertain, in other words, to determine or to receive, event data indicating whether the event (e.g. fall, movement, gesture, vital sign indicative of a risk) has occurred within or outside the predetermined sound pick-up region of the microphone of the voice assistance device. The audio-data forwarding unit is in this embodiment configured to operate the microphone to register the first audio signal and to provide first audio data indicative thereof via the audio data communication channel upon ascertaining that the fall has occurred outside the sound pick-up region. Thus, if the fall location is deemed to be within the sound pick-up region of the voice-assistant device, the audio forwarding unit is not actuated for registering the audio signal and providing the corresponding audio data to the first wireless communication device. However, audio data related to the context preceding the fall event, for fall event itself as well the immediate aftermath of the fall (before the bidirectional communication with the call center has been established) may be forwarded by the communication device to the voice assistant device. The forwarding of the context of the fall may be performed after the elderly has been brought to the hospital (as this data is relevant for fall prevention in the future rather than organizing the acute care). When forwarding the audio data about the context of the fall to the voice assistant, the communication device may add pre-amble works to wake up the voice assistant and notify him that the audio data it will receive is historic recording of the fall event.

In an exemplary embodiment, the communication devices may continuously register the audio signal and generate audio data, but delete the audio data after a predetermined time, such as after 30 seconds if no predefined event (e.g., fall or gesture) has been detected. Similarly, right after an event has been detected, audio files may be recorded and stored e.g. before the communication channel, preferably a two-way communication channel, between the person and the remote caregiver has been established. Hence, in a further developed embodiment, the audio forwarding service can also be used to forward an historic audio stream based on the detection of the event, wherein the historic audio stream is related to the event (e.g., immediately prior to, at the same time, or after the event). Preferably, the forwarded audio stream is unprocessed by the lighting device i.e., there is no speech recognition. The audio stream is not necessarily part of a conversation between the remote caregiver and the user. For that a time period before the event is selected and audio data is forwarded for the selected time period to allow the caregiver to infer context (reason) of the event.

To infer context (e.g. the reason or cause) of the event (e.g., fall) from the historic audio data recorded by the wireless communication device, a time-span before the fall event is purposefully selected the audio data for the selected time-span is forwarded. This audio data will enable the caregiver to diagnose what has led to the fall before the two-way communication is set up (e.g., person getting up from bed without help) and what happened (e.g., person hitting the floor very hard and cursing after the fall from pain).

In an embodiment configured as a fall-triggered audio-forwarding arrangement and, for instance, using WiFi sensing data, the probabilities of each fall type are determined. For instance, for a first fall event, the fall detection algorithm may infer a highest probability for a trip & fall (e.g., over 90%). As for a trip & fall event, the context of the fall (i.e. activities of the elderly before the fall) is less relevant for preventing future fall events, a short time-span (e.g., 1-5 seconds) is selected, such that the transmitted audio data starts only 1 to 5 seconds before the fall. However, as a trip and fall is a hard fall event likely leading to significant pain and/or injuries, the audio data obtained after the fall for another predetermined time-span (e.g. 1-5 minutes) after the fall event is also transmitted. This enables the caregiver listen for prolonged expressions of pain of the person, talking, calling for help, cursing etc. It also helps the caregiver to determine how long it took the person to get up again after she or he for instance told the caregiver that she does not need any help. For a second fall the fall detection algorithm may infer a highest probability for a soft fall. For a soft fall event, the context of the fall (i.e. activities of the person before the fall) may or may not be relevant for preventing future fall events. A soft fall is defined as the person manages to grab furniture to (somewhat) slow the fall to the floor or onto the couch. The soft fall however can be caused by stumbling by an object on the floor (context is less relevant for the caregiver) or may be caused by the person becoming dizzy when tying the shoelaces (very relevant context for a fall prevention system). Hence, the transmitted audio file is selected to start a moderate time-span (e.g. one minute) before the fall. However, as soft falls likely will not lead to significant pain and/or injuries, we transmit only audio data indicative of a few (e.g., five to 10) seconds of audio data after the fall event. Alternatively, the transmission of the audio data may be adaptively stopped as soon as the person has gotten up again after the soft fall. As the person is most likely uninjured, there is little value of forwarding and monitoring the audio data after the person has gotten up again.

For a third fall the fall detection algorithm may infer a highest probability for a fall associated with entering a chair and exiting a chair (static chair or wheelchair). It is believed that 41% of all falls in the frail elderly population are caused by inappropriate sit-to- stand transfers. For chair falls, the context of the fall (i.e. activities of the elderly before the fall) are highly relevant for preventing future fall events. For instance, did the elderly fall of the chair because she had fallen asleep a few minutes before. It is hence important as context information whether the elderly fall because he or she was trying to get up after having sat for a very prolonged period on the chair, leading to hypotension (it is also known that one of the factors increasing the fall risk of an elderly is low blood pressure associated with a posture change of the elderly. This effect is called postural hypotension and occurs when your blood pressure drops when you go from lying down to sitting up, or from sitting to standing.)

For chair falls the transmitted audio file is selected to start a long time-span (e.g. 30 to 60 min) before the fall. As the chair fall may or may not result in injuries, transmitting the audio data may stop at a moderate time (e.g., 1 to 5 minutes, such as 2 minutes) after the fall. Preferably, the transmission of the audio data is not adaptively stopped as soon as the elder has gotten up again after the chair fall. As the elderly may be uninjured, we monitor for expressions of pain subsequent of the elder getting up again (e.g. audible pain when walking).

For a fourth fall the fall detection algorithm may infer a highest probability for a fall associated with a brain stroke. This is a slow fall from standing position first to one knee and subsequently down to the ground. As this fall type is uncorrelated with activities of the elderly performed before the fall, we hence select the transmitted audio file to start very shortly (e.g. 5 to 15 seconds, such as 10 seconds) before the fall. As the brain stroke falls is related to a medical emergency, the audio signal is registered and the audio data forwarded until the ambulance has arrived. Preferably, the interactions with the medical staff are also registered until the room is vacated.

For a fifth fall, the fall the fall detection algorithm may infer a highest probability for a fall associated with picking-something-up-from-the-floor. This fall type is a very prolonged fall (up to two minutes) occurring when the elderly person tries to pick something up from the floor and does not have the power to get up again and eventually slowly falls to the ground. For chair falls, the context of the fall (i.e. activities of the elderly before the fall) are highly relevant for preventing future fall events. For picking up something from the floor falls, we hence select the transmitted audio file of the fall event to start a considerable time-span (e.g. 5 to 15 minutes, such as 10 minutes) before the fall. This enables to deduce which activities the elderly was performing before the fall. As the chair fall may or may not result in injuries, the transmission of the audio data may step after a moderate timespan (e.g., 1 to 5 minutes) after the fall.

For a sixth fall the fall the fall detection algorithm may infer a highest probability for a fall associated with tying the shoelaces or a dementia patient playing for a prolonged period with something on the floor. For instance, it is known that dementia patients like to play with extension cords on the floor and when bending forward for longer time to reach the cables they get dizzy. The transmitted audio file (or audio data) is thus selected to start a moderate time-span (e.g., 2 to 10 minutes, such as five minutes) before the fall. As this fall type may or may not result in injuries, the transmission of the audio data may be stopped after a moderate time-span (e.g., 1 to 5 minutes, such as two minutes) after the fall.

In another embodiment, the event-triggered audio-forwarding arrangement of the first aspect of the invention additionally comprises a hearing-range inference unit that is configured to ascertain context information pertaining to a current state of an environment in which the event-triggered audio-forwarding arrangement is arranged and, using the ascertained context-information, to determine whether a given wireless communication device is within a current sound pick-up region of the voice-assistant device. This enables to infer a current sound pick-up region with respect to the location of the event, e.g., the suspected fall, and not necessarily with respect to the position of the second wireless communication device. Context-information pertains to the current state of the environment (e.g. the house or apartment or relevant sections thereof). For example, context-information may be indicative of whether doors are open or close, which influences the current sound pick-up region of the voice-assistant device. In another example, said context-information may be provided by a user, e.g., the user may identify which devices are in the sound pick-up region of the voice-assistant device, or the user may provide location data indicative of the location of the wireless devices in a space (e.g., a room) and said hearing-range inference unit may determine using said location data said sound pick-up region. Similarly, and using recent historical or real-time sensor data, it can be assessed whether the microphone of the second wireless communication device can register or hear sounds from the room where the first wireless communication device and/or the voice-assistant device are located. If yes, the event-triggered audio-forwarding arrangement concludes that the user is likely within the sound pick-up region of the voice-assistant device. In some cases, after the person has fallen, he or she may be able to speak only with a fainter voice than usual, and hence the voiceassistant device may no longer be within sound pick-up region of the fallen person, hence nevertheless requiring the event-triggered audio-forwarding functionality. The hearing-range inference unit may also comprise a radar unit or a time-of-flight sensing unit configured to ascertain at least part of the context-information. Typically it should be avoided that the first wireless communication device repeats the spoken words of the user when the user’s voice can be heard at the voice-assistant device directly, since this will lead to interferences. If some of the spoken audio signal from the user is still heard at the voice assistant, the event- triggered audio-forwarding arrangement can be advantageously configured to detect this fact and to provide to the first wireless communication device an instruction to increase the volume of the audio signal played by the loudspeaker to dominate over the direct path audio between user and voice-assistant device, so that the voice-assistant device will prioritize the audio signal from the wireless communication device over the fainter human originated voice.

Alternatively, or additionally, in another embodiment, the event-ascertainment unit is further configured to ascertain whether the event, such as a fall, has occurred within or outside a predefined sound pick-up region of the microphone of the voice-assistant device and the audio-data forwarding unit is configured to operate the microphone of the second wireless communication device to register the first audio signal and to provide the first audio data via the audio data communication channel only if event-ascertainment unit has indicated that the event has occurred outside the predefined sound pick-up region. Preferably, in another embodiment, the first wireless communication device further comprises a microphone and an audio-data forwarding unit for providing response audio data indicative of a respective response audio signal registered by its microphone. In addition, the second wireless communication device further comprises a loudspeaker configured to play a second response audio signal indicative of the received response audio data registered by, and received from, the first wireless communication device. In this embodiment, the event-triggered audio-forwarding arrangement is further configured, upon ascertainment of event data indicative of the predetermined physical event (fall, gesture, etc.), to enable a bidirectional audio data communication channel between the first wireless communication device and the second wireless communication device for exchanging audio data between them. In this manner, the call center or monitoring station is able to engage in a bi-directional conversation to verify with the fallen person if she or he is in good condition or needs help.

In an exemplary embodiment, that includes a plurality of second wireless communication devices in the vicinity of the location of the ascertained event, it may be advantageous to utilize one of the communication devices for registering the audio signal from the person and forwarding it to the first wireless communication device, while the other communication device is used for transmitting the response audio signal (coming from the caregiver via the voice-assistant device and the first wireless communication device) to the person lying on the floor. The second wireless communication devices, e.g. two lighting devices located in the room where the person has fallen, and that can be equipped with low cost microphones and speakers, this may reduce the interference between the loudspeaker and the microphone.

Similarly, this arrangement will distribute the computation load for the bidirectional communication between the first lighting device registering the audio signal and the second lighting device reproducing the response. This is advantageous as, typically, lighting devices are computing resource limited (especially if the downlight currently performs another task such as rendering a dynamic light effect or an over the air firmware update).

In yet another embodiment of the event-triggered audio-forwarding arrangement of the first aspect of the invention, the voice-assistant device is configured to enable or establish the communication channel for providing the second audio data to the remote caregiver upon recognition of a predetermined wake word or trigger word, i.e., a predetermined or pre-agreed word or phrase or command that, upon being recognized by the voice-assistant device activates the communication channel with the remote caregiver, call center or monitoring station. In this particular embodiment, the second wireless communication device comprises a word recognition unit configured to recognize the predetermined wake word, and the audio-data forwarding unit is configured to enable or establish the audio-data communication channel further upon recognition of the predetermined wake word by the word recognition unit. The recognition of the predetermined or pre-agreed wake word is performed, in an embodiment, by an edge-AI audio analysis unit comprised by the second wireless communication device. In an embodiment, the audioforwarding functionality for providing the registered audio data to the first wireless communication device is activated in dependence on a reaction or lack of reaction to a prerecorded audio messaged played by the second wireless communication device upon ascertaining fall data indicative of a fall. For instance, in an embodiment, the audioforwarding functionality is activated when the user fails to respond, in a pre-agreed manner, to a pre-recorded audio message. In another embodiment, the audio-forwarding functionality is activated when the user positively responds with a pre-agreed wake word or phrase to a pre-recorded audio message pertaining to whether the user needs assistance. The pre-agreed wake words can be easily recognized using light-weight edge-AI algorithms, which fit on the common commodity microcontrollers use in wireless communication devices such as WiFi lighting devices.

For example, in an embodiment the response audio signal to be transmitted is enabled before the voice-assistant device has picked up the wake word. This allows the caregiver asking the person about the current situation, but without channel from the second wireless communication channel to the remote caregiver (via the first wireless communication device and the voice-assistant device) being activated until the user has spoken the wake word.

In an embodiment, a second wake word indicative of the user not wanting to initiate the audio channel is introduced. While the system may auto-activate the audio-data communication channel to first wireless communication device and then to the remote caregiver after a predetermined waiting time (e.g. two minutes) without receiving the wake word (as the user may have forgotten the wake word), it will not activate the audio-data communication channel if the user has spoken the second wake word indicating his unwillingness to transmit his audio data to the caregiver.

Optionally, in an embodiment, the second wireless communication device is configured to determine whether the user is unsuccessfully trying to contact the voice- assistant device. For instance if the sound level of the words uttered by the user is too low to be registered by the voice-assistant device.

In a particular embodiment, wherein the voice-assistant device is configured to enable or establish the communication channel for providing second audio data to the remote caregiver upon recognition of a predetermined wake word, the first wireless communication device, upon reception of the first audio data, is configured to provide, via its loudspeaker, a prerecorded wake-word audio signal that, when played, includes the predetermined wake word.

For instance, for the case that the user is, after the suspected fall event, unfortunately no longer able to talk, an embodiment of the event-triggered audio-forwarding arrangement of the invention is configured to register the faint audio indicative of the user being in pain and forward the audio data indicative thereof to the first wireless communication device. The first wireless communication device is advantageously configured to add a synthetic or pre-recorded wake word preamble to the audio data for activation of the voice-assistant device. In an embodiment, the first wireless communication device is configured to determine whether the audio coded in the audio data comprises the wake word, and to add the pre-recorded or synthetic wake word preamble in case it is not comprised.

In another embodiment, the event-triggered audio forwarding arrangement is also configured as a radio-frequency based event-detection arrangement. To this end, in the event-triggered audio-forwarding arrangement, the audio data communication channel between the first wireless communication device and the second wireless communication device is based on a wireless communication protocol such as WiFi, Bluetooth, Zigbee, Thread, LoRa or any other suitable wireless communication protocol. The arrangement further comprises an event or activity sensing unit configured to, using the wireless channel of the wirelessly transmitted audio data, to determine a wireless signal parameter correlated to an activity (event) of a subject within a sensing volume defined by the relative position of the first and the one or more second wireless communication device, and to provide activity data indicative thereof.

Thus, in this particular embodiment, the forwarding of the audio data, preferably the bi-directional forwarding (i.e. from the second to the first, but also from the first to the second wireless communication devices) in the form of radiofrequency signals is orchestrated such that the wireless signals sent for forwarding the audio data are also used for the RF-based sensing of the activity of the subject, e.g. for monitoring a position, movement, gestures or vital signs of the fallen user. The signal transmission parameters (frequency, directivity, signal strength, etc.) may be altered in order to better perform the activity sensing functionality.

In another embodiment, the voice-assistant device further comprises an eventascertaining unit configured to receive event data indicative of an physical event by a person, and, upon ascertaining, using the event data, the occurrence of a predetermined event, such as a fall, a predetermined gesture, vital signs indicative of a risk, the voice-assistant device is configured to initiate a communication channel with the remote caregiver or monitoring station. The event-triggered audio-forwarding arrangement is configured to forward audio messages reproduced by the voice-assistant device to the wireless communication device closest to the location of the fall to ensure that a lack of response from the user is not due to the user being outside the audio range of the voice-assistant device.

Non-limitative examples of predefined physical events and related interactions with the remote caregiver may include the following: a) a fall, for instance any of the fall types described above; b) a gait posture indicative of an increased risk of falling (the remote caregiver can then make use of an event-triggered audio-forwarding arrangement to communicate with the person, e.g., an elderly to try to minimize the fall risk e.g. asking the person to sit down); c) sitting up in the bed: A person, such as an elderly or a patient, who is not supposed to walk on his own, sits up in the bed during the night. This is indicative that the person might try to get up from the bed to go to the toilet during the night. The remote caregiver can then make use of an event-triggered audio-forwarding arrangement to communicate with the person and find out whether help or assistance is needed; d) A person is bent over for a prolonged time e.g. when tying the shoe laces. This is indicative that the person does not have power to get upright again (and hence a prolonged fall event is happening) The remote caregiver can then make use of an event-triggered audioforwarding arrangement to communicate with the person, e.g., to ask the elderly to remain in the bent posture as help is on the way): e) A person performing a forbidden or not recommended activity deemed risky for that person e.g. bending over to play with the cables on the floor, doing push-ups on the floor, self-lowering to the floor; f) A person physically interacting with another person e.g. physically fighting with another person, e.g. in an elderly home. g) A person is displaying abnormal vital signs, for example as recorded by a RF- based sensing arrangement from a distance. For instance, if high breathing rate is detected, the remote caregiver can then make use of an event-triggered audio-forwarding arrangement to communicate with the person and e.g., check whether the person is doing some physical exercise or is unwell. h) A person who has dementia or is otherwise incapacitated is joined by a second person. The remote caregiver can then make use of an event-triggered audio-forwarding arrangement to check-in remotely to monitor for possible abuse of the person. i) A person, e.g., an elderly, is sitting down on the toilet and remains there for a very long time. The remote caregiver can then make use of an event-triggered audioforwarding arrangement to communicate with the person and inquire what is going on. j) A person may exhibit a lack of physical movement e.g. sitting on a chair for a very long time. The caregiver can then make use of an event-triggered audio-forwarding arrangement to communicate to communicate with the person, e.g., and recommend not to get up (as this may result in a high risk of becoming lightheaded and the fall).

A second aspect of the present invention is formed by a wireless communication device for use in an event-triggered audio-forwarding arrangement in accordance with the first aspect, in particular as a second wireless communication device. The wireless communication device comprises a microphone, an event-ascertaining unit configured to ascertain an occurrence of a predefined physical event related to a person, such as a gesture, a fall, a movement, vital sign indicative of a risk situation, etc.,, and an audiodata forwarding unit configured to operate the microphone to register a first audio signal and to provide, upon ascertaining the event, first audio data indicative of the first audio signal to another wireless communication device via an audio-data communication channel.

Preferably, in an embodiment, the event-ascertaining unit comprises, or receives the event data from a radiofrequency -based event or activity-sensing arrangement that does not rely on an audio input to determine the occurrence of a fall, and thus respects the privacy of the conversations inside the environment.

In a preferred embodiment, the wireless communication device of the second aspect of the invention further comprises a loudspeaker configured to play an audio signal indicative of the received audio data, and wherein the event-triggered audio-forwarding arrangement is further configured to, upon ascertainment of the event, to enable or establish a bidirectional audio-data communication channel with the other wireless device for exchanging audio data between them. Also, in an embodiment, the wireless communication device is a wirelessly controllable lighting device, such as a luminaire or a lamp, whose operation can be controlled using wireless communication devices in accordance with a predetermined wireless communication protocol, such as, for example, WiFi, Bluetooth or Zigbee.

A third aspect of the present invention is formed by a method for operating a wireless communication device according to the second aspect. The method comprises: ascertaining an occurrence of a predefined physical event related to a person using an RF -based sensing arrangement; operating the microphone to register first audio data and upon ascertainment of the occurrence of the physical event related to the person (e.g., a fall, a movement, a gesture, vital sign indicative of a risk, etc.), providing first audio data indicative of the first audio signal to another wireless communication device.

The method of the third aspect thus shares the advantages of the wireless communication device of the second aspect of the present invention.

In a particular embodiment, the method further comprises: upon ascertainment of the physical event, enabling a bidirectional audio-data communication channel with the other wireless device for exchanging registered audio data between them; and playing a second audio signal indicative of the received first audio data using a loudspeaker.

A fourth aspect of the present invention is formed by a method for operating an event-triggered audio-forwarding arrangement according to the first aspect. The method comprises: performing the method for operating a wireless communication device of the third aspect for a second wireless communication device. As a result of this, the second wireless communication device provides first audio data to a first wireless communication device. The method further comprises; said first wireless communication device that receives the first audio data, playing a second audio signal indicative of the first audio data to be picked up by a microphone of a voice-assistant device; registering the second audio signal by the microphone of the voice-assistant device; establishing a communication channel between the voice-assistant device and a remote caregiver and providing second audio data indicative of the second audio signal to the remote caregiver via the communication channel.

The method for operating the event-triggered audio-forwarding arrangement in accordance with the invention shares the advantages of the event-triggered audio-forwarding arrangement of the first aspect.

Another aspect of the invention is formed by a respective computer program comprising instructions which, when executed by a processor of a wireless communication device in accordance with the second aspect, or of an event-triggered audio-forwarding arrangement in accordance with the first aspect, cause the wireless communication device or the event-triggered audio-forwarding arrangement to carry out the method of the third aspect or to carry out the method of the fourth aspect, respectively.

It shall be understood that the event-triggered audio-forwarding arrangement of claim 1, the wireless communication device of claim 10, the respective method of claims 12 and 14 and the computer program of claim 15, have similar and/or identical preferred embodiments, in particular, as defined in the dependent claims.

It shall be understood that a preferred embodiment of the present invention can also be any combination of the dependent claims or above embodiments with the respective independent claim.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following drawings:

Fig. 1 shows a schematic block diagram of an event-triggered audioforwarding arrangement in accordance with an embodiment of the invention, in particular implemented as a fall-triggered audio-forwarding arrangement;

Fig. 2 shows a schematic block diagram of an event-triggered audioforwarding arrangement in accordance with another embodiment of the invention, in particular implemented as a fall-triggered audio-forwarding arrangement;

Fig. 3 shows a schematic representation of an event-triggered audioforwarding arrangement in particular implemented as a fall-triggered audio-forwarding arrangement within a wirelessly controllable lighting arrangement in an apartment; Fig. 4 shows a schematic block diagram of a wireless communication device in accordance with an embodiment of the invention; and

Fig. 5 shows a flow diagram of a method for controlling operation of an event- triggered audio-forwarding arrangement in accordance with an embodiment of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS

Fig. 1 shows a schematic block diagram of an event-triggered audioforwarding arrangement 500. The event-triggered audio-forwarding arrangement 500, also referred to as the arrangement for the sake of simplicity, comprises a voice-assistant device 502. The voice-assistant device 503 includes a microphone 504 and, optionally, a loudspeaker 506. The voice-assistant device is configured to connect with an remote caregiver 600, e.g., an external call center, and to enable or establish a communication channel 602 with said remote caregiver 600. The communication channel is suitable for transmitting audio data AD2 indicative of an audio signal A2 registered by the microphone 504 in accordance with any suitable communication protocol, which may be a wireless communication protocols such as WiFi, Thread, Bluetooth, LiFi communication protocols, or a wired communication protocol, also including power line communication protocol. The voice-assistant device 502 is configured to generate the audio data AD2 using the registered audio signal A2 and to provide said audio data AD2 to the remote caregiver 600 via the communication channel 602.

The arrangement 500 also comprises a first wireless communication device 200 that includes a loudspeaker 202. The first wireless communication device 200 is arranged at a first location LI that is at a first distance DI from the voice-assistant device, and within a sound pick-up region 508 of the microphone 504 of the voice-assistant device 502. The sound pick-up region 508 is a volume in space where a sound of a given intensity can be registered with a given quality by the microphone 504 having a given sensitivity.

The arrangement 500 also includes one second wireless communication device 100. Other arrangements (see Fig. 3) may include a plurality of second wireless communication devices. The second wireless communication device 100 comprises a microphone 102 and is arranged at a second location L2 that is at a second distance D2 from the voice-assistant device, and outside the sound pick-up region 508 of the microphone 504 of the voice-assistant device 502, e.g. in another room. The first and the second locations that are at the first and second distances are such that a first sound intensity picked-up by the microphone of the voice-assistant device from a reference sound signal of a given intensity produced at the first location is higher than a second sound intensity picked-up by the microphone of the voice-assistant device from the reference sound signal of the same given intensity produced at the second location.

For instance, the voice-assistant device and the first wireless communication device are located in a bedroom 603, whereas the second wireless communication device is arranged in a bathroom 604, separated from the bedroom 603 by a wall 605. The second wireless communication device includes an event-ascertaining unit 104 that is configured to ascertain, i.e., to receive or to determine, event data FD indicative of a physical event related to a person 101. A physical event includes, for instance, a fall, a gesture, a predetermined movement, vital signs indicative of a potential risk, etc. Preferably, the ascertainment of the event data is done using radiofrequency based sensing techniques, in which a given signal quality parameter (e.g., RSSI, CSI, etc.) is correlated to a current state of a person in a sensing volume. The second wireless communication device also includes an audio-data forwarding unit 106 that is configured, upon ascertainment of event data FD, for example fall data FD indicative of a fall, to operate the microphone 102 to register a first audio signal Al, and to provide first audio data ADI indicative of the first audio signal to the first wireless communication device 200 via an audio-data communication channel 108.

In the arrangement 500, the first wireless communication device is configured, upon reception of the first audio data ADI, to operate the loudspeaker 202 to play a second audio signal A2 indicative of the first audio data ADI within the predetermined sound pickup region of the voice-assistant device such that the audio signal A2 is picked up or registered by the microphone 504 of the voice-assistant device. The voice-assistant device 502 is further configured to generate and provide second audio data AD2 indicative of the second audio signal A2 to the remote caregiver 600 via the communication channel 602.

Optionally, wherein the event-ascertaining unit 104 is further configured to ascertain whether the predefined physical event related to the subject 101 has occurred within or outside the predetermined sound pick-up region 508 and the audio-data forwarding unit is configured to operate the microphone to register the first audio signal and to enable the audio data communication channel upon ascertaining that the fall has occurred outside the sound pick-up region. Thus, if the fall has occurred within the sound pick-up region of the voice assistance device, the audio forwarding functionality is not activated.

Additionally, or alternatively, the arrangement 500 comprises a hearing-range inference unit, which in Fig. 1 is exemplary depicted as a unit 110 of the second wireless communication device. The hearing-range inference unit is configured to ascertain context information pertaining to a current state of an environment in which the event-triggered audio-forwarding arrangement is arranged and, using the ascertained context-information, to determine whether a given wireless communication device is within a current sound pick-up region of the voice-assistant device. This particular embodiment enables to infer a current sound pick-up region with respect to the location of the suspected fall and not necessarily with respect to the position of the second wireless communication device. Contextinformation pertains to the current state of the environment (e.g. the house or apartment or relevant sections thereof). For example, context-information may be indicative of whether doors in the wall 605 between the room 603 where the voice-assistant device is arranged and the room 604 where the second wireless communication device 100 is open or close, which influences the current sound pick-up region of the voice-assistant device. In another example, the user may provide location data indicative of the location of the first and the second wireless devices (LI and L2) and the hearing-range inference unit may determine the sound pick-up region based on the location data. Similarly, and using recent historical or real-time sensor data, it can be assessed whether the microphone 102 of the second wireless communication device 100 can register or hear sounds from the room where the first wireless communication device and/or the voice-assistant device are located. For example, the first wireless device 200 may play a first test audio signal and the hearing-range inference unit may determine whether the first test audio signal has been picked up by the voice-assistant device. If yes, the event-triggered audio-forwarding arrangement concludes that the user is likely within the sound pick-up region of the voice-assistant device 502. The second wireless device 100 may further comprise a loudspeaker and play a second test audio signal. The hearing-range inference unit may determine whether the second test audio signal has been picked up by the voice-assistant device and if no, determine that the second wireless device 100 is outside the sound pick-up region of the voice assistant device. In some cases, after the person has fallen, he or she may be able to speak only with a fainter voice than usual, and hence the voice-assistant device may no longer be within sound pick-up region of the fallen person, hence nevertheless requiring the event-triggered audio-forwarding functionality. The hearing-range inference unit may also comprise a radar unit configured to ascertain at least part of the context-information.

Fig. 2 shows a schematic block diagram of an event-triggered audioforwarding arrangement 500a in accordance with an embodiment of the invention. The following discussion will focus on those features of the arrangement 500a that differ from those of the arrangement 500 of Fig. 1. The features that have an identical or similar functionality will be referred to using the same reference numbers used in Fig. 1. In the arrangement 500a of Fig. 2, the first wireless communication device 200a further comprises a microphone 203 and an audio-data forwarding unit 204 for generating and providing response audio data RAD2 indicative of a respective response audio signal RAI registered by the microphone 302. The second wireless communication device 100a further comprises a loudspeaker 103 configured to play a second response audio signal RA2 indicative of the received response audio data RAD2. The event-triggered audio-forwarding arrangement is further configured, upon ascertainment of event data indicative of a predefined physical event FD, to enable a bidirectional audio data communication channel 108a between the first wireless communication device 200a and the second wireless communication 100a device for exchanging audio data between them. Thus, for example, in the arrangement 500a, when a user falls in room 604, the a event-ascertaining unit 104 of the second wireless communication device 100a ascertains (e.g., receives or determines) fall data FD indicative of said fall. Upon having ascertained the occurrence of a fall, the microphone 102 registers an audio signal Al, in particular from the fallen user, which is then converted into first audio data ADI and sent to the first wireless communication device via an audio-data communication channel 108a, in particular a wireless channel, such as, for example a WiFi communication channel. The first audio data ADI is received by the first wireless communication device 200a, which then plays the audio data using the loudspeaker 202. The played audio data is referred to as the second audio signal A2, which is correlated to the first audio signal. The second audio signal A2 is played within the sound pick-up region 508 of the voice-assistant device 502, which provides then second audio data AD2 indicative of the second audio signal A2 to the remote caregiver 600 via the communication channel 602a. The operation described up to now is also performed by the arrangement 100 of Fig 1. However, the in the arrangement 500a of Fig. 2, the communication channel between the voice-assistant device and the communication channel 602a and the audio-data communication channel 108a are bidirectional communication channels, that allow the remote caregiver (e.g., a call center, or a monitoring station) to establish a conversation with the user in room 604. To this end, and as a response to the second audio data AD2, the remote caregiver provides response audio data RADI, for example, indicative of an inquiry about the state of the user. A response audio signal RAI indicative thereof is reproduced by the loudspeaker 506 of the voice-assistant device 502. This audio signal including the response from the call center is registered by the microphone 203 of the first wireless communication device 100a, which generates corresponding second response audio data RAD2 that is provided by the audio-forwarding unit 204, for instance via a WiFi radio link, to the second wireless communication device 100a. This second wireless communication device 100a outputs, via an integrated loudspeaker 103 a second response audio signal RA2, thus enabling a bidirectional communication between the user and the remote caregiver, that allows the caregiver to assess the situation of the fallen user 101.

Optionally, as indicated by a dashed line, the second wireless communication device 100a further comprises a word recognition unit 107 that is configured to recognize a predetermined wake word, and wherein the audio-data forwarding unit 106 is configured to enable the audio data communication channel, and therefore, to provide the first audio data ADI, further upon recognition of the predetermined wake word by the word recognition unit 107. This is especially advantageous for an arrangement that includes a voice-assistant device that is configured to enable the communication channel with the remote caregiver upon recognition of a predetermined wake word, such as, for example an Amazon Echo device.

Fig. 3 shows a schematic representation of an event-triggered audioforwarding arrangement 500 implemented within an exemplarily wirelessly controllable lighting arrangement 550 in an apartment 650. The apartment has a wirelessly controllable lighting arrangement 550 including a plurality of wirelessly controllable lighting devices 100a and, in this particular example, a bridge 150. Regarding the wirelessly controllable lighting devices 100a, a corridor 651 has two luminaires L, a first bedroom 652 and a second bedroom 653 both include a luminaire L and two bed side lamps B respectively, a living room 654 includes two luminaries L and a torchiere floor lamp T, a kitchen 655 includes a luminaire L, a storage room or pantry 656 includes a luminaire L and a bathroom 657 includes a luminaire L and a bath mirror lamp M. These lighting devices are controlled using wireless communication signals exchanged between the lighting devices and the bridge, using a suitable wireless communication protocol, such as, but not limited to WiFi or Zigbee. Additionally, the wireless communication signal are used to determine an activity of a subject within the apartment, in particular by performing a known radiofrequency based sensing function, that is advantageously used for determining whether a physical event related to a person, such as a fall, has occurred in any of the rooms. The event-triggered audioforwarding arrangement 500, in addition to the lighting devices, configured as the required wireless communication devices 100a, comprises two voice-assistant devices 502 that are placed in the main bedroom 653 and in the living room 654. For the sake of simplicity, a predetermined sound pick-up region of the respective voice-assistant device is initially limited to its room. The lighting devices 200a closest to the respective voice-assistant devices 502 are considered to be the first lighting devices in accordance with the invention. It is thus clear that the density of wirelessly controllable lighting devices 100a in the apartment 650 is much higher than the density of voice-assistant devices.

In this particular arrangement 500 of Fig. 3, the voice-assistant devices are configured to enable the communication channel with the remote caregiver upon recognition of a predetermined wake word, for example “activate”. If a fall is detected in the living room 654 or in the main bedroom 653 the arrangement considers that the fall has occurred within the sound pick-up region of a voice-assistant device 502 and no further action is taken. However, if a fall happens in another room, such as in the pantry 656, as indicated by the cross in Fig. 3, the luminaire in the pantry ascertains the fall data indicative thereof and activates the audio-forwarding functionality for registering, using a microphone in the lighting device 100a an audio signal Al within the pantry. In this audio signal, the user pronounces the wake word “activate” and states that there has been an accident and needs assistance. The audio-data forwarding unit in the luminaires generates first audio data ADI indicative of the registered audio signal Al and provides said first audio data ADI to one or both of the lighting devices 200a located one in each of the main bedroom 653 and the living room. In this case, the audio data is provided to the voice-assistant device in the living room. Depending on the arrangement, the provision can be performed directly or via the bridge 150. Both possibilities are shown in Fig. 3. One or both of the lighting devices 200a generate, using a loudspeaker, a second audio signal A2 reproducing the first audio signal, i.e., the wake word “activate” and the statement about the need for help. This second audio signal is played within the sound pick-up region of a voice assistance device 502. The microphone of the voice-assistant device picks the wake word “activate” and enables a communication channel with the remote caregiver for providing second audio data indicate of the request for assistance.

In a variant of the arrangement, the event-triggered audio-forwarding arrangement 500 further comprises a hearing-range inference unit that is configured to ascertain context information pertaining to a current state of an environment in which the event-triggered audio-forwarding arrangement is arranged and, using the ascertained contextinformation, to determine whether a given wireless communication device is within a current sound pick-up region of the voice-assistant device. For instance, if the fall has occurred in the kitchen 655 and the door 658 between the kitchen 655 and the living room 654 is open, the hearing-range inference unit is configured to ascertain context information indicative of the state of the door and of the historical capacity of the voice-assistant device in the living room 654 of hearing audio signals originated in the kitchen 655 when the door is open to infer that, under current circumstances, the voice-assistant device is capable of registering an audio signal originated in the kitchen.

In a preferred arrangement 500, the first wireless communication devices 200a further comprises a microphone and an audio-data forwarding unit for providing response audio data indicative of a respective response audio signal registered by its microphone 203, and the second wireless communication devices 100a further comprise a loudspeaker configured to play a second response audio signal (RA2) indicative of the received response audio data. Here, the event-triggered audio-forwarding arrangement 500 is further configured, upon ascertainment of fall data indicative of a fall, to enable a bidirectional audio data communication channel between the first wireless communication device and the second wireless communication device for exchanging audio data between them. Thus, a response to the assistance request by the fallen user by the monitoring center is played by the voiceassistant device, picked up by the nearest wireless communication device 200a, forwarded to the lighting device closest to the fallen user, and reproduced there to make sure that the user is aware that the message has been received, to gather further information or to confirm that assistance is on its way.

Fig. 4 shows a schematic block diagram of a wireless communication device 100 according to an embodiment of the invention, configured as a wirelessly controllable lighting arrangement. Operation of a lighting unit 105 is controlled wirelessly by means of wireless communication signals in accordance with a predetermined wireless communication protocol. The signals are provided and received via a dedicated antenna 107 that is signally connected to a processing unit 140 of the lighting device. The lighting device also comprises an event-ascertaining unit 104 that is configured to, using the wireless communication signals received, determine fall data FD indicative of a fall of a person. Upon determining that a fall has occurred, a microphone 102 registers first audio signal Al and the registered audio signal is sampled and transformed into audio data at the audio-data forwarding unit 106, that is provided as part of a wireless communication signal via the antenna, for reception by another wireless communication device.

Optionally, the wireless communication device 100 od Fig. 4 comprises a hearing-range inference unit 110 that is configured to ascertain context information pertaining to a current state of an environment in which the event-triggered audio-forwarding arrangement is arranged and, using the ascertained context-information, to determine whether a given wireless communication device is within a current sound pick-up region of the voice- assistant device. To infer the current sound pick-up region of the voice-assistant device with respect to the location of the suspected fall, the lighting system utilizes context information about the current state of the apartment; for instance, a radar sensor I l l is configured to determine whether the doors between the location of the fall and the room where the voiceassistance device is located is currently open or closed. Similarly, and for example based on recent historical or real-time sensor data, it is assessed if the microphone of the lighting device closest to the location of the fall can register the ambient noise or from the room where the voice-assistant device is located. If yes, the event-triggered audio-forwarding arrangement concludes that the fallen person is likely within the sound pick-up region of the voice-assistant device. Please note that after an elderly has fallen, he may be able to only speak with a fainter voice than usual and hence the voice-assistant device kitchen may suddenly no longer be within audio range of the fallen person, hence nevertheless requiring the event-triggered audio-forwarding functionality.

Alternatively or additionally, the wireless communication device may also comprise a word recognition unit 112 that is configured to recognize a predetermined wake word or phrase. In this particular example, the audio-data forwarding unit is configured to enable the audio data communication channel for providing the audio data further upon recognition of the predetermined wake word by the word recognition unit 112.

The wireless communication device 100 of Fig. 4 also comprises a loudspeaker 103 configured to play a second response audio signal indicative of a received response audio data via the antenna. The wireless communication device 100 is further configured, upon ascertainment of fall data indicative of a fall, to enable a bidirectional audio data communication channel between both wireless communication devices for exchanging audio data between them.

Preferably, all the wireless communication devices in an exemplary event- triggered audio-forwarding arrangement are configured as the wireless communication device 100 of Fig. 4.

Fig. 5 shows a flow diagram of a method 800 for operating an event-triggered audio-forwarding arrangement according to an embodiment of the invention. The method 800 comprises, in a step 702, ascertaining fall data indicative of a fall of a person using an RF- based sensing arrangement. Upon ascertainment of fall data indicative of a fall, the method includes, in a step 704 operating the microphone to register a first audio signal and, in a step 706, providing first audio data indicative of the first audio signal to another wireless communication device via an audio-data communication channel. These steps form also a method 700 for operating a wireless communication device of an event-triggered audioforwarding arrangement. This method 700 may also comprise, in a step 705, enabling a bidirectional audio data communication channel with the other wireless device for exchanging audio data between them and, in response to receiving audio data from another device, playing a second response audio signal indicative of the received response audio data using a loudspeaker.

The method 800 further comprises, in a step 802, playing, by another wireless communication device that has received the audio data, a second audio signal indicative of the first audio data within the sound pick-up region of a voice-assistant device. The method further comprises, in a step 804, registering the second audio signal by a microphone of the voice-assistant device. The method then comprises, in a step 806, enabling or establishing a communication channel of a voice-assistant device with the remote caregiver and, in a step 808, providing second audio data indicative of the second audio signal to the remote caregiver via the communication channel.

Thus, according to the invention, and thanks to the forwarding of the audio data between the second and the first wireless communication device, where the first communication device replays the audio data so that the message can be picked by the microphone of the voice-assistant device located far from the fallen person, the call center will be able to hear the fallen person, and preferably also to engage with him or her in a bidirectional conversation, for instance to verify whether the person is fine or needs assistance. The essence of the invention is that the wireless communication devices do not need to be connected to the call center directly, as they only have to pass the audio data to a dedicated gateway, i.e. the voice-assistant device.

In another exemplary arrangement, the call center may initiate the audioforwarding functionality. The fall data indicative of a fall event can be directed to the call center. Upon receiving said notice, the call center is configured to initiate a conversation with the person to check whether the person is fine or needs assistance. If the person is detected outside the range of the voice-voice assistant device, the audio signal provided by the loudspeaker of the voice-assistant device is picked or registered by the microphone of the wireless communication device located closest to the voice-assistant device, and audio data indicative thereof is provided, via a wireless transmission, to a wireless communication device closer to the fallen person, which then plays an audio signal indicative of the received audio data via a loudspeaker. In summary, the invention is directed to an event-triggered audio-forwarding arrangement comprising a voice-assistant device configured to connect with a remote caregiver. The arrangement comprises a first wireless communication device with a loudspeaker arranged at a first location at a first distance from the microphone of the voiceassistant device a second wireless communication device with a microphone arranged at a second location at a second distance, and including an event-ascertaining unit configured to ascertain an occurrence of a physical event related to a person and an audio-data forwarding unit configured to register a first audio signal and to provide first audio data to the first wireless communication device, which is configured to play a second audio signal indicative of the first audio data to be picked-up by the voices-assistance device, thereby increasing its effective spatial hearing-range.

Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality.

A single unit or device may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Procedures like the processing of the detection signals for determining a fall, the authentication, the control of the fall monitoring system, et cetera, performed by one or several units or devices can be performed by any other number of units or devices. These procedures, particularly the control of the fall monitoring system in accordance with the control method carried out by the event-triggered audio-forwarding arrangement, can be implemented as program code means of a computer program and/or as dedicated hardware.

A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:

1. Event-triggered audio-forwarding arrangement (500) for conditionally forwarding audio received at a second wireless communication device, outside a sound pickup region of a voice assistant device, to a first wireless communication device for rendering said audio within said sound pick-up region; said arrangement comprising: said first wireless communication device (200) comprising a loudspeaker (202), the first wireless communication device (200) arranged at a first distance (DI) from a microphone (504) of the voice-assistant device (502) and within said sound pick-up region of the microphone of the voice-assistant device (502); and said second wireless communication device (100) comprising a microphone (102), the second wireless communication device arranged at a second distance from the microphone of the voice-assistant device and outside said sound pick-up region, the second wireless communication device further comprising:

- an event-ascertaining unit (104) configured to ascertain an occurrence of a predefined physical event related to a person (101); and

- an audio-data forwarding unit (106) that is configured to operate the microphone (102) to register a first audio signal (Al) and to provide, upon ascertainment of the physical event, first audio data (ADI) indicative of the first audio signal, to the first wireless communication device (200) via an audio-data communication channel (108); and wherein

- the first wireless communication device is configured, upon reception of the first audio data (ADI), to operate the loudspeaker (202) to play a second audio signal (A2) indicative of the first audio data (ADI) to be picked-up by the microphone of the voiceassistant device.

2. The event-triggered audio-forwarding arrangement of claim 1, wherein the event-ascertaining unit is further configured to ascertain whether the event has occurred within or outside the predefined sound pick-up region of the microphone of the voiceassistant device; and wherein the audio-data forwarding unit is further configured to operate the microphone of the second wireless communication device to register the first audio signal and to provide the first audio data via the audio data communication channel only if the event-ascertainment unit has indicated that the event has occurred outside the predefined sound pick-up region.

3. The event-triggered audio forwarding arrangement of claim 1 or 2, wherein the audio forwarding unit is configured to register the first audio signal and to provide, upon ascertainment of the occurrence of the predefined physical event, to the first wireless communication device, audio data indicative of the audio signal registered during a first timespan previous to the ascertainment of the occurrence of the predefined physical event and/or during a second time-span after the ascertainment of the occurrence of the predefined physical event.

4. The event-triggered audio-forwarding arrangement of any of the preceding claims, further comprising a hearing-range inference unit (110) that is configured to ascertain context information pertaining to a current state of an environment in which the event- triggered audio-forwarding arrangement is arranged and, using the ascertained contextinformation, to determine whether a given wireless communication device is within a current sound pick-up region of the microphone of the voice-assistant device.

5. The event-triggered audio-forwarding arrangement (500a) of any of the preceding claims, wherein the first wireless communication device (200a) further comprises a microphone and an audio-data forwarding unit for providing response audio data (RAD2) indicative of a respective response audio signal (RAI) registered by the microphone 203 of the first wireless communication device, and wherein the second wireless communication device further comprises a loudspeaker configured to play a second response audio signal (RA2) indicative of the received response audio data (RAD2), the event-triggered audioforwarding arrangement being further configured, upon ascertainment of an occurrence of the predefined physical event, to enable a bidirectional audio data communication channel (108a) between the first wireless communication device (200a) and the second wireless communication device (100a) for exchanging audio data (ADI, RAD2) between them.

6. The event-triggered audio-forwarding arrangement (500) of any of the preceding claims further comprising a voice assistant device (502) configured to connect with a remote caregiver (600) via a communication channel (602), wherein the voice assistant device is configured to provide second audio data (AD2) indicative of the second audio signal (A2) to the remote caregiver via the communication channel, and wherein the voiceassistant device is configured to establish the communication channel for providing the second audio data to the remote caregiver upon recognition of a predetermined wake word and wherein the second wireless communication device (100) comprises a word recognition unit (112) configured to recognize the predetermined wake word, and wherein the audio-data forwarding unit is configured to enable the audio-data communication channel further upon recognition of the predetermined wake word by the word recognition unit.

7. The event-triggered audio-forwarding arrangement of any of the preceding claims further comprising a voice assistant device (502) configured to connect with a remote caregiver (600) via a communication channel (602) wherein the voice assistant device is configured to provide second audio data (AD2) indicative of the second audio signal (A2) to the remote caregiver via the communication channel, and wherein the voice-assistant device is configured to establish the communication channel for providing the second audio data to the remote caregiver upon recognition of a predetermined wake word, and wherein the first wireless communication device, upon reception of the first audio data, is configured to provide, via its loudspeaker, a prerecorded wake-word audio signal that, when played, includes the predetermined wake word.

8. The event-triggered audio-forwarding arrangement of any of the preceding claims, wherein the audio data communication channel between the first wireless communication device and the second wireless communication device is based on a wireless communication protocol or a power line communication protocol or a LiFi communication protocol.

9. The event-triggered audio-forwarding arrangement of any of the preceding claims, wherein the audio communication channel is based on a wireless communication protocol, the event-triggered audio-forwarding arrangement further comprising a radio- frequency-based activity sensing unit (140) configured to, using the wirelessly transmitted audio data (ADI, RAD2), to determine a signal parameter correlated to an activity of a subject within a sensing volume defined by the relative position of the first and the second wireless communication device, and to provide activity data indicative thereof.

10. Wireless communication device (100) for use in an event-triggered audioforwarding arrangement in accordance with any of the preceding claims, the wireless communication device comprising: a microphone; a event-ascertaining unit configured to ascertain an occurrence of a predefined physical event related to a person; and an audio-data forwarding unit configured to operate the microphone to register a first audio signal and to provide, upon ascertainment of the event, first audio data indicative of the first audio signal to another wireless communication device via an audio-data communication channel.

11. The wireless communication device of claim 10, further comprising a loudspeaker configured to play a second response audio signal (RA2) indicative of a response audio data (RAD2), and wherein the wireless communication device is further configured to, upon ascertainment of the event, to enable a bidirectional audio data communication channel with the other wireless communication device for exchanging audio data between them.

12. Method (700) for operating a wireless communication device (100) according to claim 10 or 11, the method comprising: ascertaining (702) an occurrence of a predefined physical event related to a person using an RF -based sensing arrangement; operating (704) the microphone to register a first audio signal, and, upon ascertainment of the physical event, providing (706) first audio data indicative of the first audio signal to another wireless communication device via an audio-data communication channel.

13. The method (700) of claim 12, further comprising: upon ascertainment of the physical event, enabling (705) a bidirectional audiodata communication channel with the other wireless device for exchanging audio data between them; and playing a second response audio signal (RA2) indicative of a received response audio data (RAD2) provided by another wireless communication device (200a) using a loudspeaker.

14. Method (800) for operating an event-triggered audio-forwarding arrangement according to any of the claims 1 to 9, the method comprising: performing the method (700) for operating a wireless communication device of claim 11; - the wireless communication device receiving the first audio data playing (802) a second audio signal indicative of the first audio data to be picked-up by a microphone of a voice-assistant device (502); registering (804) the second audio signal by the microphone of the voiceassistant device.

15. Computer program comprising instructions which, when executed by a processor of a wireless communication device or of an event-triggered audio-forwarding arrangement cause the wireless communication device or of an event-triggered audioforwarding arrangement to carry out the method of claims 12 and 13 or to carry out the method of claim 14 respectively.