WO2018109120A1

WO2018109120A1 - Children monitoring system

Info

Publication number: WO2018109120A1
Application number: PCT/EP2017/082918
Authority: WO
Inventors: Deepthi SHIVAKUMAR; Vinayakkumar UDIKERI; Viswaradh REDDY; Rajendra Singh Sisodia
Original assignee: Koninklijke Philips N.V.
Priority date: 2016-12-15
Filing date: 2017-12-14
Publication date: 2018-06-21

Abstract

According to an aspect of the invention, a child monitoring system (100) is provided which comprises a voice recorder (110) configured to detect audio signals in a vicinity of a child (10), an a source identifying unit (121) configured to identify and separate a first speech signal corresponding to the child and at least a second speech signal of a person speaking to the child (10) based on the detected audio signal, and a trust score determining unit (125) configured to determine a trust score associated to the person speaking to the child (10) based on the second speech signal. The child monitoring system also comprises a panic mode detector (160) configured to detect a panic mode of the child based on the first speech signal and the detected trust score.

Description

CHILDREN MONITORING SYSTEM

FIELD OF THE INVENTION

The invention relates to a children monitoring system as well as to a method of monitoring children. BACKGROUND OF THE INVENTION

Baby monitoring devices are well-known and provide an acoustic and/or video monitoring of a baby in order to detect the crying of a baby.

However, a panic detection for older children can be difficult as kids younger than 4 years may not be able to express their feelings via speaking. In situation where young children are threatened or feel threatened, their behaviour modification may be very subtle and can may only be observed by close relatives. If the children are outside their home the detection of this distress or panic may be very difficult for other people like teachers.

US 2006/0047187 Al discloses a method of tracking and monitoring the location, health, emotional state, position, environment, sound, appearance, and other characteristics of a subject to monitor where being of a person.

SUMMARY OF THE INVENTION

It is therefore an objective of the invention to provide a children monitoring system and a method of monitoring children to efficiently detect panic or distress of a children.

According to an aspect of the invention a child monitoring system is provided which comprises a voice recorder configured to detect audio signals in a vicinity of a child, a source identifying unit configured to identify and separate a first speech signal corresponding to the child and at least a second speech signal of a person speaking to the child based on the detected audio signal, and a trust score determining unit configured to determine a trust score associated to the person speaking to the child based on the second speech signal. The child monitoring system also comprises a panic mode detector configured to detect a panic mode of a child based on the first speech signal and the detected trust score.

According to a further aspect of the invention the child monitoring system is embodied as a wrist worn device.

According to a further aspect of the invention, the child monitoring system also comprises a location determining unit configured to detect a location of the child. The panic mode detector is further configured to detect a panic mode of a child based on the location of the child.

According to a further aspect of the invention, the child monitoring system also comprises a heart rate determining unit configured to determine a heart rate of a child. The panic mode detector is further configured to detect a panic mode of a child based on the heart rate of the child.

According to a further aspect of the invention an activity analyzing unit is configured to detect and analyze an activity of a child. The panic mode detector is further configured to detect a panic mode of a child based on the detected activity of a child.

The invention also relates to a child monitoring method. Audio signals in vicinity of a child are detected by a voice recorder. The detected audio signals are analyzed to identify and separate a first speech signal corresponding to the child and at least a second speech signal of a person speaking to the child based on the detected audio signal. A trust score associated to the person speaking to the child is determined based on the second speech signal. A panic mode of a child is detected based on the first speech signal and the trust score by means of a panic mode detector.

According to an aspect of the invention, a child monitoring computer program comprising program code means for causing a child monitoring system to carry out the child monitoring method is provided when the computer program is run on the child monitoring system.

According to the invention a children monitoring system is provided which monitors and detects the activity of the children and can associate the activities with the context. Children's activity can be recorded continuously to detect variations which may be used to determine whether the kids behaviour relate to panic or stress situations. Such a system could be used by parents or teachers to better understand panic situations or stress situations of children. Audio recordings are used to identify which people are talking to the children. Based on the audio recordings of children's proximity, a trust network can be created. Optionally a heart rate tracker can be provided to find changes in the heart rate pattern of the child during panic or distress situations. Typically panic or distress situations are associated with an elevated heart rate. Optionally location information can also be considered to detect any changes in children's behaviour.

According to a further aspect of the invention a trust network can be build either automatically based on profile builder or in a "assisted manner". In the "assisted manner", the trust network is periodically profiled and a score of proximity network is manually assigned by a trusted node.

For e.g., an audio profile unit can identify all the unique node (or persons) in the proximity network and store their audio input. The trust node can then look at each of the proximity network profile and assign trust score. This trust score then is used for the panic mode detector. For e.g. the audio unit find 5 people in the proximity network which are not assigned the score. The unit play back the audio of each unassigned unit and provide location information. Based on this information, the trust node (parent/teacher) can then assigned different trust score for each unsigned node.

In addition, the system can also provide a detailed analysis of proximity interaction and situation on a daily basis in a visualized manner so that higher trust node (parent/teacher) can determine how the trust network is evolving.

Moreover, the gesture and mimic of the children can be detected for example with motion detector units such as "Microsoft kinetic game controller" type devices to improve the determination whether the child is in distress or in panic.

It shall be understood that a preferred embodiment of the present invention can also be a combination of the dependent claims or above embodiments or aspects with respective independent claims.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment s) described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following drawings:

Fig. 1 shows a schematic block diagram of a child monitoring system according to an aspect of the invention, Fig. 2 shows a block diagram of a trust score determination according to an aspect of the invention,

Fig. 3 shows a block diagram of a trust score determination according to an aspect of the invention,

Fig. 4 shows a block diagram of a trust score determination according to an aspect of the invention,

Fig. 5 shows a block diagram of a trust score determination according to a further aspect of the invention, and

Fig. 6 shows a block diagram of a trust score determination according to a further aspect of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS

Fig. 1 shows a schematic block diagram of a child monitoring system according to an aspect of the invention. The child monitoring system 100 according to an aspect of the invention comprises a voice recorder 110, trust score determining unit 120, a location determining unit 130, a heart rate determining unit 140, an activity analyzing unit 150, a panic mode detector 160 and an alert unit 170. The voice recorder 1 10 comprises a microphone, which can be digitized. The voice recorder 1 10 can continuously record an audio signal around a child 10. The audio signal is analyzed in the trust score determining unit 120 to determine a speech pattern of a child, i.e. how often and to whom the child speaks. Furthermore, based on the detected audios signal all persons with whom the child is in contact are identified. Furthermore, a level of conversation pattern is detected.

Furthermore, a panic detection can be derived from this information.

The trust score determining unit 120 comprises a source identifier 121, an audio profiling unit 122, a proximity identifier 123, an audible activity detecting unit 124 and a trust score unit 125. The source identifier 121 is used to identify the child and the people who are speaking to the child. To separate the speech signals of these individual subjects, Blind Source Separation (BSS) based Independent Component Analysis (ICA) can be applied to identify unique speakers in the child proximity networks.

To create a trust network, an open-set speaker identification method can be employed to verify whether an input speech corresponds to the child proximity networks or not. In this technique, speech specific features are extracted on short-time basis for each of identified unique speakers. The features may include: prosody, pitch, formants and Mel- frequency cepstral coefficients. A speaker specific model is trained on the extracting features using Gaussian Mixture Modelling (GMM). To verify whether the spoken speech utterance belong to any of the trusted network speakers and to build the specific profiles of proxy network:

A pre- determined likelihood threshold is used to identify whether the speech utterance is in the trusted network speakers.

A best-match model is used to identify the test utterance produced by the speaker associated with speech.

The audible activity detecting unit 124 can be used to detect an activity of the child based on the detected audio signals.

The trust score determining unit 125 is used to determine a trust score based on the detected audio signals.

The location determining unit 130 comprises a location sensor 131 like a GPS sensor. Furthermore, the location determining unit 130 comprises a reference location unit 132 which is used to compare the location detected by the location sensor with a reference location like home, school etc.

The heart rate determining unit 140 comprises a heart rate sensor 141 which can be an optical heart rate sensor like a PPG sensor as well as a reference location unit 141.

The activity analyzing unit 150 comprises an activity analyzer 151 and a gesture and physical activity profiling unit 152. The activity analyzing unit 151 can be used to analyze the activity and/or the gestures of the user. In the profiling unit 152 the detected activity can be analyzed and profiled for example based on previously detected gestures and activity in particular when a panic is detected.

The panic mode detector 160 receives the output of the audible activity unit 124, the output of the location determining unit 130, the output of the heart rate

determining unit 140 and the output of the activity analyzing unit 150. In the panic mode detector where these signals are analyzed and a panic mode is detected. If a panic mode is detected by the panic mode detector 160 then this information is forwarded to an alerting unit 170 which can alert the parents or the teacher of the child. The heart rate determining unit 140 is particularly used to detect elevated heart rates of a child. In the panic mode detector 160 an elevated heart rate of a user is compared with the activity data from the activity analyzing unit in order to determine whether the elevated heart rate is due to a physical activity like running or not. If this is the case then the panic mode detector 160 will not detected panic. However, if this is not the case, i.e. if the child is not running or if the activity detected by the activity analyzing unit 150 is low, this can be an indication of a distress situation or a panic situation.

The trust network or trust score unit 125 is used to identify to whom the children are talking or interacting to. Furthermore, it is detected and analyzed how frequent a child is interacting with certain person. Furthermore, the detecting unit 125 is used to determine whether a behavior of a child changes in the presents of a specific individual.

The location information provided by the location determining unit 130 can also be used to detect whether a child is in a distress situation or panic situation when it is in a certain location like a certain position outdoor. If a correlation is found between a location and a distress situation or panic situation of a child such information can be stored. If later on a child is then close to the position where it has previously been in a distress situation or panic situation this can be notified via the alert unit 170 to prevent a distress situation or a panic situation.

According to an aspect of the invention the voice recorder 1 10, the position determining unit 130, the heart rate determining unit 140 and the activity analyzing unit

150 can be embodied in a wrist worn device (like a smart watch). Such a wrist worn device may optionally include the panic mode detector. Alternatively, the panic mode detector may be implemented on an external device. In such a case the information from the voice recorder 1 10, the trust score unit 120, the location information, the heart rate information and the activity information may be transmitted for a example wirelessly from the wrist warn device to an external device to perform the panic mode detection.

Fig. 2 shows a block diagram of a trust score determination according to an aspect of the invention. The audio signal detected by the voice recorder is analyzed in the trust score determining unit 120 to detect an audio signal as a speech signal from a child 10 as well as from other speakers 1 1-13. The source identifying unit 121 is used to determine who is speaking, i.e. the child 10 or any of the other speakers 1 1-13. The audio signals relating to the child as well as the speakers 1 1-13 can be analyzed and profiled with a period of time Dl, D2-DN. In other words, the audio signals from the child 10 as well as the other speakers 1 1-13 can be monitored over a period of several days Dl-DN.

Fig. 3 shows a block diagram of a trust score determination according to an aspect of the invention. In Fig. 3 a situation is described where the child 10 is at home and a speaker 1 1 is talking to the child at home. Therefore, the location information for example provided by the location determining unit 130 is considered in the trust score determining unit 125. Furthermore, as described with reference to Fig. 2, the speech pattern or audio signal of the child 10 as well as of the speakers 1 1-13 are analyzed and are also considered as part of the determining of the trust score in the trust score determining unit 125. In the situation as described in Fig. 3 the trust score is high as the location of the child is at home and the speech pattern of the speaker 1 1 is also considered trustworthy.

Fig. 4 shows a block diagram of a trust score determination according to an aspect of the invention. In the situation of Fig. 4 the child 10 is not at home but rather in a mall or in a crowed place. This information is also considered by the trust score determining unit 125. As the child is not at home and in a crowed place the trust score will be considered lower as in the situation in Fig. 3. Furthermore, the speech pattern or conversation pattern is different as in the situation of Fig. 3 and is much more erratic then in the situation of Fig. 3. Accordingly, the trust score determining unit 125 will determine that the trust score for the situation and for the speaker is low.

Fig. 5 shows a block diagram of a trust score determination according to a further aspect of the invention. In the situation of Fig. 5 the child 10 is at school and several speakers 1 1-13 are present. The position of the child can be determined for example by the location determining unit 130. In the determining unit 120 the speech of the child as well as other speakers 1 1 -13 is analyzed for example also during several days. In the situation of Fig. 5 the trust score determining unit 125 can determine that the trust score can vary. In particular regarding the speaker 13 the trust score can be low because of the speaking or conversation pattern of the speaker 13.

Fig. 6 shows a block diagram of a trust score determination according to a further aspect of the invention. In Fig. 6 the situation is described where the child is at home as determined by the location determining unit 130 while the speaker 1 1 has a low trust score. In such a case the trust score determining unit 125 can determine that the trust score of the situation is low. According to the invention a low trust score can be used by the system to a panic mode by the panic mode detector 160.

According to an aspect of the invention the child monitoring system can analyze the audio signal detected by the voice recorder 1 10 to determine conversation patterns. Furthermore, trust scores associated to persons which are talking to the child may also be considered in the panic mode detector 160. Furthermore, the heart rate information from the heart rate detecting unit 140 may also be used by the panic mode detector. In particular an elevated heart rate may be an indication of a distress situation or a panic situation.

According to a further aspect of the invention the behavior of the child can be analyzed to detect any variations in its behavior. For example, a base location based activity analyzes can be provided in order to find earlier indicators of a panic mode.

According to an aspect of the invention trust network of a child including trusted persons can be created automatically by the trust score determining unit 120.

In addition or alternatively the trust score of the trust network can be manually assigned by a trusted note namely a trusted person. The trust score determining unit 120 can identify the network nods namely the network nods (persons in the proximity network of the child). The audio input of these network nods can be stored. A trusted network nod (parent/teacher) can examine each proximity network profile and manually assign a trust score. This trust score can then be used by the panic mode detector for detecting a panic mode of the child.

If for example the voice recorder 1 10 detects five people in the proximity network which have not yet been assigned a score the trust score determining unit 120 can analyze the audio signal and can also receive location information from the location determining unit 130 to determine a trust score.

According to an aspect of the invention a detailed analyze of proximity interactions and situations on a daily base can be visualized such that a trust note

(parent/teacher) can determine if and how the trust network is evolving.

According to an aspect of the invention, if the child is at home a motion sensor like Kinetic sensor can be used to track gesture or motions of the user. This can be further used to build a pattern and to determine a trust score. Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:

1. Child monitoring system (100), comprising:

a voice recorder (1 10) configured to detect audio signals in a vicinity of a child (10),

a source identifying unit (121) configured to identify and separate a first speech signal corresponding to the child and at least a second speech signal of a person speaking to the child (10) based on the detected audio signal,

a trust score determining unit (125) configured to determine a trust score associated to the person speaking to the child (10) based on the second speech signal, and a panic mode detector (160) configured to detect a panic mode of the child (10) based on the first speech signal and the determined trust score.

2. Child monitoring system (100) according to claim 1, comprising a location determining unit (130) configured to detect a location of the child (10), wherein the panic mode detector (160) is further configured to detect a panic mode of the child (10) based on the location of the child (10).

3. Child monitoring system (100) according to claim 1 or 2, comprising a heart rate determining unit (140) configured to determine a heart rate of the child (10), wherein the panic mode detector (160) is further configured to detect a panic mode of the child (10) based on the heart rate of the child (10).

4. Child monitoring system (100) according to claim 1, wherein the child monitoring system is embodied as a wrist worn device.

5. Child monitoring system according to claim 1, further comprising an activity analyzing unit (150) configured to detect and analyze an activity of the child (10), wherein the panic mode detector (160) is further configured to detect a panic mode of the child (10) based on the detected activity of the child (10).

6. Child monitoring method, comprising the steps of

detecting audio signals in a vicinity of a child (10) by a voice recorder (1 10), analyzing the detected audio signals, so as to identify and separate a first speech signal corresponding to the child and at least a second speech signal of a person speaking to the child (10) based on the detected audio signal,

determining a trust score associated to the person speaking to the child (10) based on the second speech signal, and

detecting a panic mode of the child (10) based on the first speech signal and the determined trust score by a panic mode detector (160).

7. A child monitoring computer program comprising program code means for causing a child monitoring system according to claim 1 to carry out the child monitoring method of claim 6 when the computer program is run on the child monitoring system.