WO2020218738A1

WO2020218738A1 - Method and system to calibrate electronic devices based on a region of interest

Info

Publication number: WO2020218738A1
Application number: PCT/KR2020/003619
Authority: WO
Inventors: Vijayanand KUMAR; Aditi PAWAR; Anand DESHBHRATAR; Divanshu KHUNGAR; Oshin GUPTA; Ravi ArunKumar JAIN; Vidushi Chaudhary
Original assignee: Samsung Electronics Co., Ltd.
Priority date: 2019-04-24
Filing date: 2020-03-17
Publication date: 2020-10-29

Abstract

Embodiments herein disclose a method for calibrating an electronic device based on a Region of Interest (ROI). The method includes detecting a plurality of events and rotating the electronic device in a direction of a first event from the plurality of events. The method further includes determining whether the ROI is available in the direction of the first event. The method further includes calibrating the electronic device towards the ROI when the ROI is available in the direction of the first event. The method further includes determining that the ROI is available in the direction of at least one second event and calibrating the electronic device towards the ROI.

Description

METHOD AND SYSTEM TO CALIBRATE ELECTRONIC DEVICES BASED ON A REGION OF INTEREST

The present disclosure relates to device calibration and more specifically to a method and system for calibrating electronic devices based on a region of interest (ROI) and detecting high intensity snore for sleep analysis.

The audio based approach for sleep analysis enables systematic detection and analysis of breathing and snoring sounds from a full night recording. Therefore, it is quite important to capture audio with high intensity which is free from any background noise. Conventional solutions for sleep analysis are based on whole night audio recording and produce good result in silent environment, which is not the case for a real life scenario.

Following problems are faced in conventional method and systems of sleep analysis. The device's microphone capturing the snoring sound of a user may be facing away from the user. The background noise is also processed due to appliances present in the room. Further, capturing the snoring of the required user is a problem if multiple people are sleeping in the same room.

Further the conventional systems uses camera for monitoring the user which raises concern for privacy. Obtaining maximum accuracy and identifying the intended area of interest in such cases is a problem.

Further it is very important to record clear and high intensity audio with minimum background noise. Further, for multiple people talking on the same side of the call, capturing the intended speaker's voice is difficult with conventional system. Further there is a need to reduce the requirements for multiple sensors and also minimizing the need for multiple microphones for noise reduction from an audio signal.

Thus, it is desired to address the above mentioned disadvantages or other shortcomings or at least provide a useful alternative.

Embodiments herein disclose a method for calibrating an electronic device based on a ROI. The method includes detecting, by an electronic device, a plurality of events and rotating the electronic device in a direction of a first event from the plurality of events. The method further includes determining whether the ROI is available in the direction of the first event. If the ROI is available in the direction of the first event, then the electronic device is calibrated towards the ROI. If the ROI is available in the direction of the at least one second event, then the electronic device determines that the ROI is available in the direction of at least one second event form the plurality of event, and the electronic device is calibrates towards the ROI in the direction of the second event.

In an embodiment the plurality of events indicate at least one of: a state of a user, and a noise from at least one source in proximity to the user. The first event from the plurality of events is detected at a first time unit, and the at least one second event from the plurality of events is detected at at least one second time unit.

In another embodiment rotating the electronic device includes determining, by the electronic device, a difference between values of the first event and the at least one second event. The rotating further includes detecting, by the electronic device, that the difference between values of the first event and the at least one second event meets a threshold criteria. A trigger signal is generated to rotate the electronic device based on the difference value and threshold criteria. The electronic device is rotated after receiving the trigger signal. In an embodiment the electronic device can be rotated by a motor internal or external to the electronic device.

In yet another embodiment, determining whether the ROI is available in the direction of the first event includes, determining an area of interest in the direction of the first event based on sensor data. The determining further includes recognizing parameters associated with at least one object in the area of interest. The determining further includes recognizing whether the at least one object is the ROI, by comparing the recognized parameters of the at least one object with at least one predefined identity parameters stored at the electronic device. The method further includes separating the at least one recognized ROI from the area of interest based on at least one of: the sensor data and the at least one predefined identity parameters.

In yet another embodiment, the identity parameters associated with the ROI includes ROI physical attributes, a ROI profile data, a ROI context, ROI preferences, and ROI signature. The identity parameters of the ROI is periodically learned and updated in the electronic device using at least one machine learning model.

In an embodiment the recognized parameters associated with the at least one object includes object physical attributes, an object profile data, an object context, object preferences, and object signature. Further detecting the plurality of events includes receiving a sensor data, classifying the sensor data in at least one of a primary event and a secondary event based on a predefined pattern and detecting, by the electronic device, the event based on at least one of the classified primary event and the secondary event.

The method further includes detecting a hazard associated with the ROI based on at least one event form the plurality of event and performing at least one action based on detected hazard.

The method further includes tracking the ROI based on the calibration, determining a presence of the ROI based on at one third event form the plurality of events, and performing at least one action based on the presence of the ROI.

The method further includes detecting a presence of the ROI at at least one IoT device based on the calibration and performing at least one action using the IoT device.

Accordingly, the embodiments herein disclose an electronic device comprising a memory, a processor coupled to the memory and a ROI based calibrator coupled to the memory. The ROI based calibrator is configured to detect a plurality of events, rotate the electronic device in a direction of a first event from the plurality of events and determine whether the ROI is available in the direction of the first event. The ROI is further configured to calibrate the electronic device towards the ROI, when the ROI is available in the direction of the first event. Further if the ROI is available in the direction of the at least one second event, the ROI calibrator determines that the ROI is available in the direction of at least one second event form the plurality of event, and calibrate the electronic device towards the ROI.

These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.

This method and system is illustrated in the accompanying drawings, throughout which like reference letters indicate corresponding parts in the various figures. The embodiments herein will be better understood from the following description with reference to the drawings, in which:

FIG. 1 is a block diagram of an electronic device for calibration based on a ROI, according to a prior art;

FIG. 2 is a block diagram of a ROI based calibrator for calibrating the electronic device based on the ROI, according to the embodiments as disclosed herein;

FIG. 3 is a flow diagram illustrating the process for calibrating the electronic device based on the ROI, according to the embodiments as disclosed herein;

Fig.4 is a block diagram of a system for calibrating the electronic device based on the ROI, according to an embodiment disclosed herein;

Fig.5 is a flow diagram, illustrating a training phase of electronic device calibration, according to an embodiment disclosed herein;

Fig.6 shows the schematic diagram, illustrating a calibration phase of the electronic device after the training phase, according to an embodiment disclosed herein;

Fig.7 is a flow diagram, illustrating an example for the calibration phase of the electronic device, according to an embodiment disclosed herein;

Fig.8A and Fig. 8B are a schematic diagram, illustrating an example scenario for calibrating the electronic device during change in sleep position of the user;

Fig. 9 is a flow diagram for generating a user profile for users according to an embodiment as disclosed herein;

Fig. 10 is a flow diagram for calibrating the electronic device based on the user profile signature according to an embodiment as disclosed herein;

Fig. 11A is a schematic diagram, illustrating an example scenario of calibrating the electronic device when a plurality of people is conversing, according to an embodiment as disclosed herein;

Fig, 11B shows the flow chart for calibrating the electronic device in the example scenario of Fig.11A, according to an embodiment as disclosed herein;

Fig. 12A and Fig. 12B are a schematic diagram, illustrating an example scenario for calibrating the electronic device according to an embodiment as disclosed herein;

Fig.13A, Fig.13B and Fig.13C are schematic diagram, illustrating an example scenario for detecting user presence and calibrating the IoT device based on the user presence, according to an embodiment as disclosed herein;

Fig. 14 is a schematic diagram, illustrating an example for calibration of the IoT device, according to an embodiment as disclosed herein;

Fig. 15 is a schematic diagram, illustrating an example for calibration of the IoT devices based on the contextual spatial awareness, according to an embodiment as disclosed herein;

Fig. 16 is a schematic diagram, illustrating an example for calibration of the IoT devices based on the contextual spatial awareness, according to an embodiment as disclosed herein;

Fig. 17A - Fig. 17C are schematic diagrams, illustrating a method for detecting unusual activities; according to an embodiment as disclosed herein;

The embodiments herein and the various features and advantageous details thereof are explained with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. Also, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments. The term "or" as used herein, refers to a non-exclusive or, unless otherwise indicated. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein can be practiced and to further enable those skilled in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.

As is traditional in the field, embodiments may be described and illustrated in terms of blocks which carry out a described function or functions.　These blocks, which may be referred to herein as managers, units, modules, hardware components or the like, are physically implemented by analog and/or digital circuits such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits and the like, and may optionally be driven by firmware and software. The circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like.　The circuits constituting a block may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some functions of the block and a processor to perform other functions of the block.　Each block of the embodiments may be physically separated into two or more interacting and discrete blocks without departing from the scope of the disclosure.　Likewise, the blocks of the embodiments may be physically combined into more complex blocks without departing from the scope of the disclosure.

The accompanying drawings are used to help easily understand various technical features and it should be understood that the embodiments presented herein are not limited by the accompanying drawings. As such, the present disclosure should be construed to extend to any alterations, equivalents and substitutes in addition to those which are particularly set out in the accompanying drawings. Although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are generally only used to distinguish one element from another.

Accordingly, the embodiments herein disclose a method for calibrating an electronic device based on a Region of Interest (ROI). The method includes detecting, by the electronic device, a plurality of events and rotating the electronic device in a direction of a first event from the plurality of events. The method further includes determining whether the ROI is available in the direction of the first event. If the ROI is available in the direction of the first event, then the electronic device is calibrated towards the ROI. If the ROI is available in the direction of the at least one second event, then the electronic device determines that the ROI is available in the direction of at least one second event form the plurality of event, and the electronic device is calibrates towards the ROI.

Unlike convention methods, the proposed load balancer for an RTOS running on MP minimizes energy consumption while maintaining an even or balanced computational load on the multiple cores

Referring now to the drawings and more particularly to Fig.1-Fig17 where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments.

Fig. 1 is a bock diagram of an electronic device 100 for calibration based on the ROI, according to the embodiments as disclosed herein. The electronic device 100 can be, for example, but not limited to a smart social robot, a smart watch, a cellular phone, a smart phone, a Personal Digital Assistant (PDA), a tablet computer, a laptop computer, a music player, a video player, an Internet of things (IoT) device or the like. The electronic device 100 includes a memory 110, a processor 120, a ROI based calibrator 130 and a communicator 140.

The processor 120 is coupled to the ROI based calibrator 130, and the communicator 140. The processor 120 is configured to execute instructions stored in the memory 110 and to perform various other processes.

In an embodiment the electronic device 100 is configured to detect a plurality of events. The plurality of events may be for example but not limited to a snoring voice of a user while sleeping, a presence of the user in a particular region, a group conversation of plurality of user, a state of a user, a noise from at least one source in proximity to the user and the like. In an embodiment the first event is detected at a first time unit and at least one second event is detected at at least one second time unit.

The electronic device 100 is further configured to rotate in a direction of a first event from the plurality of events. In an embodiment the electronic device 100 may be rotated by a wireless rotatable charger or by a motor which is external or internal to the electronic device 100.

The electronic device 100 further, determine whether the ROI is available in the direction of the first event. The electronic device 100 is calibrated towards the ROI in the direction of first event if the ROI is available in the direction of the first event. The electronic device 100 is calibrated towards the ROI in the direction of the at least one second event if the ROI is available in the direction of the at least one second event.

Fig. 2 is a block diagram of the ROI based calibrator 130. The ROI based calibrator 130 includes an event detector 132 and a ROI determiner 134. In an embodiment ROI based calibrator 130 receives the plurality of event. In an embodiment the plurality of events may be a sensor data received by the event detector 132. The sensor data is classified by the event detector 132 in at least one of the primary event and the secondary event based on a predefined pattern. In an example embodiment, the primary event may be the snoring of a user sleeping in a room and the secondary event may be the surrounding noise of the devices such as TV, AC and the like.

The ROI based calibrator 130 may be a software algorithm. For example, the ROI based calibrator 130 may be a set of various program codes and instructions for detecting various events and determining the ROI. The ROI based calibrator 130 is stored in memory 110 and can be executed by the processor 120. The operation of the ROI based calibrator 130 described in this document may be performed by the processor 120 executing a program code or an instruction of the ROI based calibrator 130.

Based on the classification of the events, the event detector 132, detects the event based on at least one of the primary event and the secondary event as the first event and the at least one second event.

In an embodiment the ROI determiner 134 determines a value of the first event and a value of the at least one second event. Further the ROI determiner 134, detects a difference between values of the first event and the at least one second event. Further the ROI determiner 134, determines whether the difference value meets a threshold criteria. If the threshold criteria are met, the ROI determiner 134 generates a trigger signal to rotate the electronic device 100. After receiving the trigger signal the electronic device 100 is rotated.

After rotating the electronic device 100, the ROI determiner 134 determines whether the ROI is available in the direction of the first event. In an embodiment the ROI determiner 134 determines an area of interest in the direction of the first event based on a sensor data. Parameters associated with at least one object in the area of interest are recognized by the ROI determiner 134. The ROI determiner 134 determines whether the at least one object is the ROI, by comparing the recognized parameters of the at least one object with at least one predefined identity parameters stored at the electronic device 100. Further the at least one recognized ROI is separated from the area of interest based on at least one of the sensor data and the at least one predefined identity parameters. Thus the ROI determiner 134 determines whether the ROI is available in the direction of the first event.

Further, if the ROI is available in the direction of the first event then the electronic device 100 is calibrated in the direction of the first event. If the ROI is not available in the direction of the first event, but is available in the direction of the at least one second event, then the electronic device 100 is calibrated in the direction of the at least one second event.

In an embodiment the identity parameters associated with the ROI includes at least one of ROI physical attributes, a ROI profile data, a ROI context, ROI preferences, or ROI signature. The identity parameters of the ROI is periodically learned and updated in the electronic device 100 using at least one machine learning model. The recognized parameters associated with the at least one object includes at least one of object physical attributes, an object profile data, an object context, object preferences, or object signature.

In another an embodiment the electronic device 100, detects a hazard associated with the ROI based on at least one event form the plurality of events. Further, the electronic device 100 performs at least one action based on detected hazard.

In an another embodiment, the electronic device 100 tracks the ROI based on the calibration of the electronic device 100 and determines a presence of the ROI based on at least one third event form the plurality of events. Based on the presence of the ROI at least one action is performed by the electronic device.

In an embodiment the presence of the ROI is detected at at least one IOT device based on the calibration of the electronic device. Further at least one action is performed using the IoT device based on the presence of the IoT devices.

FIG. 3 is a flow diagram illustrating the process for calibrating the electronic device 100 based on the ROI, according to the embodiments as disclosed herein. At operation 302, a change in a state of: the user, the event and the electronic device 100 is sensed. In an example embodiment the change in the state of the user relates to a change in the user's sleeping position, the user's snoring intensity, change in user's breathing, and the like. In another example embodiment the change in state of the electronic device 100 may relate to addition or removal of other sensor devices which monitors the user states and electronic device 100. Further, in yet another embodiment the change in state of the event may relate to addition of new events, an event looking for user's movements, events produced by user, events related to user and the like.

At operation 304, a changed state significance analysis is performed. The effect of the changed state in the ROI is checked. Further effect of the changed user position on the snore intensity is determined. Also the effect of the changed user action or user event on the ROI is checked.

At operation 306, based on the changed state analysis a movement is induced in the ROI. In an embodiment if the change in the state of the user meets the threshold criteria then the electronic device 100 is calibrated. Further if the changed state of the user is threatening, then a notification is sent to the user. After inducing the movement in the ROI, the movements in the ROI is monitored and updated in the database.

At operation 308, a change effect minimization analysis is performed. If multiple users are present in the ROI, then the event is deduced specific to user at operation 310. Noise profile based noise filters are applied for removing unwanted noises. Further boundaries affecting the accuracy of the primary events are removed.

At operation 312, state evaluation is performed. An analysis for a intended user is performed. The output of a sleep analysis is displayed. The user profiles are updated with the changed user's position and the changes user behavior. A biometric is constructed for the intended user. Further new ROI are generated if there are any updates.

Fig.4 is a block diagram of a system 400 for calibrating the electronic device 100 based on the ROI. The system 400 includes sensors 410, machine learning modules 420 and hardware components 430. The sensors 410 for example may include, but not limited to at least one of a body heat map sensor 412, temperature sensor 413, mechanical sensor 414, accelerometer sensor 415, gyroscope sensor 416, or magnetometer sensor 417. The machine earning modules 420 may include for example but not limited to at least one of an audio source separator module 421, biometrics module 422, calibration module 423, sound classification module 424, or contextual spatial module 425. Further the hardware components 430 may include for example but not limited to at least one of a microphone 431, a speaker 432, a rotator device 433, or a camera 434.

In an embodiment, the sensors 410 receive at least one of events. Further, the machine learning modules 420 send a signal to the hardware components 430 using machine learning technique for rotating at least one of the hardware components 430 according to the ROI. The hardware components 430 are referred as the electronic device 100 for simplicity and better understanding.

In an embodiment the calibration of the electronic device 100 is divided into two parts namely a training phase, and a calibration phase.

Training Phase:

In an example of the training phase, the system 400 learns about the day to day activities of the user and the user's preference to provide accurate and personalized results, using the machine learning modules 420. In an embodiment a user profile is generated by the machine learning modules 420 and is stored in a user profile database (not shown in fig). In an embodiment for generating the user profile, the system 400 may determine at least one of a physical state of the user, the user's activity timeline, the user's preferences, or a context of the user. The user's activity timeline may provide detail about the user's action in morning, at daytime and at night. Further the user's preference may provide information about for example but not related to an application usage by the user, a device usage by the user, a user's work and the like. Further the user's context may relate to the surrounding environment of the user. The surrounding example may be but not limited to a user's home, user's office and the outdoor environment of the user. Based on at least one of the user's activity timeline, the user's preference or the user's context, the calibration of the electronic device 100 may be performed. Further the IoT devices are trained based on the user's profile.

Further, based on the user profile the electronic device 100 is calibrated. In an embodiment for calibrating the electronic device 100 based on the user profile, a plurality of parameters may be determined. The plurality of parameters includes a calibration need of the electronic device 100 based on the user profile, an action to be taken by the electronic device 100 based on the user profile and an analysis and output obtained by the system 400 for the functioning of the electronic device 100 based on the user profile. In an embodiment the calibration need of electronic device 100 may refer to for example but not limited to the user's presence, the user's need for action and the user's preferences. Further the action to be performed by the electronic device 100 while calibration may be for example but not limited to changing the position of the electronic device 100 such that the electronic device 100 is closer to the user, providing comfort to the user and alerting the user. Further based on the user profile, the calibration need and the actions to be performed while calibrating the electronic device 100, the output is obtained. The output determines at least one of: a need to recalibrate the electronic device 100, whether to perform a physical movement of the electronic device 100, or whether to perform internal computation.

In other example scenario, the training phase includes generating a profile of devices termed as device profile. The devices can be for example but not limited to IoT devices. The device profile is generated and updated by the system based on the user profile. In an embodiment the system 400 learns information about the user's presence and the user's schedule in a day through machine learning modules 420. The machine learning modules 420 may learn the user's schedule at least one of at night, at morning, at afternoon or at midnight. Based on the user's schedule the IoT devices may be profiled.

In another example scenario, the user may have specific cooking time, watching television time, working on the desk time and the like. During training phase the user profile generated may include this user data. Based on the user profile, calibrating the IoT devices may produce result accurately. In an embodiment, notification may be provided on the IoT devices in the kitchen such as a display message on the refrigerator when the user is likely to cook there. There is less probability that user may miss the message if phone is not nearby.

Fig.5 is a flow diagram 500, illustrating the training phase of electronic device 100 calibration, according to an embodiment disclosed herein. The flow diagram 500 illustrates the learning of the day to day activities of the user and the user's preference by the system 400 for providing accurate and personalized calibration results of the electronic device 100. At operation 502, the system 400 checks for user information in the user profile database and proceeds to operation 504. After checking the user profile, at operation 504 the electronic device 100 is calibrated based on the user's profile. Further at operation 506, the system 400 authenticates the user profile. If the user profile information is true then the flow proceeds to operation 508 and if the user profile information is false then the flow 500 proceeds to operation 510. At operation 508, where the user profile is authenticated to be true, the system 400 waits for next activity or a change in the user profile information and the flow 500 returns to operation 502. At authenticate 510, the system 400 searches for at least one of: the user's current location or the user's activity. At operation 512, the system 400 updates the user profile database based on at least one of: the user's current location or the user's activity. At operation 514 the electronic device 100 is calibrated based on the updated user profile. Thus the system 400 learns and updates the user profile for calibrating the electronic device 100.

Fig.6 shows the schematic diagram 600, illustrating the training phase of the electronic device 100, according to an embodiment disclosed herein. At operation 602, the system 400 checks for at least one of: the user's presence or the state of the user. In an example scenario the system 400 may perform at least one of but not limited to: determining whether the user is asleep or awake, identifying the presence of the user, or identifying the environment surrounding the user. At operation 604, the system 400 performs an analysis based on the user profile. The analysis may include a high intensity audio analysis, a noise cancellation based on a noise profile and a heat map analysis. At operation 606 based on the results from the analysis at operation 604, the system 400 updates the user profile database and calibrates the electronic device 100 based on the updated user profile.

Calibration Phase

Fig.7 is a flow diagram 700, illustrating an example for the calibration phase of the electronic device 100, according to an embodiment disclosed herein. The example relates to change in sleep position of the user. In the example scenario a user is sleeping in a room having number of noise making devices such as a AC, a TV and the like. The electronic device 100 is for detecting the user's activities.

At operation 702, the system 400 performs an event analysis for observing the change in the user position stored in the user profile from the current position of the user. The event analysis performed may include generating a noise profile at current position of the use and performing noise cancellation is performed if the noise profile is previously generated. The noise profile includes determining the noise of the user along with the surrounding noise. Noise calibration includes stopping the external noise using a noise filter and passing only the user's snoring noise through the noise filter.

At operation 704, the system 400 determines whether the user position is changed or not using a heat map profile. If the user position is changed then the flow 700 proceeds to operation 706 and if the user position is not changed the flow returns back to operation 702.

At operation 706, the system 400 performs an analysis for observing the spatial movement of the user. The event analysis for determining the spatial movement of the user relates includes determining the intensity of snore of the user. Further the event analysis for spatial movement may also include determining the user's motion by generating a heat map profile for the user. Further the event analysis for spatial movement may also include determining an intensity of breathing of the user and also determining the user's speech.

At operation 708, the system 400, determines whether there is a significant change in the position of the user based on the analysis performed at operation 706. At operation 708, the system 400 performs a comparison for detecting a spatial change in the event. The system 400 compares a previously obtained point of interest and compares with the determined point if interest. Further the system 400 also compares previously obtained use noise profile with the current noise profile. If there is a significant change in the position of the user then the flow 700 proceeds to operation 710, and if there is no significant change in the position of the user then the flow 700 return back to operation 706.

At operation 710, the system 400 invokes a calibration module after determining a significant change in the position of the user for calibrating the electronic device 100 with respect to user's changed position. At operation 712, the system 400 determines whether a spatial direction is found. Determining the spatial direction includes performing an analysis on the sensor data for detecting spatial change and performing event data analysis for user's positional change.

If a spatial direction is found then the flow 700 proceeds to operation 714 or else back to operation 710. At operation 714, the system 400, evaluates the area of interest for calibrating the electronic device 100, and the point of interest. The system also determines the noise profile at the new position of the user.

Fig.8A and Fig. 8B are schematic diagrams, illustrating an example scenario for calibrating the electronic device 100 during change in sleep position of the user. As seen in Fig. 8A a user 800 is sleeping. The system 400 detects the first event 802, representing the position of the user 800 at first unit of time. Relative to the position of the user 800 at the first event the electronic device 100 is at position 806.

Further the system 400 detects a second event 804, representing the position of the user 800 at second unit of time. As the position of the user 800 is changed from 802 to 804, the system 400 changes the position of the electronic device 100 to 808 and calibrates the electronic device 100 in the direction of the second event 804. Thus the system 400 changes the direction of the electronic device 100 based on the change in the user position and calibrates the electronic device 100 based on the new position of the user. As seen in Fig. 8B the user 800 changes its position from position 1 to position 2. Based on the change in the user position the electronic device 100 is also rotated by the system 400.

Fig. 9 is a flow diagram 900 for generating a user profile for users according to an embodiment as disclosed herein. At operation 902, the system 400 calibrates the heat map sensor for determining the number of users in a room. At operation 904, the system 400 determines whether the users are found. If the users are determined, then the flow 900 proceeds to operation 906 or else returns back to operation 902. At operation 906, the system 400 generates the user profile for the user detected. The user profile may include at least one of the heat map profile, the snore profile of the user, the area of interest representing the presence of the user or the point of interest representing the user's location. At operation 908, the system 400 generates a profile signature for each of the user profile. In an embodiment the profile signature may include biometric information of the user.

Fig. 10 is a flow diagram 1000 for calibrating the electronic device 100 based on the user profile signature according to an embodiment as disclosed herein. At operation 1002, the system 400 analyses the user's sleep. In an embodiment, analyzing the user's sleep may include at least one of analyzing the users' snoring, determining whether the user is sleep walking or sleep talking, analyzing the user's body temperature, or determining the change in the user's heat map.

At operation 1004, the system 400 determines whether the user's event is detected or not. If the user's event is detected then the flow 1000 proceeds to operation 1006 or returns to operation 1002. At operation 1006, the electronic device 100 may perform a set of action based on the user profile, the user context and the user's position. The electronic device 100 may alert the user of interest, communicate to the IoT devices about the user's event and calibrating smart devices for users comfort.

Fig. 11A is a schematic diagram, illustrating an example scenario of calibrating the electronic device 100 when people are conversing, according to an embodiment as disclosed herein. Consider an example scenario, where the people are conversing sitting around a table with the electronic device 100 at the center of the table. In an event the system 400 may require rotating the electronic device 100 towards a particular user. In such a case the electronic device 100 is rotated towards the particular user, when the particular user starts conversing and the electronic device 100 is calibrated in the direction of the particular user. Thus by calibrating the electronic device 100 in the direction of the particular user, accuracy and efficient performance is achieved.

Fig. 11B shows the flow chart 1100 for calibrating the electronic device 100 in the above stated example scenario of Fig.11A. At operation 1102, the system 400 initiates a group conversing calibration process. At operation 1104, the system 400 determines whether a speech is detected from at least one user from the plurality of users, within a particular time unit. In an embodiment the time unit may be five seconds. If the speech is not detected within 5 second then the flow 1100 proceeds to operation 1106 and if the speech is detected within 5 seconds then the flow 1100 returns to operation 1102. At operation 1106, the system 400 induces calibration by an intensity module for determining the presence of particular user in a predicted direction. In an embodiment an audio separator module is used for distinguishing and separating the particular user's voice form the plurality of users sitting around the table. Further the heat map profile is also used for determining the particular user's presence.

At operation 1108, the system 400 send a signal to a heat map rotatory system for calibrating the electronic device 100. At operation 1110, the system 400 determines whether the particular user is detected in the predefined direction. The method proceeds to operation 1112 if the user is present in the predefined direction and if the user is not detected in the predefined direction, then the flow 1100 returns to operation 1108. At operation 1112, the system defines the region of interest for capturing the high speech event corresponding to the particular user's presence.

Fig. 12A and Fig. 12B are schematic diagrams, illustrating an example scenario for calibrating the electronic device 100 according to an embodiment as disclosed herein. As seen in Fig, 12A, a plurality of users 1201-1209 are sitting around a table and conversing. Further a speaker 1210 is placed at the table. In an embodiment the speaker is playing a music. The user 1201-1205 are discussing an important matter and do not wish to hear the music played by the speaker 1210, whereas the other users 1206-1209 want to hear the music. In such a case, the system 400 detects the users 1201-205 conversing with each other. Based on the event detected by the system 400, the speaker 1210 is calibrated. The system 400 turns off the speaker 1210 in the direction of the users 1201-1205. As seen in Fig. 12A, the speaker 1210 in the direction of the users 1201-1205 is switched off.

Figs.13A, 13B and 13C are schematic diagrams, illustrating an example scenario for detecting user presence and calibrating an IoT device based on the user presence, according to an embodiment as disclosed herein.

Fig.13A shows a room with a light 1310 which is an Iot device. In an example scenario, a user 1320 has switched on the light 1310. After some time, the user 1320 went out and forgot to switch off the light 1310. The electricity is wasted as the light 1310 is on even when no one is present in the room. The proposed method provides a solution to such cases. The proposed method detects the presence of the user 1320 and switch the light 1310 on and off based on the presence of the user 1320.

Fig.13B and Fig. 13C are consecutive figures. In Fig. 13B, the light 1310 in the room is switched off as the user 1320 is not present in the room. Further in Fig. 13C the user 1320 enters the room. Here the system 400 recognizes the presence of the user 1320. After detecting the presence of the user 1320, the system 400 calibrates the light 1310 in the direction of the user and switch on the light 1310.

Fig. 14 is a schematic diagram, illustrating an example for calibration of the IoT device. Fig.14 shows a mother 1410 standing near the stairs on the ground floor. Further room 1 and room 2 are located on the first floor. A microphone 1420 which is the IoT device is used by the mother for calling out the kids in room 1 and room 2. In an example a first kid Mary is present in room 1 and a second kid Ronaldo is present in the room 2. The mother 1410 has some work and wants to call Mary for her help. The mother 1410 says "Mary, I need your help, please come !!". The system 400 detects an event, wherein the event is the noise of the mother 1410 calling out Mary. Now the system 400 checks the context of the event. Here the context of the content is calling out Mary. Further the system 400 searches for Mary in a pre-defined database and find out that Mary is associated with room 1. The system 400, now calibrates the microphone 1420 in the direction of Mary. As shown in Fig.14, the position 1430 of the microphone 1420 is the original position and position 1440 of the microphone 1420 is the new position after rotation.

Fig. 15 is a schematic diagram, illustrating an example for calibration of the IoT devices based on the contextual spatial awareness. As seen in Fig. 15, the user 1520 is sitting in a room and watches the television 1540. A speaker 1530 is communicatively coupled with the television 1540 and plays the sound from the television. It may happen that the speaker 1530 is not in proper place with respect to the user 1520 and the user 1520 is not able to hear the sound properly. For such cases the proposed method calibrates the speaker 1530 with respect to the position of the user 1520. The system 400, detects the user 1520 presence using the heat map profile of the user. After detecting the user presence, the system 400, determines the position of the user. Based on the position of the user, the system 400 creates a region of interest being the user and calibrates the speaker in the direction of the user. As seen in Fig. 15 the position of the speaker is changed from position 1 to position 2.

Fig. 16 is a schematic diagram, illustrating an example for calibration of the IoT devices based on the contextual spatial awareness. Fig. 16 shows a kitchen comprising a smart refrigerator with a display 1610, and a microwave with display 1620 and a user 1630. The system 400 detects the presence of the user 1630 using the heat map analysis and calibrates the microwave and the refrigerators, such that the refrigerator and the microwave will show results on display only when the user 1630 is present in the kitchen.

Fig. 17A - Fig. 17C are schematic diagrams, illustrating a method for detecting unusual activities. As seen in Fig. 17A, a first user 1710 and a second user 1720 are sleeping. While sleeping the system 400 keeps a track of the heat map, the sleeping noise profile, the snoring intensity, and the physical state of the user.

In an embodiment Fig. 17B slows the first user 1710 walking in sleep. The system 400 detects the event of first user 1710 walking and sends an alert to the electronic device 100. In such a way the system 400 is able to detect unusual activities of the first user 1710 and the sleeping disorder.

In another embodiment Fig. 17C shows the second user 1720. The system 400 determines the position of the second user 1720 and calibrates the electronic device 100 relative to the position of the second user 1720. Further the system 400 may detect various sleep related disorder such as sleep apnea, restless leg syndrome, and the like, based on the analysis performed. Further the system 400 sends alert signals to the electronic device 100 indicating the ill health of the second user 1720.

The principal object of the embodiments herein is to provide a method for calibrating an electronic device based on a region of interest (ROI).

Another object of the embodiments herein is to rotate the electronic device in a direction of a first event from a plurality of events.

Another object of the embodiments herein is to determine whether the ROI is available in the direction of the first event.

Another object of the embodiments herein is to detect a hazard associated with the ROI and perform at least one action based on detected hazard.

Another object of the embodiments herein is to determine a presence of the ROI and perform at least one action based on the presence of the ROI.

Another object of the embodiments herein is to detect a presence of an IoT device and perform at least one action based on the presence of the IoT device.

According to various embodiments, a method for calibrating an electronic device based on a Region of Interest (ROI), the method comprising detecting, by the electronic device, a plurality of events, rotating the electronic device in a direction of a first event from the plurality of events, determining whether the ROI is available in the direction of the first event, and performing one of: when the ROI is available in the direction of the first event, calibrating the electronic device towards the ROI, and when the ROI is available in the direction of the at least one second event, determining that the ROI is available in the direction of at least one second event form the plurality of event, and calibrating the electronic device towards the ROI.

According to various embodiments, the plurality of events indicate at least one of a state of a user, a noise from at least one source in proximity to the user and wherein the first event from the plurality of events is detected at a first time unit, and the at least one second event from the plurality of events is detected at at least one second time unit.

According to various embodiments, rotating the electronic device comprises determining, by the electronic device, a difference between values of the first event and the at least one second event, detecting, by the electronic device, that the difference between values of the first event and the at least one second event meets a threshold criteria, generating, by the electronic device, a trigger signal to rotate the electronic device; and rotating the electronic device based on the trigger signal.

According to various embodiments, determining whether the ROI is available in the direction of the first event comprises determining, by the electronic device, an area of interest in the direction of the first event based on a sensor data, recognizing, by the electronic device, parameters associated with at least one object in the area of interest, recognizing, by the electronic device, whether the at least one object is the ROI, by comparing the recognized parameters of the at least one object with at least one predefined identity parameters stored at the electronic device, and separating, the at least one recognized ROI from the area of interest based on at least one of the sensor data and the at least one predefined identity parameters.

According to various embodiments, the identity parameters associated with the ROI comprises ROI physical attributes, a ROI profile data, a ROI context, ROI preferences, and ROI signature and wherein the identity parameters of the ROI is periodically learned and updated in the electronic device using at least one machine learning model.

According to various embodiments, the recognized parameters associated with the at least one object includes object physical attributes, an object profile data, an object context, object preferences, and object signature.

According to various embodiments, detecting the plurality of events comprises receiving, by the electronic device, a sensor data, classifying, by the electronic device, the sensor data in at least one of a primary event and a secondary event based on a predefined pattern, and detecting, by the electronic device, the event based on at least one of the primary event and the secondary event.

According to various embodiments, the method further includes detecting, by the electronic device, a hazard associated with the ROI based on at least one event form the plurality of event, and performing, by the electronic device, at least one action based on detected hazard.

According to various embodiments, the method further includes tracking, by the electronic device, the ROI based on the calibration, determining, by the electronic device, a presence of the ROI based on at one third event form the plurality of events, and performing, by the electronic device, at least one action based on the presence of the ROI.

According to various embodiments, the method further includes detecting, by the electronic device, a presence of the ROI at at least one IoT device based on the calibration, and performing, by the electronic device, at least one action using the IoT device.

According to various embodiments, an electronic device, comprising a memory, and a processor coupled to the memory, wherein the processor is configured to detect a plurality of events, rotate the electronic device in a direction of a first event from the plurality of events, determine whether a region of interest (ROI) is available in the direction of the first event, when the ROI is available in the direction of the first event, calibrate the electronic device towards the ROI, and when the ROI is available in the direction of the at least one second event, determine that the ROI is available in the direction of at least one second event form the plurality of event, and calibrate the electronic device towards the ROI.

According to various embodiments, the plurality of events indicate at least one of a state of a user, a noise from at least one source in proximity to the user and the first event from the plurality of events is detected at a first time unit, and the at least one second event from the plurality of events is detected at at least one second time unit.

According to various embodiments, rotating the electronic device includes determine a difference between values of the first event and the at least one second event, detect that the difference between values of the first event and the at least one second event meets a threshold criteria, generate a trigger signal to rotate the electronic device, and rotating the electronic device based on the trigger signal.

According to various embodiments, determining whether the ROI is available in the direction of the first event comprises determine an area of interest in the direction of the first event based on sensor data, recognize parameters associated with at least one object in the area of interest, recognize whether the at least one object is the ROI by comparing the recognized parameters of the at least one object with at least one predefined identity parameters stored at the electronic device and separate the at least one recognized ROI from the area of interest based on at least one of the sensor data and the at least one predefined identity parameters.

According to various embodiments, the identity parameters associated with the ROI comprises ROI physical attributes, a ROI profile data, a ROI context, ROI preferences, and ROI signature, and wherein the identity parameters of the ROI is periodically learned and updated in the electronic device using at least one machine learning model.

According to various embodiments, the recognized parameters associated with the at least one object comprises object physical attributes, an object profile data, an object context, object preferences, and object signature.

According to various embodiments, detecting the plurality of events comprises receive a sensor data, classify the sensor data in at least one of a primary event and a secondary event based on a predefined pattern, and detect the event based on the on at least one of the primary event and the secondary event.

According to various embodiments, the electronic device is further configured to detect, a hazard associated with the ROI based on at least one event form the plurality of event, and perform at least one action based on detected hazard.

According to various embodiments, the electronic device is further configured to track the ROI based on the calibration, determine a presence of the ROI based on at one third event form the plurality of events, and perform at least one action based on the presence of the ROI.

According to various embodiments, the electronic device is further configured to detect a presence of the ROI at at least one IoT device based on the calibration, and perform at least one action using the IoT device.

The embodiments disclosed herein can be implemented using at least one software program running on at least one hardware device and performing network management functions to control the elements.

The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the embodiments as described herein.

Claims

An electronic device, comprising:

a memory; and

a processor coupled to the memory;

wherein the processor is configured to:

detect a plurality of events;

rotate the electronic device in a direction of a first event from the plurality of events;

determine whether a region of interest (ROI) is available in the direction of the first event; and

when the ROI is available in the direction of the first event, calibrate the electronic device towards the ROI, and

when the ROI is available in the direction of the at least one second event, determine that the ROI is available in the direction of at least one second event form the plurality of event, and calibrate the electronic device towards the ROI.
The electronic device of claim 1, wherein

the plurality of events indicate at least one of a state of a user, a noise from at least one source in proximity to the user and

the first event from the plurality of events is detected at a first time unit, and the at least one second event from the plurality of events is detected at at least one second time unit.
The electronic device of claim 1, wherein rotate the electronic device includes:

determine, by the ROI based calibrator, a difference between values of the first event and the at least one second event;

detect, by the ROI based calibrator, that the difference between values of the first event and the at least one second event meets a threshold criteria;

generate, by the ROI based calibrator, a trigger signal to rotate the electronic device; and

rotating the electronic device based on the trigger signal.
The electronic device of claim 1, wherein determine whether the ROI is available in the direction of the first event comprises:

determine, by the ROI based calibrator, an area of interest in the direction of the first event based on sensor data;

recognize, by the ROI based calibrator, parameters associated with at least one object in the area of interest;

recognize, by the ROI based calibrator, whether the at least one object is the ROI by comparing the recognized parameters of the at least one object with at least one predefined identity parameters stored at the electronic device; and;

separate, by the ROI based calibrator, the at least one recognized ROI from the area of interest based on at least one of the sensor data and the at least one predefined identity parameters.
The electronic device of claim 1, wherein the identity parameters associated with the ROI comprises ROI physical attributes, a ROI profile data, a ROI context, ROI preferences, and ROI signature, and wherein the identity parameters of the ROI is periodically learned and updated in the electronic device using at least one machine learning model.
The electronic device of claim 1, wherein the recognized parameters associated with the at least one object comprises object physical attributes, an object profile data, an object context, object preferences, and object signature.
The electronic device of claim 1, wherein detect the plurality of events comprises:

receive, by the ROI based calibrator, a sensor data;

classify, by the ROI based calibrator, the sensor data in at least one of a primary event and a secondary event based on a predefined pattern; and

detect, by the ROI based calibrator, the event based on the on at least one of the primary event and the secondary event.
The electronic device of claim 1, wherein the processor is further configured to:

detect, a hazard associated with the ROI based on at least one event form the plurality of event; and

perform at least one action based on detected hazard.
The electronic device of claim 1, wherein the processor is further configured to:

track the ROI based on the calibration;

determine a presence of the ROI based on at one third event form the plurality of events; and

perform at least one action based on the presence of the ROI.
The electronic device of claim 1, wherein the processor is further configured to:

detect a presence of the ROI at at least one IoT device based on the calibration; and

perform at least one action using the IoT device.
A method for calibrating an electronic device based on a Region of Interest (ROI), the method comprising:

detecting, by the electronic device, a plurality of events;

rotating the electronic device in a direction of a first event from the plurality of events;

determining whether the ROI is available in the direction of the first event; and

when the ROI is available in the direction of the first event, calibrating the electronic device towards the ROI; and

when the ROI is available in the direction of the at least one second event, determining that the ROI is available in the direction of at least one second event form the plurality of event, and calibrating the electronic device towards the ROI.
The method of claim 11, wherein the plurality of events indicate at least one of a state of a user, a noise from at least one source in proximity to the user and wherein the first event from the plurality of events is detected at a first time unit, and the at least one second event from the plurality of events is detected at at least one second time unit.
The method of claim 11, wherein rotating the electronic device comprises:

determining, by the electronic device, a difference between values of the first event and the at least one second event;

detecting, by the electronic device, that the difference between values of the first event and the at least one second event meets a threshold criteria;

generating, by the electronic device, a trigger signal to rotate the electronic device; and

rotating the electronic device based on the trigger signal.
The method of claim 11, wherein determining whether the ROI is available in the direction of the first event comprises:

determining, by the electronic device, an area of interest in the direction of the first event based on a sensor data;

recognizing, by the electronic device, parameters associated with at least one object in the area of interest;

recognizing, by the electronic device, whether the at least one object is the ROI, by comparing the recognized parameters of the at least one object with at least one predefined identity parameters stored at the electronic device; and

separating, the at least one recognized ROI from the area of interest based on at least one of the sensor data and the at least one predefined identity parameters.
The method of claim 11, wherein the identity parameters associated with the ROI comprises ROI physical attributes, a ROI profile data, a ROI context, ROI preferences, and ROI signature and wherein the identity parameters of the ROI is periodically learned and updated in the electronic device using at least one machine learning model.