WO2003100585A2

WO2003100585A2 - Context-aware portable device

Info

Publication number: WO2003100585A2
Application number: PCT/IB2003/002212
Authority: WO
Inventors: Vincentius P. Buil; Bartel M. Van De Sluis; Gerrit Hollemans
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2002-05-24
Filing date: 2003-05-22
Publication date: 2003-12-04
Also published as: AU2003232990A8; WO2003100585A3; AU2003232990A1

Abstract

The invention relates to a data-processing device which comprises a sensor for sensing an ambient factor so as to control an operating parameter. The data-processing device according to the invention comprises at least one further sensor for sensing an ambient factor, and context means for deriving a context of use of the data-processing device by comparing the output of the sensor and the at least one further sensor.

Description

Context-aware portable device

FIELD OF THE INVENTION

The invention relates to a data-processing device comprising a sensor for sensing an ambient factor so as to control an operating parameter. The invention forther relates to a method of determining a context of use of a data-processing device, the method comprising a step of sensing an ambient factor of the data-processing device. The invention further relates to a computer program product implementing the above method.

BACKGROUND OF THE INVENTION

A known example of such a data-processing device is a display device, for example a personal digital assistant (PDA), which comprises a light sensor to measure ambient light in order to adjust the brightness of the displayed picture. Another known example is a car radio which adjusts its audio volume in dependence upon ambient noise or the car's speed. Such measures indeed improve the usability of the data-processing device, but their usefulness is still limited because it allows only a very simple measurement of ambient factors. A device that behaves more attentively with respect to the user's situation is aimed at behaving more appropriately to the situation at hand. As such, the result is preferably a system that causes less irritation, which is something that tends to go unnoticed. Current devices are deaf to the outside world and simply follow the predefined behavior. Consequently, mobile phones always ring with the same volume, independent of the situation, which causes a lot of frustration especially during business meetings.

OBJECT AND SUMMARY OF THE INVENTION

It is an object of the invention to provide an improved system and method of the type defined in the opening paragraph. To this end, the data-processing device according to the invention comprises at least one further sensor for sensing an ambient factor, and context means for deriving a context of use of the data-processing device by comparing the output of the sensor and the at least one further sensor. In this measure of adding at least one additional sensor and comparing its output with the first sensor, it is achieved that the context of use of the data-processing device can be derived in a much more precise way. For example, by comparing the light conditions on two sides of the data-p can be derived about its context of use, e.g. whether it is lying on a table, kept in a pocket, hand-held etc.

An embodiment of the data-processing device according to the invention further comprises alert means for generating an alert so as to notify a user of an event, said context means being arranged to control at least one parameter of the alert, hi some situations, mobile phones and PDAs are felt to behave quite rudely when there is a call or an appointment they want to alert the user about. Everyone will recognize the embarrassing situations of a phone or a PDA going off during a meeting or presentation, during conversations, or during diner at home or in other social situations. However, people do like to be alerted for information of interest, be it an incoming message, an appointment, or some other type of information relevant for the moment, such as entertainment events that are time- related, be it sport matches, live concerts, and other types of broadcast content on TV and radio. However, this alerting may not be obtrusive in any way in order to be accepted by the user. In the data-processing device according to the invention, reminders and incoming suggestions are presented to the user in the form that is suitable to the current context of use. Contextual cues that are obtained via a number of sensors determine how obtrusively these signals should be presented to the user both auditory and visually. For example, based on the detected context of use, the system may select the most suitable modality of the alert, e.g. auditory, visually or both at the same time, and how intensive the alert signal has to be.

Furthermore, these cues may be used to decide when certain information is offered to the user. Obviously, an incoming telephone call has to be alerted immediately, only the modality and duration of the alert may depend on the context of use. However, to some extent, alerts for incoming messages or reminders may be shifted to an earlier or later point of time if they have a relatively low priority and the user is currently involved in some activity, e.g. watching a program. The alerting may thus be postponed until the activity has finished, for example, the system may detect whether the watched program has ended.

In an embodiment of the data-processing device according to the invention, the sensor and further sensors are light sensors, preferably at opposite sides of the device, for example the front and rear sides. However, other types of sensors like tilt sensors and heat sensors may be utilized to supply additional information about the context of use.

The invention is particularly suitable for personal digital assistants, mobile phones and advanced remote controls. Throughout this specification, the term "data" is to be include all forms of information, such as audio, video, graphical and textual data.

BRIEF DESCRIPTION OF THE DRAWINGS These and other aspects of the invention are apparent from and will be elucidated, by way of a non-limitative example, with reference to the embodiment(s) described hereinafter. In the drawings,

Figure 1 shows a schematic overview of a system comprising a data- processing device according to the invention. Figure 2 shows an example of a data-processing device according to the invention.

DESCRIPTION OF EMBODIMENTS

Figure 1 shows a schematic overview of a system comprising a data- processing device according to the invention. A normal desktop PC 101 collects EPG information from the Internet and matches this with a user profile to generate recommendations. The EPG data together with the recommendations are then downloaded onto the PDA device 102, via a USB or serial cradle or via a wireless connection (e.g. bluetooth or wireless Ethernet). The technology behind the filtering and retrieval is not within the scope of the present invention. On the PDA 102, the user can interact with the EPG data and the recommendations. One of the functions offered is switching the TV 103 to the channel where the program displayed on the PDA 102 is currently playing. The TV 103 will be controlled by the PDA 102 via the PC 101 through a wireless link between the PDA 102 and the PC 101. Besides controlling the TV 103, the user can also communicate with other PDA users 104, by sending suggestions for TV programs. The suggestions can be transferred via the SMS or via other communication protocols. The PDA 102 collects context information via a variety of sensors 105. The sensors 105 are connected to a data-acquisition unit (DAQ 106), which essentially converts the analog sensor data into digital signals that can be read by the PDA 102. Figure 2 shows an example of a data-processing device according to the invention. The PDA uses 4 types of sensors: two light sensors 201 and 202, a touch sensor 203, two passive infrared sensors 204, and a microphone 205. These sensors are placed on the PDA, to be able to detect various aspects of the environment.

The sensors and their positioning in the prototype casing are given below: Two light sensors 201 and 202: The light sensors used (e.g. type BPW34 from Siemens) that react to visible light. They are merely used to distinguish between low and high levels of illumination that correspond to darkness and light, respectively. The sensors are positioned one on the front panel and another on the back panel of the casing. This is to determine whether the device is in the pocket when both sensors report darkness or when it lies on the table when one of them reports light and another darkness.

A touch sensor 203 : The touch sensor consists of a pair of thin copper strips positioned on both sides of the device. These strips are used to detect whether the user holds the device or not. A small potential difference is placed on the strips, and as soon as they are both touched with the hand, the conductive properties of the skin will cause a small current to run from one strip to another. This is sensed and reported to the data acquisition unit 106.

Two passive infrared sensors 204: The two passive infrared sensors (e.g. type IRA-E700ST0 from Murata) are pyroelectric sensors that are sensitive to a narrow band of the infrared light spectrum, around a wavelength of 9.4mm, which corresponds to the infrared spectrum emitted by the human body. These sensors are typically used in motion sensors for outdoor lamps and burglar alarms. Each sensor has a wide sensitivity angle, close to 180 degrees. Thus, by using two sensors mounted at the top and the bottom of the casing, it is possible to cover nearly the whole space around the device. In this configuration, the sensors can detect when people move around the device within a range of 2 to 3 meters, in which case they will report oscillating voltage changes.

A microphone 205: The microphone will be used mainly to sense the acoustic noise level in the room, which corresponds to the signal intensity registered by the microphone. A certain threshold needs to be determined for discriminating between quiet and noisy conditions.

The raw signals received from the sensors are not meaningful yet until they have gone through a first-order pre-processing operation. This pre-processing operation, which can be realized in hardware and/or software, modifies the signal into a useful and meaning form, such as calculating the average, standard deviation, minimum or maximum over a sliding window. For example, the light sensors are very sensitive and can even sense the update frequency emitted by the lights in the room. However, this detailed signal is less relevant so it is desirable to average out much of the noise that is picked up by the sensor. As such, the signal is filtered to produce a slower but more steady signal. This signal is then evaluated against a set of critical values, such as experimentally defined thresholds, to decide whether the light sensor measures darkness or light, which is the cont the situation. The same is done for the microphone signal, the touch sensor, and the infrared sensors. For example, the infrared sensors need pre-processing to cancel out all the unwanted influences of the environment, being the heat caused by holding the device, lights, central heating, and so on. The infrared sensor has two sensitive elements. A body passing by will activate two sensitive elements sequentially whereas other heat sources will simultaneously affect both. The motion sensor has to be tuned to react only to sequential activation. At rest, when no motion is detected, the signal reading fluctuates slowly around zero. The oscillation of the signal increases in amplitude when motion is registered. Consequently, signal energy can serve as an indicator of the detected movement.

A context model facilitates translation of the sensor inputs into the meaningful situations. The situations described in the context model are relevant to the application and should be recognized by the system. The sensor inputs are described in the model as contextual variables that can be derived from the sensor cues. For example, darkness is registered when the light sensor reports a low signal level.

The context model for the prototype is depicted in Table 1. On the upper row are the sensors whose values are mapped on the situations in the first column. The rows that correspond to a specific situation contain the sensor inputs that should be valid to detect a certain situation.

Table 1: The context model, explaining how situations are derivet context cues generated from the sensor signals

The interpretation of the situations from the sensor cues is based on the following assumptions:

In_pocket is valid for the situations when a mobile device is either in a pocket or in a bag of the user. In this case it is expected that both light sensors on the front and back panel of the device report a low signal level corresponding to a low illumination. The touch sensor should not report any signal because there is no contact with the human body. There is a big chance that the microphone reports a low signal level because the outside noise is damped by the pocket or bag, but this factor has low priority for the situation detection.

Lyingjxround corresponds to the situations when the device is placed on a surface in the illuminated room. In this context, one of the two light sensors should report darkness (the one that sits on the side facing the surface) and the other should report light which is indicated in Table 1 by the XOR logical function. The touch sensor in this context does not report skin contact.

Injtand context corresponds to the situation when the device is held in the hand of the user. Consequently, the touch sensor reports skin contact.

Motion _detected is interpreted when the infrared sensor reports motion. This situation corresponds to the case when a person comes in or goes out of the room, or when a person was already in the room but started to move. Consequently, motion_detected may correspond to two contexts: userjxrrives or user present. It is suggested that there should be discrimination between these two situations with a descending probability function. The function is drawn so that the motion, when detected, reaches the maximum level of 1. If no motion is then detected, the function degrades reporting decreasing probability of people's presence in time. The degrading speed should be defined experimentally.

Noisy corresponds to the high signal level reported by the microphone.

Quiet corresponds to the low signal level reported by the microphone.

TVjpresent is valid when the cricket system reports that the device is inside the room with an available TV set.

A context model facilitates the interpretation of contextual cues into a number of specific situations that correspond to those aspects of the context that we are interested in for our application. After this interpretation stage, the contextual information has to be translated into appropriate system behavior. The most straightforward method is to use a set of production rules that define the appropriate system behavior based variables describing the context at hand. The use of production rules is one of the most popular approaches to knowledge representation. A typical format of a production rule is as follows: If conditionl and or conditiorώ and/ or ...conditionN, then behaviorl .

It contains a condition clause and an action clause. The condition clause is typically a logical formula with conditions as arguments. The conditions represent the contexts such as in_pocket that are valid or not at a particular moment. The conditions are combined with logical operators such as AND, OR, etc. In several situations, it may be necessary to take record of the previous state of the system in order to make a decision on the system action at a current moment. The action clause contains a set of actions that correspond to the system behavior in a specific context.

The advantage of the IF-THEN rules is that they are modular and, in principle, consist of independent pieces of knowledge. This allows instant modification of the rules by adding new or deleting old ones.

For alerting the user of some event, a few contexts and the desired behavior in these contexts are now described.

Context 1: The device is in a suitcase:

While the PDA lies in the suitcase, a reminder set by the user goes off, producing a loud alert signal. The loud signal makes sure that it is heard by the user, to overcome the damping of the audio signal in a closed suitcase. Context 2: The suitcase is opened:

When the user opens the suitcase, the device does not need to produce a loud audio signal anymore to get through the suitcase. The user will now likely be near and watching the device and as such it will produce a less prominent sound in terms of loudness and urgency, while the screen backlight will be put on and an animated icon will be displayed to give a visual indication of the ringing reminder.

Context 3: The device is picked up by the user:

When the user then picks up the device, the device has caught the user's attention. The device changes its sound to be soft and calm and the reminder is opened automatically anticipating the intention of the user to read a message. The description of the reminded program is then displayed on the screen.

Context 4: The device lies on the table, the user is present in the room: The device is lying on the living room table, the user is close to the device and is watching TV. The user has set a reminder for a program that he does not want to miss and which starts within five minutes. The system presents the reminder auditory with a sound that adapts its volume to the noise level in the room for the user to hear it. Thus, it is ringing loudly if it is noisy, or vice versa. Furthermore, the user may be able to see the screen, but not necessarily read it. Therefore, the system switches on the backlight and displays an animated icon in the top corner of the screen in an attempt to also visually attract the user's attention. As soon as the user were to approach the device, he would see the blinking icon, and if he picks up the device, the reminder will be shown full screen. In a very simple embodiment, using only two cheap light sensors and straightforward reasoning via a finite state machine, it is easily possible to improve the context awareness. By putting one of the light sensors on the front side of the PDA, remote control or mobile phone, and the second one on the rear side, the user can indicate the context and thus its behavior in a very natural way by positioning the device concurrently. In this way, four states can be distinguished:

1. The device is lying on a table, front face downwards, in a normally lit environment. The front light sensor is covered (not receiving light) and the back light sensor is open (receiving light).

2. The device is lying on a table, front face upwards, in a normally lit environment: the front light sensor is open and the back light sensor is covered.

3. The device is standing upright or is in a plastic bag, for example. Both the front and back light sensors are open.

4. The device is in the dark, e.g. in one's pocket, suitcase, or a very dark environment. Both the front and back light sensors are covered. These four states can be used to adapt the behavior of the device. State 1 can be interpreted as an indication of a user that he does not want to be disturbed, and only warned by a small flashing light when the phone or an alarm goes of. State 2 can be interpreted as an indication that he is ready to receive phone calls or alarms, though preferably in a discrete way. State 3 can be used in case the user is alone or in a social situation where it is accepted to be fully open to incoming calls and the user wants to be sure he will not miss any. State 4 is appropriate when the user has the device in his pocket or in a suitcase, or at night when sleeping. In that case, the alarm will be less easy to hear and needs to be even louder than in state 3. As such, the following finite state machine can be made in the form of production rules: 1. IF front = dark AND back = light THEN use small light only

2. IF front = light AND back = dark THEN use soft ring signal

3. IF front = light AND back = light THEN use normal ring signal

4. IF front = dark AND back = dark THEN use loud ring signal. The invention can be applied to any mobile device that asks for attention now and then, such as mobile phones (incoming calls), PDAs (appointments with alarms, incoming e-mails, etc.) and personal remote controls (PRC, reminders for television programs). As PDAs come in different form factors, it may be necessary to organize the sensors differently, or use other sensors. A PDA is likely to be always on its back for quick lookup. Also, a PDA may not be suitable for standing upright. However, many PDAs have a lid that needs to be opened first. One could use a light detector on top and underneath the lid to detect various states. A suitable model could then be:

1. IF upside = light AND inside = dark THEN use small light only

2. IF upside = light/dark (don't care) AND inside = light THEN use soft ring signal 3. IF upside = dark AND inside = dark THEN use loud ring signal.

Again, here it is assumed that state 1 refers to being in a closed position on the table, state 2 in an open, accessible way on the table, and state 3 refers to the device being in a pocket, suitcase, or very dark environment (e.g. at night, sleeping). Sensing whether the device is opened or closed could also be done with a switch, as is done nowadays in many PDAs.

Throughout the Figures, the same reference numerals indicate similar or corresponding features. While the invention has been described in connection with preferred embodiments, it will be understood that modifications thereof within the principles outlined above will be evident to those skilled in the art, and thus the invention is not limited to the preferred embodiments but is intended to encompass such modifications. The invention resides in each and every novel characteristic feature and each and every combination of characteristic features. Reference numerals in the claims do not limit their protective scope. Use of the verb "to comprise" and its conjugations does not exclude the presence of elements other than those stated in the claims. Use of the article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. 'Means', as will be apparent to a person skilled in the art, are meant to include any hardware (such as separate or integrated circuits or electronic elements) or software (such as programs or parts of programs) which perform in operation or are designed to perform a specified function, be it solely or in conjunction with other functions, be it in isolation or in co-operation with other elements. A 'computer program' is to be understood to mean any software product readable medium, such as a floppy disk, downloadable via a network, such as the Internet, or marketable in any other manner.

Claims

CLAIMS:

1. A data-processing device comprising a sensor for sensing an ambient factor so as to control an operating parameter, wherein the data-processing device comprises at least one further sensor for sensing an ambient factor, and context means for deriving a context of use of the data-processing device by comparing the output of the sensor and the at least one further sensor.

2. A data-processing device as claimed in claim 1, further comprising alert means for generating an alert so as to notify a user of an event, said context means being arranged to control at least one parameter of the alert.

3. A data-processing device as claimed in claim 2, wherein said parameter of the alert is the modality of said alert.

4. A data-processing device as claimed in claim 2, wherein said parameter of the alert is the audio volume of an audio alert.

5. A data-processing device as claimed in claim 2, wherein said parameter is the time and/or duration of an alert.

6. A data-processing device as claimed in claim 1, wherein said sensor and said at least one further sensor are light sensors.

7. A data-processing device as claimed in claim 5, wherein said sensor and said at least one further sensor are located at mutually distant positions.

8. A data-processing device as claimed in claim 6, wherein said sensor and said further sensor are located at a front side and a rear side, respectively.

9. A data-processing device as claimed in claim 1, compri. tilt sensor, a heat sensor, a touch sensor or a microphone, said context means being arranged to partly or entirely derive the context of use from the output of said at least one sensor.

10. A data-processing device as claimed in claim 1, wherein the data-processing device is a remote control, a personal digital assistant or a mobile phone.

11. A computer program product enabling a programmable device when executing said computer program product to function as a data-processing device as defined in any one of claims 1 to 10.

12. A method of determining a context of use of a data-processing device, the method comprising a step of sensing an ambient factor of the data-processing device, wherein the method further comprises a step of comparing an output of at least two sensors connected to the data-processing device for sensing ambient factors.