WO2010102835A1

WO2010102835A1 - Pervasive display system and method for operating a pervasive display system

Info

Publication number: WO2010102835A1
Application number: PCT/EP2010/001588
Authority: WO
Inventors: Martin Strohbach; Gerd Kortuem; Ernoe Kovacs
Original assignee: Nec Europe Ltd.; University Of Lancaster
Priority date: 2009-03-12
Filing date: 2010-03-12
Publication date: 2010-09-16

Abstract

A pervasive display system, including at least one display device for presenting content, is characterized in that the system further includes a content creator being configured to annotate content with contextual presentation goals that specify when, where, and how said content should be presented, a context component being configured to sense and manage context information about humans, objects and/or places as well as about said display devices, and a content distribution and scheduling component being configured to distribute and schedule said content to appropriate display devices of said network on the basis of said annotated contextual presentation goals together with said context information. Furthermore, a method for operating a pervasive display system is disclosed.

Description

PERVASIVE DISPLAY SYSTEM AND METHOD FOR OPERATING A PERVASIVE DISPLAY SYSTEM

The present invention relates to a pervasive display system, including at least one display device for presenting content.

Furthermore, the present invention relates to a method for operating a pervasive display system, said system including at least one display device for presenting content.

Driven by the Digital Signage market, displays are becoming more and more ubiquitous. Display systems are common place in shop displays and in specific environments such as restaurants, pubs, train stations, or airports.

Many commercial solutions exist that are able to schedule content for various environments. All such existing solutions address displays either individually or as comparatively static groups. Usually displays are addressed using identifier or symbolic names. Currently content is selected for specific environments based on a priori demographic knowledge about the environment and scheduling is performed based on time.

It is known that displays can be sensitive to various context information ranging from personalization (e.g. described in J. F. McCarthy et al. "UniCast, Outcast & GroupCast: Three steps toward ubiquitous, peripheral displays", in 3^rd International Conference on Ubiquitous Computing (UbiComp'01 ), pages 332-345, Springer Verlag, Atlanta, Georgia, 2001 ), time of the day, display location and orientation (e.g. described in Ch. Kray et al. "Adaptive navigation support with public displays", in IUI '05: Proceedings of the 10th international conference on Intelligent user interfaces, pages 326-328, ACM, New York, NY, USA, 2005, ISBN 1 -581 13-894- 6) as well as activity in the vicinity of the displays or users (e.g. described in US 2007/0024580 A1 ), wherein most of the known systems focus on specific applications. It is also known how a system can support flexible scheduling approaches and how programming abstractions for scheduling in display networks can be provided. These abstractions are for scheduling in which displays are addressed with symbolic identifiers that identify logical displays that are directly mapped to physical displays or parts of physical displays as described in EP 1 804 158 A1. Therefore content is, as in commercial systems, tightly coupled to concrete displays.

This introduces two severe constraints: (1 ) Changes in the technical infrastructure, the built environment, and properties of the space (e.g. demographic information about the audience) require human intervention to adapt content presentation, like e.g. manual schedule updates, and (2) users can only be addressed indirectly by comparatively static and estimated, or in the best case statistical, knowledge about the physical environment. This indirect addressing suffers from the problem that content cannot be delivered to relevant people on a fine granular basis as characterizing properties of currently present individuals or subgroups (e.g. demographic information and preferences) cannot be taken into account. In addition, it also ignores the dynamic context of potential viewers that are in the vicinity of the displays. In particular, such properties include their location (absolute and relative to displays), orientation, their actual viewing behaviour, their physical activities (e.g. walking, standing, running), the amount of attention for various content types, and their intent with respect to future activities.

In Payne et al. "Auction mechanisms for efficient advertisement selection on public displays", in Proceedings of European Conference on Artificial Intelligence, pages 285-289, 2006 an auctioning mechanism is described in which content advertisers bid to obtain exposure to a maximum number of people using scanned Bluetooth devices. Context information of a single display is used for content scheduling.

US 2006/0287913 A1 discloses a system and method that can be used to dynamically map advertisements to displays based on products that can be bought in the vicinity of displays. The publication focuses on describing properties of the content that are matched against properties of the display. It is therefore an object of the present invention to improve and further develop a pervasive display system and a method for operating such system of the initially described type in such a way that content that is presented on displays can be automatically adapted to the respective audience without requiring any human intervention.

In accordance with the invention, the aforementioned object is accomplished by a system comprising the features of claim 1. According to this claim, such a system is characterized in that the system further includes a content creator being configured to annotate content with contextual presentation goals that specify when, where, and how said content should be presented, a context component being configured to sense and manage context information about humans, objects and/or places as well as about said display devices, and a content distribution and scheduling component being configured to distribute and schedule said content to appropriate display devices of said network on the basis of said annotated contextual presentation goals together with said context information.

Furthermore, the aforementioned object is accomplished by a method comprising the features of independent claim 17. According to this claim, such a method is characterized in the steps of annotating content with contextual presentation goals by means of a content creator, wherein said contextual presentation goals specify when, where, and how said content should be presented, sensing and managing context information about humans, objects and/or places as well as about said display devices by means of a context component, and on the basis of said annotated contextual presentation goals together with said context information, distributing and scheduling said content to appropriate display devices of said network by means of a content distribution and scheduling component.

According to the invention it has been recognized that prior art display systems are disadvantageous in that they are rather static and, in particular, do not reflect the actual audience located in the vicinity of displays. To solve this problem the present invention proposes to annotate content with contextual presentation goals. Contextual presentation goals express in abstract terms when, where and how content should be presented. Furthermore, the display system according to the present invention introduces a context component for gathering context information about humans, objects and/or places on the one hand and about display devices on the other hand. Generally, context is information about entities or groups of entities, wherein an entity may be, but not restricted to, a person, a display, a component of the built environment, a physical space or a device (e.g. a vehicle or a machine). Examples of context attributes include location, identity, sex, size, activity, etc. The term should be understood as by researches and practitioners in the field of mobile, ubiquitous and pervasive computing and is derived from the original definition given by Dey (in Dey et al. "Towards a better understanding of context and context-awareness", Georgia Institute of Technology, 1999).

According to the invention both the annotated contextual presentation goals and the context information is taken into consideration by a content distribution and scheduling component for distributing and scheduling content to appropriate target display devices, i.e. context-based addressing is used to deliver the content. Consequently, content is dynamically mapped to a display on the basis of contextual presentation goals and context information. Thus, the system according to the invention is characterized by the behaviour that it provides a dynamic adaptation of the schedules of each physical display in such a way that it fulfils the contextual presentation goals associated with the content. In this way, content is not bound to specific displays anymore.

With the present invention a commercial digital signage system can be built that better targets individuals and makes the process of content creation more cost effective. Compared to existing solutions the proposed system would have a richer feature set enabling new kind of applications, like for instance better targeted digital advertisement, navigation and way finding. The present invention achieves that the content creation process is decoupled from concrete physical displays and that content can be better targeted to groups and individuals people.

Embodiments of the invention may range from a completely centralized to a full decentralized system without central control. Beyond the typical digital signage environments like airports, shops, pubs/restaurants and exhibitions, we can see an embodiments of the invention in Industrial Sites. Other domains include hotels that are increasingly equipped with displays. Here, digital displays could assist users in case of emergencies, e.g. dynamic guidance cues can be shown after a fire alarm has been raised. Other environments may include entertainment parks. Applications range from more targeted and flexible advertisement, to navigation and guides (entertainment part) to health and safety specific applications.

According to a preferred embodiment the system further includes a network of display devices, each of the display devices being uniquely addressable. The display devices should be suitable to show visually observable annotated content. In this context it is important to note that the term display device as used herein refers to a device that may include several display components with also the display components being individually addressable. For instance, the screen of a display device may be subdivided into specific regions, each of which constituting an individual display component that presents individual content.

Advantageously, after the annotation process, i.e. after the content was tagged with contextual presentation goals, such generated content may be stored in a repository from where it can be retrieved for scheduling. Alternatively, it can also directly be passed to the content distribution and scheduling component for scheduling.

In a specific embodiment the annotated content includes audiovisual content items, for instance pictures and/or video streams with optional audio data. Alternatively or additionally, the annotated content may include any kind of applications, i.e. code that is executable on a display device and that creates audiovisual effects. However, in the context of the present invention the term "content" may be understood in a broader sense; in particular, annotated content may also include an abstract description of audio-visual effects (e.g., show dangerous area) and/or an abstract description of actuation tasks (e.g., move from a to b, turn around, vibrate etc.). Furthermore, content may include a reference to legacy content (e.g. a URI pointing to media content), or even a group of content items. As already mentioned above, a key idea of the invention is to annotate content with contextual presentation goals. Contextual presentation goals express in abstract terms when, where and how content should be presented. A contextual content presentation language could be defined using CALA and the underlying context model, as described for instance in Jacobson, M. et al., "Specifications of PN networking and security components", MAGNET Beyond Public Deliverable D2.3.1, 2007. Presentation goals may contain static information (e.g. profiles and preferences) or dynamic information, in particular related to people or groups of people. For instance, presentation goals may include their demographics, their location (absolute and relative to displays), their actual viewing behaviour, their physical activities (walking, standing, running), the amount of attention people pay to content on displays, and their intent with respect to future activities.

In addition, presentation goals may include information about the built environment in which a display system is located, about an abstract location (e.g., a space centred around a GPS location) or a concrete location (e.g., a building, a public square, a street), and/or about temporal constraints (e.g., after a user group is finished with their meeting). Presentation goals may further contain information about individual display components (e.g., their location, orientation, viewing angle, size), and/or about a collection of display components (e.g. their spatial density, spatial distribution, etc.). This information may be absolute (e.g. GPS-based) or relative (e.g., in local coordinate system, relative to a display or group of displays), specified as geometric V2, 1 , 1 V≥, 2, 2 Vz or 3D shapes or abstract places and zones that define the location on which the content should be displayed.

Presentation goals may implicitly define concrete display candidates by containing any of the following types of presentation goals

• spatio-temporal constraints between individual items of a content group (allows scripting interaction and abstracts from concrete locations of displays)

• Abstract zones

• An identifier that identifies a physical, logical displays, display area or any combination thereof • A virtual identity of a user or user group

• Users described by profiles and preferences

• Users described by dynamically changing context (e.g. activity, location, orientation, appointments, attention, displays being watched)

• feedback information about previously scheduled content (e.g. display at location where this type of content is well suitable, 'content hotspots')

• Historic context information of an entity such as people, places, displays and things

• Lifetime of the content

With respect to an efficient distribution and scheduling of content it proves to be advantageous to take into consideration the fact that in a display network there is typically more content to be presented than display resources are available. Thus, contextual presentation goals may enable the system to determine a set of display candidates of which it can choose to schedule the content. The scheduling process may therefore also takes certain scheduling preferences into account by annotating content with a set of alternative presentation goals that provide the system a degree of freedom for choosing the actual displays. Such sets can be associated (1 ) with priorities and/or (2) with upper cost boundaries that will be used by an auction-based method to schedule the content in the system.

As regards the display devices that are employed for content presentation, many different types can be envisioned. Generally, any device that is able to show digital visual media or visual effects (e.g. light) with at least binary resolution, brightness and colour can be deployed. However, besides these conventional displays it is possible to use logical display components that may consist of parts of a physical display component or be composed of parts of multiple physical display components. As already pointed out in connection with the term "context", also the term "display device" is to be understood in a brought sense. For example, it is possible to employ as display device an actuator that conveys information by changing its human perceivable physical appearance at least in a binary way, e.g. ambient displays such as Weiser's dangling string (in M. Weiser, JS Brown "Designing calm technology", PowerGrid Journal, 1996), robots, etc. Moreover, display devices do not necessarily have to be static, i.e. rigidly mounted in a fixed location, but can be mobile, like robots, PDAs, mobile phones, etc. Even wearable displays (e.g. smart clothes) may be employed as display devices for content presentation.

Generally, the context component acquires, stores and provides access to context information. According to a preferred embodiment the context component is specifically configured to manage knowledge about the location, the orientation and network locators (e.g. URL) of the display devices. Furthermore, the context component may acquire, store and provide access to feedback measurements of scheduled content.

The context component may be implemented as a central context server what would be particularly advantageous in for instance in small settings. In larger systems, as they are envisioned to constitute preferential application scenarios of the present invention, context information may be distributed across various physical devices. In settings in which context will be gathered from mobile device that cannot rely on an external infrastructure the context component may be distributed in a self-organizing P2P network. If a central infrastructure is available, the context component can be implemented by a set of controlling servers and clients that exchange context peer-to-peer, as described in WO 2008/083954 A2.

In the scope of the present invention, context can be static and immutable (e.g. sex of a person) or dynamic (e.g. location of a person). Context information may be known (e.g. sex of a person) or derived from observations of the physical world (e.g. location and activity of a person). Observations in turn may be gathered automatically through sensors attached to entities, e.g. displays or mobile phones, or dispersed throughout an environment. Observations can also be used to determine static context information (e.g. detecting the sex of a person with cameras).

As part of the scheduling process the system has to make sure that only displays are selected that are able to execute the content type specified in the presentation goals. This functionality can be provided by a content mapping component that contains information about whether a concrete display or certain display types are able to show the content or not. If it turns out that content cannot be displayed on a display, an additional functionality may be provided according to which the system is enabled to dynamically deploy execution environments for a certain content type.

According to a specific embodiment the annotation process of content consists of annotating content with contextual presentation goals and annotating content with scheduling preferences. The scheduling process for scheduling the content to be presented may then involve the step of first evaluating the presentation goals and scheduling preferences, and, secondly, the step of determining and selecting a set of concrete display candidates. In a next step the content may be inserted in a schedule, and the content may be displayed on at least one display component. Furthermore, the scheduling process may include the steps of re-evaluating and adapting the schedule based on changes in the environment and removing content from a display component.

Presentation goals are typically evaluated by sending queries and subscriptions to the context component resulting in a set of alternative associations between content and display components. As content is scheduled for a specific display the presentation goals are constantly re-evaluated, e.g. using subscriptions. This evaluation can either be made by an execution environment on displays and/or central components, or code migration can be used in which a software agent moves with the content. In any case each display locally schedules content it receives, potentially also pre-fetching content.

Scheduling content may also include methods for pulling content or pushing content to displays ahead of time (pre-caching) and making local scheduling decisions.

Advantageously, the scheduling process takes presentation goals as well as scheduling preferences into account. In a naϊve approach the contextual presentation goals can first be evaluated to obtain a set of display candidates. In a second step scheduling preferences can be evaluated to schedule the content on an appropriate subset of displays. Scheduling may be based on various forms of random scheduling, round robin scheduling, prioritized first come first serve, or auction-based scheduling.

Depending on its configuration and the presentation goals the content distribution and scheduling component may provide feedback about how content has been scheduled (when, how long, where, why, which presentation goals and scheduling preferences have been used, etc.). In a preferred embodiment this information is provided to the context component allowing content creators (i) to obtain feedback information about specific content and (ii) to perform a better evaluation of presentation goals. Feedback information also may include information about to what degree the objective has been achieved. For advertisement for instance it is important to know how many users acted upon the advertisement.

There are several ways how to design and further develop the teaching of the present invention in an advantageous way. To this end, it is to be referred to the patent claims subordinate to patent claims 1 and 17 on the one hand and to the following explanation of preferred embodiments of the invention by way of example, illustrated by the figure on the other hand. In connection with the explanation of the preferred embodiments of the invention by the aid of the figure, generally preferred embodiments and further developments of the teaching will we explained. In the drawings

Fig. 1 is a high level overview of a pervasive display system according to a first embodiment of the present invention, and

Fig. 2 illustrates a pervasive display system with a central management component and several local display devices according to a second embodiment of the present invention.

Fig. 1 illustrates a pervasive display system according to an embodiment of the present invention that is able to schedule and distribute content to display devices and/or display components of a display network. Fig. 1 gives a high level overview of the pervasive display system. In the following the components included in the embodiment as shown in Fig. 1 are explained in some more detail.

A content creation environment including a content creator (not explicitly shown) creates annotated content with contextual presentation goals and scheduling priorities annotations. Examples of such contextual presentation goals may be, for instance:

• Show content on all displays that can be seen from a certain area at 12pm

• Show content on all displays in a distance of 10m around the ice cream parlor if it is sunny and the temperature is above 25C after school finishes and at least three hours a day with a maximum cost for scheduling of X EUR

• Show red alert in the area where a fork lifting operation is going on

• Show a navigation arrow in the vicinity of the user leading him to his destination

• Show content at a location in the city were at least n people are gathering that are interested in technology

• Show content on displays in the underpass after the news

Additionally, content may be annotated with instructions that tell the system to collect feedback information that should be acquired during scheduling and presenting the content.

After having generated such annotated content, it can either be stored in an annotated content repository component, or it can directly be passed to a context- based content distribution and scheduling component - content scheduler - that distributes and schedules annotated content to appropriate display devices.

Generally, the system according to the present invention includes at least one uniquely addressable display component suitable to show visually observable annotated content under the constraints given by the contextual presentation goals. In the embodiment of Fig. 1 an entire network of such display components is provided. For coordination between the display components a communication infrastructure component is configured between the single display components. The system further includes a context component that senses and manages access to context information both about human, objects and places as well as the display components. In addition, the context component may be in charge of resolving abstract places and zones in coordinate-based, geographic shapes and in evaluating presentation goals. Furthermore, the context component may store static and dynamic information required to distribute and schedule the content (e.g. amount/anticipated presentation time of locally stored content)

As can be obtained from Fig. 1 , the system includes a content mapping component - content mapper - that maps content types, including abstract content descriptions, to display types or individual displays. By this means the system can make sure that only displays are selected that can execute abstract content descriptions.

Fig. 2 illustrates another potential embodiment of the invention, in which a central PDN (Pervasive Display Network) Manager and several local displays exist. The PDN Manager contains the Content Management System that manages content annotated with contextual presentation goals and scheduling priorities (essentially Content Creation Environment and Content Repository in Fig. 1 ). A mapping component analyzes the content based on the contextual presentations goals, the scheduling priorities, and the given context of the total PDN. It then forwards selected content objects to a content distribution component. That component pushes the content to the target display. Here the content is stored in the cache and the local scheduler presents the content based on the defined schedule as well as on contextual events.

To summarize, advantages of the present invention are to be seen in the fact that content can be scheduled based on high level abstractions. It allows for a targeted delivery of content on a fine granular basis including dynamic properties. Furthermore, it reduces the effort for content creators, as content is adaptive to context of displays, schedules, time, users and the physical space no manual intervention is required for rescheduling content. A limitation of the system may be imposed by the amount of available context information. For some desirable presentation goals it may not technically be possible to sense information reliably enough. Another limitation may be related to scheduling problems, i.e. if more content needs to be scheduled than display resources available. The present invention uses priority-based scheduling and auctioning mechanisms for selecting the content.

Many modifications and other embodiments of the invention set forth herein will come to mind the one skilled in the art to which the invention pertains having the benefit of the teachings presented in the foregoing description and the associated drawings. Therefore, it is to be understood that the invention is not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

C l a i m s

1. Pervasive display system, including at least one display device for presenting content, c h a r a c t e r i z e d i n that the system further includes a content creator being configured to annotate content with contextual presentation goals that specify when, where, and how said content should be presented, a context component being configured to sense and manage context information about humans, objects and/or places as well as about said display devices, and a content distribution and scheduling component being configured to distribute and schedule said content to appropriate display devices of said network on the basis of said annotated contextual presentation goals together with said context information.

2. System according to claim 1 , further including a network of display devices, each of said display devices being uniquely addressable.

3. System according to claim 1 or 2, further including a repository for storing said annotated content.

4. System according to any of claims 1 to 3, wherein said annotated content includes audiovisual content items and/or an application that creates audiovisual effects.

5. System according to any of claims 1 to 4, wherein said annotated content includes an abstract description of audio-visual effects and/or actuation tasks.

6. System according to any of claims 1 to 5, wherein said contextual presentation goals include static and/or dynamic information.

7. System according to any of claims 1 to 6, wherein said contextual presentation goals include information about people or groups of people.

8. System according to any of claims 1 to 7, wherein said contextual presentation goals include information about the built environment in which said display system is located.

9. System according to any of claims 1 to 8, wherein said contextual presentation goals include information about individual display devices and/or a collection of display devices.

10. System according to any of claims 1 to 9, wherein said content distribution and scheduling component is configured to take scheduling preferences into account, in particular a prioritization of said contextual presentation goals, auctioning strategies and/or upper cost boundaries.

1 1. System according to any of claims 1 to 10, wherein said display devices include an actuator being configured to convey information by changing is physical appearance at least in a binary way.

12. System according to any of claims 1 to 11 , wherein said context component is configured to manage knowledge about the location, the orientation and network locators of said display devices.

13. System according to any of claims 1 to 12, wherein said context information is stored in a distributed way across various physical devices.

14. System according to any of claims 1 to 13, wherein said context information includes static and/or dynamic information.

15. System according to any of claims 1 to 14, wherein said context information is derived from observations automatically obtained through sensors.

16. System according to any of claims 1 to 15, further including a content mapping component for mapping content types to display types and/or individual displays.

17. Method for operating a pervasive display system, in particular a system according to any of claims 1 to 16, said system including at least one display device for presenting content, c h a r a c t e r i z e d i n the steps of annotating content with contextual presentation goals by means of a content creator, wherein said contextual presentation goals specify when, where, and how said content should be presented, sensing and managing context information about humans, objects and/or places as well as about said display devices by means of a context component, and on the basis of said annotated contextual presentation goals together with said context information, distributing and scheduling said content to appropriate display devices of said network by means of a content distribution and scheduling component.

18. Method according to claim 17, wherein said content distribution and scheduling component takes into account scheduling preferences, in particular a prioritization of said contextual presentation goals, auctioning strategies and/or upper cost boundaries.

19. Method according to claim 18, wherein said scheduling includes the steps of evaluating said contextual presentation goals and said scheduling preferences, and determining and selecting a set of concrete candidates of appropriate display devices.

20. Method according to claim 19, wherein said scheduling further includes the steps of inserting said content in a schedule, and displaying said content on at least one display device of said set of concrete candidates of appropriate display devices.

21. Method according to claim 19 or 20, wherein said scheduling further includes the step of re-evaluating and adapting said schedule based on changed context.

22. Method according to any of claims 19 to 21 , wherein said scheduling further includes the step of removing content from a display device after having been presented.

23. Method according to any of claims 17 to 22, wherein content is additionally annotated with the instruction to collect feedback information about how content has been scheduled.

24. Method according to claim 23, wherein said content distribution and scheduling component provides said feedback to said context component.