GB2512626A - Interface mechanism for massive resolution displays - Google Patents

Interface mechanism for massive resolution displays Download PDF

Info

Publication number
GB2512626A
GB2512626A GB1306077.7A GB201306077A GB2512626A GB 2512626 A GB2512626 A GB 2512626A GB 201306077 A GB201306077 A GB 201306077A GB 2512626 A GB2512626 A GB 2512626A
Authority
GB
United Kingdom
Prior art keywords
layout
widget
display
user interface
display panel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB1306077.7A
Other versions
GB201306077D0 (en
GB2512626B (en
Inventor
Paul Harding
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Synamedia Ltd
Original Assignee
NDS Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NDS Ltd filed Critical NDS Ltd
Priority to GB201306077A priority Critical patent/GB2512626B/en
Publication of GB201306077D0 publication Critical patent/GB201306077D0/en
Priority to PCT/IB2014/060127 priority patent/WO2014162239A2/en
Publication of GB2512626A publication Critical patent/GB2512626A/en
Application granted granted Critical
Publication of GB2512626B publication Critical patent/GB2512626B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G06F3/1446—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display display composed of modules, e.g. video walls
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H04N21/41265—The peripheral being portable, e.g. PDAs or mobile phones having a remote control device for bidirectional communication between the remote control device and client device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G06F3/1438—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display using more than one graphics controller
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14—Display of multiple viewports
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42206—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor characterized by hardware details
    • H04N21/42208—Display device provided on the remote control
    • H04N21/42209—Display device provided on the remote control for displaying non-command information, e.g. electronic program guide [EPG], e-mail, messages or a second television channel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42206—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor characterized by hardware details
    • H04N21/4222—Remote control device emulator integrated into a non-television apparatus, e.g. a PDA, media center or smart toy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H04N21/42206—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor characterized by hardware details
    • H04N21/42224—Touch pad or touch panel provided on the remote control
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4314—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/482—End-user interface for program selection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/06—Adjustment of display parameters
    • G09G2320/0606—Manual adjustment
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0464—Positioning
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/14—Solving problems related to the presentation of information to be displayed
    • G09G2340/145—Solving problems related to the presentation of information to be displayed related to small screens
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00—Aspects of interface with display user
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/06—Consumer Electronics Control, i.e. control of another device by a display or vice versa
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/20—Details of the management of multiple sources of image data

Abstract

A system (see block diagram in figure 7) provides user control of the presentation and layout of network widgets on to a large multiple panel array 110 (710) from a remote companion device 300, such as a smart-phone (770) or tablet computer. The display array may be a video wall in a domestic dwelling (figure 1). The system includes a plurality of display adaptors (720) each driving at least one display panel of the array and controlled by a layout engine (730) that arranges one or more user interface widgets on the array, based on at least metadata published by each widget, a current widget layout and the display area of the array. A system controller (740) manages runtime context for each widget, which includes output produced from multimedia input received by at least one network adapter (750). The system controller sends information on the layout and widget metadata describing layout constraints to a management application on the remote companion device so that a user may command the device to cause the system controller to act on one or more of : a widget command, a widget selection and a layout change. The companion device may be in wireless communication (760) and show a representation of the widget layout, and optionally widget content, on a local screen and respond to touch gestures from a user (figures 4, 5). The widgets may comprise a TV feed, local domestic video and security data, and internet data such as weather forecasts, news feeds and social media feeds. The system can provide for a tablet computer to interface with a domestic video wall.

Description

INTERFACE MECHANISM FOR MASSIVE RESOLUTION DISPLAYS
FIELD OF THE INVENTION
The present invention relates to controllers for video displays, and particularly to controllers for massive resolution video displays.
BACKGROUND OF THE INVENTION
Massive resolution displays, typically comprised of either a single or multiple physical panels, are decreasing in cost each year. It is envisaged that displays of a magnitude to cover the entire wall of a room -or a significant proportion of the entire wall of a room -will become commonplace in homes.
In such an environment, the display would offer the potential to be configured to show a number of "widgets" in different areas, such as (but not limited to): live TV, social networking feeds, a Flickr photostream, and upcoming reminders/appointments.
It is appreciated that the term "widget", in all of its grammatical forms, as used in the present specification and claims is understood to refer to an element of a graphical user interface (i.e. the display referred to, above) that displays an information arrangement changeable by the user (see the list of exemplary widgets, above). The defining characteristic of a widget is to provide a single interaction point for the manipulation of a given kind of data. Widgets may be thought of as basic visual building blocks which, combined in an application, hold all the data processed by the application and the available interactions on this data.
As such, the display becomes a shared resource which can be used and enjoyed by all members of the household as an "entertainment hub", simultaneously.
However, having a graphical user interface of such scale renders existing interaction solutions (enumerated below) impractical, or inefficient.
* Conventional JR remote control: the interaction possible is too limited for many tasks such as typing text.
* Direct touch & Mobile touch (using near field communication [NFC]): the display covers too large a surface area for this to be practical without inducing human fatigue; the solution needs to be "lean-back". Also screen touch would be a distraction or obstruction for other users in the room.
* Motion tracking I gestures: too much of a distraction for other users in the room.
* Wand / pointing devices: not immediate enough to get results, and not flexible enough to support all the different interactions and operations required by multitude of different widgets.
* Voice control: Too much of a distraction for other users in the room.
The following non-patent and patent literature is believed to reflect the state of the art: A Remote Control Interface for Large Displays, Azam Khan, et al., available at www.autodeskresearch.comlpdf/2004UlSTFrisbee.pdf; A Comparative Study of Interaction Metaphors for Large-Scale Displays, Ricardo Jota, et al. CHI 2009, Spotlight on Works in Progress, Session 2; Literature Survey on Interaction Techniques for Large Displays, Brian Badillo, et al. available at people.cs. vt.edu/-bowmanlcsô724/large_display_survey.pdf; Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch-and Pen-Operated Systems, Patrick Baudisch, et al, available at research.microsoft.comlen-us/um!peoplelcutrell/DragAndPop_Interact2003. pdf; Exploring Pen and Paper interaction with High-Resolution Wall Displays, Nadir Weibel, et al.. available at ucsd.academia.edu/weibel/PapersI2275 87/Exploring_Pen_and_Paper_lnteraction_ with_High-Resolution_Wall_Displays; A Survey of Interaction Techniques and Devices for Large High Resolution Displays, Taimur K. Khan, available at drops.dagstuhl.de/opuslvolltexte/20 1 1130941pdf15.pdf;
I
Natural throw and tilt interaction between mobile phones and distant displays, Raimund Dachselt. et al., available at dl.acm.org/citation.cfm?id= 1520340.1520467; Touch & Interact: Touch-based Interaction of Mobile Phones with Displays, Robert Hardy, et al.. available at eprints.lancs.ac. ukl 1 3090/l/mobilehci2008_hardy.pdf; Fluid interaction with high-resolution wall-size displays, François Guimbretière, et al. In Proceedings of the 14th annual ACM symposium on User interface software and technology (UTST 01). ACM, New York, NY, USA, 2 1-30; Courtyard: integrating shared overview on a large screen and per-user detail on individual screens, Masayuki Tani, et al. In Proceedings of the SIUCHI conference on Human factors in computing systems: celebrating interdependence (CIII 94), Beth Adelson, Susan Dumais, and Judith Olson (Eds.).
ACM, New York, NY, USA, 44-50; and A toolset for creating iconic interfaces for interactive workspaces, Jacob T. Biehl, et al., In Proceedings of the 2005 IFIP ICl3 international conference on Human-Computer Interaction (INTERACT'OS). Maria Francesca Costabile and Fabio PaternO (Eds.). Springer-Verlag, Berlin, Heidelberg, 699-712.
US 6,832,353 to Itavaara et al.; US 7,634,780 to Rhoten et al.; US 2009/247222 of Bonnat; and
SUMMARY OF THE INVENTION
The present invention, in certain embodiments thereof, seeks to provide an improved interface mechanism for massive resolution displays.
The embodiments of the present invention described herein enable mirroring of widget positions from a massive resolution display to an app running on a companion device. Minoring of widget positions enables last locating and selection of objects for users by leveraging spatial awareness and recognition.
Additionally, mirroring of widget branding and the display of headline status information enables fast locating and selection of objects for users by leveraging visual cues.
Furthermore, interaction using standard gestures on multitouch interfaces ensures a minimal user learning curve to operate the system.
Moreover, a user's local interactions and selections are being tracked and logically duplicated on the massive resolution display, which thus enables some basic user interface functions to be operated without looking down at the control app, by users familiar with the control interface. For example, without looking down at the control app, the user is able to perform widget selection, widget movement, and trigging a refresh or page scroll of widget data.
There is thus provided in accordance with an embodiment of the present invention a system for controlling a display panel array, the display panel array including a plurality of display panels and being operative to display a plurality of user interface widgets, the system including a plurality of display adaptors, each one of the plurality of display adaptors controlling a layout of an output on at least one display panel of the display panel array, a layout engine for arranging each of the plurality of user interface widgets on the display panel array, the layout engine arranging each of the plurality of user interface widgets on the display panel array by utilizing at least metadata published by each of the plurality of user interface widgets, the metadata including constraints on a user interface widget aspect ratio or orientation, a layout of the plurality of user interface widgets presently in use on the display panel array, and an area of the display panel array, the layout engine being operative to split a plurality of outputs among each one of the display adaptors, the plurality of outputs including data which each of the display adaptors requires in order to display the layout, thereby producing the desired layoLtt on the display panel array, a system controller which receives a multimedia input from at least one network adapter, the system controller managing runtime context for each of the plurality of user interface widgets, the system controller operative to produce an output including at least a portion of the multimedia input for each one of the plurality of ii ser interface widgets, the system controller being operative to provide the produced output, as the layout of the plurality of user interface widgets presently in use, to the layout engine, the system controller being in bi-directional communication with a companion device external to the system, the companion device including one of a computer or processing device running a management application in bi-directional communication with the system controller, wherein the system controller sends information on the layout and metadata about layout constraints for each of the plurality of user interface widgets to the management application, such that when a user of the companion device interacts with the management application, the management application sends at least one of layout changes, widget selection notifications, and widget commands to the system controller for relay to the system controller, thereby causing at least one of layout changes, widget selection notifications, and widget commands to be rendered on the display panel array, and the plurality of display adaptors, the layout engine, and the system controller are each implemented on at least one of a computer or processing device.
Further in accordance with an embodiment of the present invention the area and configuration of the display panel array is stored by the layout engine during one of installation of the system, and a system update.
Still further in accordance with an embodiment of the present invention the management application provides visual cues to the user of the companion device, thereby enabling fast locating and selection of objects.
Additionally in accordance with an embodiment of the present invention the plurality of user interface widgets presently in use on the display panel array is mirrored by a layout of thumbnail representations of the plurality of user interface widgets in the management application.
Moreover in accordance with an embodiment of the present invention each of the thumbnail representations appear proportionally in the same size and position as its corresponding user interface widget as the corresponding user interface widget appears on the display panel array.
Further in accordance with an embodiment of the present invention each of the thumbnail representations appears in the management application with a lower graphical resolution than each corresponding user interface widget appears on the display panel array.
Still further in accordance with an embodiment of the present invention each of the thumbnail representations displays headline status information within its boundaries.
Additionally in accordance with an embodiment of the present invention the user of the companion device interacts with the management application using standard gestures.
Moreover in accordance with an embodiment of the present invention the standard gestures include a pinch and zoom gesture, and a drag gesture.
Further in accordance with an embodiment of the present invention the metadata published by the user interface widget is published via a common application programming interface.
Still further in accordance with an embodiment of the present invention the management application is operative to enter a tracking mode, causing a selected one of the plurality of user interface widgets to be highlighted on the disp'ay panel array.
There is a'so provided in accordance with another embodiment of the present invention a method for controlling a display panel array, the display panel array including a plurality of display panels and being operative to display a plurality of user interface widgets, the method including controlling a layout of an output on at least one display panel of the display panel array at each one of a plurality of display adaptors, arranging each of the plurality of user interface widgets on the display panel array by a layout engine, by utilizing at least metadata published by each of the plurality of user interface widgets, the metadata including constraints on a user interface widget aspect ratio or orientation, a layout of the plurality of user interface widgets presently in use on the display panel array, and an area of the display panel array, the layout engine being operative to split a plurality of outputs among each one of the display adaptors, the plurality of outputs including data which each of the display adaptors requires in order to display the layout, thereby producing the desired layout on the display panel array, receiving at a system controller a multimedia input from at least one network adapter, the system controller managing runtime context for each of the plurality of user interface widgets, the system controller operative to produce an output including at least a portion of the multimedia input for each one of the plurality of user interface widgets, the system controller being operative to provide the produced output, as the layout of the plurality of user interface widgets presently in use, to the layout engine, communicating bi-directionally between the system controller and a companion device external to the system, the companion device including one of a computer or processing device running a management application in bi-directional communication with the system controller, wherein the system controller sends information on the layout and metadata about layout constraints for each of the plurality of user interface widgets to the management application, such that when a user of the companion device interacts with the management application, the management application sends at least one of layout changes, widget selection notifications, and widget commands to the system controller for relay to the system controller, thereby causing at least one of layout changes, widget selection notifications, and widget commands to be rendered on the display panel array, and the plurality of display adaptors, the layout engine, and the system controller are each implemented on at least one of a computer or processing device.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention will be understood and appreciated more fully from the following detailed description, taken in conjunction with the drawings in which: Fig. 1 is a simplified pictorial illustration of a massive resolution display controlled by a mechanism constructed and operative in accordance with an embodiment of the present invention; Fig. 2 is a depiction showing widget types identified for the display configuration of Fig. I; Fig. 3A is a depiction of a thumbnail layout, referred to as a viewport, on a companion device for the display configuration shown in Figure 1; Fig. 3B is a depiction of a large scale widget and its corresponding thumbnail; Fig. 4 is a depiction of the use of a "pinch and zoom" gesture in order to zoom in to the viewport of Fig. 3A; Fig. 5 is a depiction of a method for using a drag gesture in order to scroll the viewport of Fig. 3A; Fig. 6 is a depiction of the massive resolution display of Fig. 1 with the system of Fig. 1 in tracking mode; Fig. 7 is a system diagram depicting an embodiment of an implementation of the interface mechanism of Fig. 1; and Fig. 8 is a simplified flowchart diagrams of methods of operation of the system of Fig. 1.
DETAILED DESCRIPTION OF AN EMBODIMENT
Reference is now made to Fig. 1, which is a simplified pictorial illustration of a massive resolution display controlled by a mechanism constructed and operative in accordance with an embodiment of the present invention.
Fig. 1 depicts two users bOA, 100B seated before a massive resolution display 110. The massive resolution display 110 is depicted as filling an entire wall of a room, however, the massive resolution display 110 may be smaller than such a wall, or, in principle, may "bend" around the corner of a room, occupying at least a portion of two separate walls (although, in principle there is no reason to limit the massive resolution display 110, which might also be on the ceiling, or even, in some structures, on at least a portion of a floor).
Various widgets appear on the massive resolution display 110.
Reference is "ow additionally made to Fig. 2, which is a depiction showing a widget types identified for the display configuration of Fig. 1. Thus, from right to left, the massive resolution display 110 is depicted as displaying a weather widget 120, 220; a home security widget 130, 230; a TV sidebar widget 140, 240; a live TV widget 150, 250; and a social networking widget 160, 260.
Those of skill in the art will appreciate that the massive resolution display 110 typically comprises a plurality of display panels which are arranged to appeal as a single massive display. The plurality of user interface widgets 120, 130, 140, 150, 160, 220, 230, 240, 250, 260 depicted in Figs. 1 and 2 are depicted by way of example. Any other appropriate user interface widget may appear, or, alternatively, one of the user interface widgets 120, 130, 140, 150, 160, 220, 230, 240, 250, 260 depicted in Figs. I and 2 may not appear in actual usage.
The components comprising the system of Figs. I and 2 are described below with reference to Fig. 7.
Reference is now made to Fig. 3A, which is a depiction of a thumbnail layout, referred to as a viewport 310, on a companion device 300 for the display configuration shown in Figure 1. The viewport 310 comprises an app depicted as appearing on the companion device 300 (the term "app" is a common short hand term for an "application", and is commonly used to refer to a specialized program downloaded onto a device, often, but not limited to a handheld or mobile device -for example, and without limiting the generality of the foregoing, a tablet compLiting device, or a smartphone).
The term "thumbnail" as used herein, in all of its grammatical forms, is understood to refer to a reduced-size image or representation of a corresponding widget on the massive resolution display. A number of thumbnails are depicted on the viewport 310 on the companion device 300. (For instance, a calendar widget on the massive resolution display will have a corresponding calendar thumbnail on the companion device.) The term "companion device" as uscd herein, in all of its grammatical forms, is understood to refer to an additional electronic device (for example, and without limiting the generality of the foregoing, a tablet computing device, or a smartphone) that allows a viewer of the massive resolution display to interact with any of the massive resolution display 110 itself or with a'iy of the widgets displayed thereon.
The companion device 300 of Fig. 3A is depicted running an app which displays the viewport 310 on the screen of the companion device 300. The viewport 310 comprises a thumbnail layout on which a representation of one of a complete or a partial section of the massive resolution display 110 appears. For example, the thumbnail images appearing on the viewport 310 (i.e. the weather thumbnail 320; the home security thumbnail 330, TV sidebar thumbnail 340, the live TV thumbnail 350, and the social networking thumbnail 360) correspond, respectively, to the weather widget 120, 220; a home security widget i30. 230; a TV sidebar widget 140, 240; a live TV widget 150, 250; and a social networking widget 160, 260 ofFigs. I and2.
The thumbnail images on the viewport 310 are rendered such that all objects appear proportionally having the same size and in the same position as the corresponding objects appear on the massive resolution display 110. However, due to the substantially lower resolution available on the companion device, graphical detail which appears on the massive resolution display 110 is not shown on the viewport 310. For example, and without limiting the generality of the foregoing, a widget displayed on the massive resolution display might show a I0 cluster of images in full, while the corresponding thumbnail representation of the widget might be displayed as an outline of the cluster.
As depicted in Fig. 3A, each of the thumbnails images appearing on the viewport 310 corresponding to each widget 120, 130, i40, 150, 160, 220, 230, 240, 250, 260 depicted in Figs. 1 and 2 displays headline status information within its boundaries, which information is specific and pertinent to the type of widget represented. For example and without limiting the generality of the foregoing, the live TV thumbnail 350 corresponding to a live TV widget 150 could show the current channel to which the live TV widget is tuned. The headline status information displayed comprises a presentation of the elements considered by the system to be the most important elements from a user's perspective of the widget's current state. For example, and without limiting the generality of the foregoing, the headline status information displayed comprises a subset of and in some cases complement to the information displayed on the main widget.
The live TV thumbnail 350 could also display now and next programming information. Also, the live TV thumbnail 350 may be branded or styled in such a way as to be visually similar at a glance to its larger counterpart on the remote display, thus to provide a further associational link.
The home security thumbnail 330 corresponding to the home security widget 130 may display a list of currently displayed home security cameras currently displayed in the home security widget 130, as well as a list of other home security cameras which are available to display in the home security widget 130.
Other exemplary widget types might include: a calendar widget. The thumbnail (not depicted) of the calendar widget may display the current day displayed on the widget and details of a next appointment.
a social networking widget. The thumbnail 360 of the social networking widget may display the logged-in user's news feed, the date/time data last refreshed, and a list of available sources which can be switched onloff to filter the widget data (e.g. Facebook, Twitter, Instagram, and so forth).
II
Reference is now made to Fig. 3B, which is a depiction of a large scale widget 370 and its corresponding thumbnail 410. Large scale widgets 370 on the massive resolution display 110 may be comprised of a number of panels 380, 385, 390, which have logically different operations available to be performed on them. In such cases, its corresponding thumbnail 410 will paint the outline of the panels 480, 485, 490 in proportionally the same positions as in the full scale widget 370, to facilitate quick recognition and selection by the user bOA, bOB.
Specifically, Fig. 3B depicts a social network widget 370 as it may appear on the massive resolution display 110. The social network widget 370 is depicted, in a non-limiting fashion, as comprising three panels: a panel comprising a feed of socially shared photos 380, a panel comprising a socially shared status update feed 385, and a panel comprising a view of linked content 390. The corresponding thumbnail 410 also comprises three corresponding panels, a photo stream thumbnail 480, a status stream thumbnail 485, and a linked content thumbnail 490.
Reference is now made to Fig. 4, which is a depiction of the use of a "pinch and zoom" gesture in order to zoom in to the viewport 310 of Fig. 3A. (It is appreciated that the "opposite" gesture would result in a zoom out.) The viewport 310 is zoomed in or out using a two finger pinch and drag, as is known in the art, and often referred to as a pinch and zoom gesture. The live TV thumbnail 350, which is depicted in Fig. 4 as being the widget which is zoomed in on appears in the center of Fig. 4, enlarged in accordance with the zoom. The TV sidebar thumbnail 340 appears on the right side of the viewport 310. Since the TV sidebar thumbnail 340 is also zoomed, the text (i.e. the headline status information as described above) which is displayed in the TV sidebar thumbnail 340 appears cut off A fragment of the social networking thumbnail 360 also appears, on the left side of the viewport 310.
Reference is now made to Fig. 5, which is a depiction of a method for using a drag gesture in order to scroll the viewport 310 of Fig. 3A. The viewport 310 is scrolled in x, y, or both directions using a single finger flick or drag gesture, as is known in the art.
Selection of a thumbnail object, for example, the social networking thumbnail 360, as depicted in Fig. 5, in order to perform operations on the selected thumbnail object is accomplished by a single tap on the thumbnail while the viewport 310 is static (i.e. not still rendering scrolling motion as a result of a swipe or pinch and zoom gesture).
In some embodiments of the present invention, upon selection of one of the thumbnails 320, 330, 340, 350, 360, the viewport 310 will be automatically zoomed in to the right detail level to display the selected one of the thumbnails 320, 330, 340, 350, 360, in order to ensure that the headline status information for the selected one of the thumbnails 320, 330, 340, 350, 360, is fully visible and legible. Selection of one of the thumbnails 320, 330, 340, 350, 360, will also render a "highlight" around the selected one of the thumbnails 320, 330, 340, 350, 360, in order to clearly indicate that the selected one of the thumbnails 320, 330, 340, 350, 360, is in the selected state.
In some embodiments of the present invention, when a thumbnail corresponding to a "simple" widget is selected, the thumbnail will render options available to the user bOA. 10DB within the thumbnail boundary, thereby enabling quick selection. (It is appreciated that the term "simple" widget is intended as a verbal construct, referring to a widget which meets this requirement.
On the other hand, when a thumbnail corresponding to a "complex" widget is selected (refer to Fig. 3B, and the description thereof), the thLtmbnail selected will render a button enabling the user bOA, IDOB to access a full-detail view of the thumbnail. The full-detail view of the thumbnail may fill the entire screen on the companion device. Navigating back to the massive resolution display 110 display viewport 310 view is achieved with the standard back button on the companion device 310, or, alternatively, with a swipe right gesture. When the user bOA. 10DB has access to the full-detail view of the thumbnail, the user 1 QUA. 10DB has access to the complete feature set or configuration (if relevant) of the selected widget. By way of a non-limiting example, a live TV widget, this would be deemed a complex widget because the live TV widget needs a full screen to show an EPG grid, in the thumbnail view, the live TV widget might simply display now/next info for favorite channels.
Reference is now made to Fig. 6, which is a depiction of the massive resolLttion display of Fig. 1 with the system of Fig. 1 in tracking mode.
Those skilled in the art will appreciate that for some tasks, it may be preferable for the user 1 QUA, 1 DUB to interact with the app while remaining focused on the massive resolution display 110. (Having to move eyes between the screens of the massive resolution display 110 and companion device 300 excessively would be inefficient, and would likely detract from the user experience). Hence, to facilitate effective navigation in this mode, as the user bOA, IDOB scrolls around the viewport 310 on companion device 330, the widget covering the center of the viewport 310 is highlighted (using a thicker border or other visual cue) on the massive resolution display 110. In this case, the live TV widget 150 is highlighted and the live TV thumbnail 350 covers the center of the viewport 310. Widget selection is implicit (i.e. a thumbnail is centered on the viewport, and the corresponding widget is selected based on no other user action), based on viewport 310 position and zoom -no tap is requ red. Hence, the full screen widgets mirror the selection on the viewport 310. This allows users to swipe and scroll on the companion device 330 while looking up at the massive resolution display 110, seeing the current widget selection moving as it is being tracked on massive resolution display 110.
This "tracking" feature is typically not the default system behavior because of the potential distraction to other individuals in the room watching elements of which appear on the massive resolution display 110, as the user iUOA, 11DB of the viewport 310 app navigates around the larger surface (i.e. the massive resolution display 110). Hence, it is a feature optionally switched on by the user IDDA. 10DB or when deemed appropriate by a display controller based (see the discussion below, with reference to Fig. 7) on the current widget layout.
It is appreciated, however, that when operating using this "tracking feature" the system allows basic manipulation of the selected widget without looking at the companion device, via further gestures on the companion device app, including, bitt not limited to: -Hold two fingers and move: to move the widget location and have the layout engine reflow the layout; -Hold two fingers and drag to very top of screen and hold there: make the widget it's maximal size across the display panel array and hide all other widgets; -Hold two fingers and drag to very bottom of screen and hold there: to remove the widget from the display; and -Fast swipe down then up with one or two fingers: to refresh the widget content immediately.
Reference is now made to Fig.7, which is a system diagram depicting an embodiment of an implementation of the interface mechanism of Fig. 1. The components comprising the system 700 of Figs. 1 and 2 are now described.
The massive resolution display 110, as was noted above, typically comprises an array of display panels 710. The array of display panels 710 is operative to display a plurality of user interface widgets. For example, and without limiting the generality of the foregoing, the weather widget 120; the home security widget 130; the TV sidebar widget 140; the live TV widget 150; and the social networking widget 160 are examples of typical user interface widgets which might be displayed on the array of display panels 710.
A plurality of display adaptors 720 implemented on at least one of a computer or processing device are in wired HDMI communication (or other appropriate communication protocol) with the array of display panels 710. Each one of the display adaptors of the plurality of display adaptors 720 control the display of an oLttpLtt on at least one display panel of the display panel array 710.
A layout engine 730 implemented on one of a computer or processi rig device arranges each of the plurality of user interface widgets 120, 130, 140, 150, 160 on the array of display panels 710. The layout engine 730 utilizes at least the following information in determining the layout of the plurality of user interface widgets 120, 130, 140, 150, 160 on the array of display panels 710: metadata published by each of the plurality of user interface widgets 120, 130, 140, 150, 160 via a common interface, the common interface comprising a programmatic interface or a static manifest file, the metadata including, but not limited to display sizes andlor aspect ratios, and orientations which are supported on the output display array. These constraints must be respected and adhered to by the layout engine 730 in determining the layout of the overall interface; a layout of the plurality of user interface widgets 120, 130, 140, 150, 160 presently in use on the display panel array 710; and an area (i.e. the length x width) of the display panel array 710.
The layout engine 730 is operative to split a plurality of outputs among each one of the display adaptors 720. The plurality of outputs comprise data which each of the display adaptors 720 requires in order to display the layout, thereby producing the desired layout on the display panel array 710.
A system controller 740 implemented on one of a computer or processing device receives a multimedia input from at least one network adapter 750. For example and without limiting the generality of the foregoing, the network adapter 750 may input content for the various widgets, the content being provided via TCP/iP from various internet services 755.
The system controller 740 manages runtime context (i.e. a sandboxed environment in which each widget process executes, that is to say the widgct's code, data, its input/output, and current state) for each of the plurality of user interface widgets 120, 130, 140, 150, 160. The system controller 740 is further operative to produce an output comprising at least a portion of the multimedia input for each one of the plurality of user interface widgets 120, 130, 140, 150. 160, the system controller being operative to provide the produced output to the layout engine 730.
The system controller 740 via the network adaptor 750 is further in bi-directional WIFI communication with via a WTFI module 760, utilizing either a wireless IEEE 802.11 home network; a Bluetooth connection, or other appropriate communication protocol, with the companion device 770. The companion device 770 is running a management application 780, such as the viewport app 310, described above.
The system controller 750 sends information on the layout and metadata about layout constraints for each of the plurality of user interface widgets to the management application 780 on the companion device 770.
When a user bOA, 11DB of the companion device 770 interacts with the management application 780 (such as the viewport app 310 described above), the management application 780 sends at least one of: layout changes; widget selection notifications; and widget commands to the system conifoller 740 for relay to the layout engine 730, thereby causing the least one of: layout changes; widget selection notifications; and widget commands to be rendered on the display panel array 710.
The area and configuration of the display panel array 710 is stored by the layout engine 730 during one of: installation of the system 700; and during a system 700 update. The system 700 update may be a software update in which any of the software components which are described herein as a part of the system 700 are updated, or alternatively, the system 700 update may be a hardware update, in which new hardware (for example, and without limiting the generality of the foregoing, new display panels may be added to the display panel array 710) is added, or already existing hardware is either replaced or upgraded.
Reference is now made to Fig. 8, which is a simplified flowchart diagram of preferred methods of operation of the system of Fig. I. The method of Fig. 8 is believed to be self-explanatory in light of the above discussion.
It is appreciated that software components of the present invention may, if desired, be implemented in ROM (read only memory) form. The software components may, generally, be implemented in hardware, if desired, using conventional techniques. It is further appreciated that the software components may be instantiated, for example: as a computer program product or on a tangible medium. In some cases, it may be possiNe to instantiate the software components as a signal interpretaNe by an appropriate computer, although such an instantiation may be excluded in certain embodiments of the present invention.
It is appreciated that various features of the invention which are, for clarity, described in the contexts of separate embodiments may also be provided in combination in a single embodiment. Conversely, various features of the invention which are, for brevity, described in the context of a single embodiment may also be provided separately or in any suitable subcombination.
It will be appreciated by persons skilled in the art that the present invention is not limited by what has been particularly shown and described hereinabove. Rather the scope of the invention is defined by the appended claims and equivalents thereof:

Claims (12)

  1. What is claimed is:CLAIMS1. A system for controlling a display panel array, the display panel array comprising a plurality of display panels and being operative to display a plurality of user interface widgets, the system comprising: a plurality of display adaptors, each one of the plurality of display adaptors controlling a layout of an output on at least one display panel of the display panel army; a layout engine for arranging each of the plurality of user interface widgets on the display panel array, the layout engine arranging each of the plurality of user interfttce widgets on the display panel array by utilizing at least: metadata published by each of the plurality of user interface widgets, the metadata comprising constraints on a user interface widget aspect ratio or orientation; a layout of the plurality of user interfhce widgets presently in use on the display panel array; and an area of the display panel array, the layout engine being operative to split a plurality of outputs among each one of the display adaptors, the plurality of outputs comprising data which each of the display adaptors requires in order to display the layout, thereby producing the desired layout on the display panel array; a system controller which receives a multimedia input from at least one network adapter, the system controller managing mntime context for each of the plurality of user interface widgets, the system controller operative to produce an output comprising at least a portion of the multimedia input for each one of the plurality of user interface widgets, the system controller being operative to provide the produced output, as the layout of the plurality of user interface widgets presently in rise, to the layoLlt engine; the system controller being in bi-directional communication with a companion device external to the system, the companion device comprising one of a computer or processing device running a management application in bi-directional communication with the system controller; wherein the system controller sends information on the layout and metadata about layout constraints for each of the plurality of user interface widgets to the management application, such that when a user of the companion device interacts with the management application, the management application sends at least one of: layout changes; widget selection notifications; and widget commands to the system controller for relay to the system controller, thereby causing at least one of: layout changes; widget selection notifications; and widget commands to be rendered on the display panel array, and the plurality of display adaptors, the layout engine, and the system controller are each implemented on at least one of a computer or processing device.
  2. 2. The system of claim 1 wherein the area and configuration of the display panel array is stored by the layout engine during one of: installation of the system; and a system update.
  3. 3. The system of either claim I or claim 2 wherein the management application provides visual cues to the user of the companion device, thereby enabling fast locating and selection of objects.
  4. 4. The system of any of claims 1 -3 wherein the plurality of user interface widgets presently in use on the display panel array is mirrored by a layout of thumbnail representations of the plurality of user interface widgets in the management application.
  5. 5. The system of claim 4 wherein each of the thumbnail representations appear proportionally in the same size and position as its corresponding user interface widget as the corresponding user interface widget appears on the display panel array.
  6. 6. The system of either claim 4 or claim 5 wherein each of the thumbnail representations appears in the management application with a lower graphical resolution than each corresponding user interface widget appears on the display panel array.
  7. 7. The system of any of claims 4 -6 wherein each of the thumbnail representations displays headline status information within its boundaries.
  8. 8. The system of any of claims i -6 wherein the user of the companion device interacts with the management application using standard gestures.
  9. 9. The system of claim 8 wherein the standard gestures inclLtde a pinch and zoom gesture; and a drag gesture.
  10. 10. The system of any of claims I -9 wherein the metadata published by the user interface widget is published via a common application programming interface.
  11. 11. The system of any of claims 1 -10 wherein the management application is operative to enter a tracking mode, causing a selected one of the plurality of user interface widgets to be highlighted on the display panel array.
  12. 12. A method for controlling a display panel array, the display panel array compnsing a plurality of display panels and being operative to display a plurality of user interface widgets, the method comprising: controlling a layout of an output on at least one display panel of the display panel array at each one of a plurality of display adaptors; arranging each of the plurality of user interface widgets on the display panel array by a layout engine, by utilizing at least: metadata published by each of the plurality of user interface widgets, the metadata comprising constraints on a user interface widget aspect ratio or orientation; a layout of the plurality of user interface widgets presently in use on the display panel array; and an area of the display panel array, the layout engine being operative to split a plurality of outputs among each one of the display adaptors, the plurality of outputs comprising data which each of the display adaptors requires in order to display the layout, thereby producing the desired layout on the display panel array; receiving at a system controller a multimedia input from at least one network adapter, the system controller managing runtime context for each of the plurality of user interface widgets, the system controller operative to produce an output comprising at least a portion of the multimedia input for each one of the plurality of user interface widgets, the system controller being operative to provide the produced output, as the layout of the plurality of user interface widgets presently in use, to the layout engine; communicating bi-directionally between the system controller and a companion device external to the system, the companion device comprising one of a computer or processing device running a management application in bi-directional communication with the system controller; wherein the system controller sends information on the layout and metadata about layout constraints for each of the piLirality of user interface widgets to the management application, such that when a user of the companion device interacts with the management application, the management application sends at least one of: layout changes; widget selection notifications; and widget commands to the system controller for relay to the system controller, thereby causing at least one of: layout changes; widget selection notifications; and widget commands to be rendered on the display panel array, and the plurality of display adaptors, the layout engine, and the system controller are each implemented on at least one of a computer or processing device.
GB201306077A 2013-04-04 2013-04-04 Interface mechanism for massive resolution displays Expired - Fee Related GB2512626B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
GB201306077A GB2512626B (en) 2013-04-04 2013-04-04 Interface mechanism for massive resolution displays
PCT/IB2014/060127 WO2014162239A2 (en) 2013-04-04 2014-03-25 Interface mechanism for massive resolution displays

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB201306077A GB2512626B (en) 2013-04-04 2013-04-04 Interface mechanism for massive resolution displays

Publications (3)

Publication Number Publication Date
GB201306077D0 GB201306077D0 (en) 2013-05-22
GB2512626A true GB2512626A (en) 2014-10-08
GB2512626B GB2512626B (en) 2015-05-20

Family

ID=48483311

Family Applications (1)

Application Number Title Priority Date Filing Date
GB201306077A Expired - Fee Related GB2512626B (en) 2013-04-04 2013-04-04 Interface mechanism for massive resolution displays

Country Status (2)

Country Link
GB (1) GB2512626B (en)
WO (1) WO2014162239A2 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019020883A1 (en) * 2017-07-28 2019-01-31 Psa Automobiles Sa Device for providing a widget-counting graphical interface in a vehicle
CN111048016A (en) * 2018-10-15 2020-04-21 广东美的白色家电技术创新中心有限公司 Product display method, device and system

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10348997B2 (en) * 2015-07-09 2019-07-09 Genetec Inc. Security video monitoring client

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110025706A1 (en) * 2005-01-18 2011-02-03 Etelaeperae Esa User interface for different displays
WO2012153290A1 (en) * 2011-05-10 2012-11-15 Nds Limited Adaptive presentation of content
US20120319927A1 (en) * 2008-12-02 2012-12-20 Nvidia Corporation Remote management of a simultaneous display of multimedia content in display devices

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020163597A1 (en) * 2001-05-03 2002-11-07 Kevin Kelleher Digital multi-tile video display system
CA2554766A1 (en) * 2004-02-09 2005-08-25 Regis Development, L.L.C. Computer presentation and command integration apparatus and method
US9159298B2 (en) * 2010-09-08 2015-10-13 Lg Electronics Inc. Terminal and contents sharing method for terminal
CN103168325B (en) * 2010-10-05 2017-06-30 西里克斯系统公司 For the display management of local user's experience

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110025706A1 (en) * 2005-01-18 2011-02-03 Etelaeperae Esa User interface for different displays
US20120319927A1 (en) * 2008-12-02 2012-12-20 Nvidia Corporation Remote management of a simultaneous display of multimedia content in display devices
WO2012153290A1 (en) * 2011-05-10 2012-11-15 Nds Limited Adaptive presentation of content

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019020883A1 (en) * 2017-07-28 2019-01-31 Psa Automobiles Sa Device for providing a widget-counting graphical interface in a vehicle
FR3069676A1 (en) * 2017-07-28 2019-02-01 Psa Automobiles Sa DEVICE FOR PROVIDING A GRAPHICAL INTERFACE COMPRISING WIDGETS IN A VEHICLE.
CN111048016A (en) * 2018-10-15 2020-04-21 广东美的白色家电技术创新中心有限公司 Product display method, device and system
CN111048016B (en) * 2018-10-15 2021-05-14 广东美的白色家电技术创新中心有限公司 Product display method, device and system

Also Published As

Publication number Publication date
WO2014162239A2 (en) 2014-10-09
GB201306077D0 (en) 2013-05-22
WO2014162239A3 (en) 2015-01-29
GB2512626B (en) 2015-05-20

Similar Documents

Publication Publication Date Title
AU2019216686B2 (en) System for organizing and displaying information on a display device
US20220230603A1 (en) Adaptable user interface with dual screen device
KR101867644B1 (en) Multi-application environment
EP3859501B1 (en) Application execution method by display device and display device thereof
US9268423B2 (en) Definition and use of node-based shapes, areas and windows on touch screen devices
KR102391699B1 (en) Dynamic joint dividers for application windows
EP2690543B1 (en) Display device for executing multiple applications and method for controlling the same
US9247303B2 (en) Display apparatus and user interface screen providing method thereof
KR102027612B1 (en) Thumbnail-image selection of applications
EP3057312A2 (en) Image display apparatus and method
EP2405337B1 (en) Method for presenting human machine interface, handheld device using the same, and computer readable medium therefor
EP3141997B1 (en) Mobile terminal and method for controlling the same
EP2741192A2 (en) Display device for executing a plurality of applications and method for controlling the same
EP2720127A2 (en) Display device and method of providing tool therefor
US9811303B2 (en) Display apparatus, multi display system including the same, and control method thereof
EP3611611A1 (en) User terminal device with pen and controlling method thereof
KR20140133353A (en) display apparatus and user interface screen providing method thereof
KR20170067863A (en) Heterogeneous application tabs
KR20120002727A (en) Apparatus for displaying 3d ui
KR20140068573A (en) Display apparatus and method for controlling thereof
US9513795B2 (en) System and method for graphic object management in a large-display area computing device
US20130127826A1 (en) Parallax image authoring and viewing in digital media
US20180165793A1 (en) Display apparatus for providing user interface and controlling method thereof
KR20130054072A (en) Mobile apparatus for processing multiple applications and method thereof
GB2512626A (en) Interface mechanism for massive resolution displays

Legal Events

Date Code Title Description
PCNP Patent ceased through non-payment of renewal fee

Effective date: 20220404