WO2017090916A1 - Managing display of information on multiple devices based on context for a user task - Google Patents
Managing display of information on multiple devices based on context for a user task Download PDFInfo
- Publication number
- WO2017090916A1 WO2017090916A1 PCT/KR2016/012782 KR2016012782W WO2017090916A1 WO 2017090916 A1 WO2017090916 A1 WO 2017090916A1 KR 2016012782 W KR2016012782 W KR 2016012782W WO 2017090916 A1 WO2017090916 A1 WO 2017090916A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- application
- electronic device
- relevant information
- relevant
- Prior art date
Links
- 238000000034 method Methods 0.000 claims abstract description 38
- 230000004044 response Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 description 19
- 230000008569 process Effects 0.000 description 17
- 238000004891 communication Methods 0.000 description 15
- 230000015654 memory Effects 0.000 description 13
- 230000009471 action Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000003993 interaction Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 229920001621 AMOLED Polymers 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010367 cloning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012015 optical character recognition Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/20—Services signaling; Auxiliary data signalling, i.e. transmitting data via a non-traffic channel
- H04W4/203—Services signaling; Auxiliary data signalling, i.e. transmitting data via a non-traffic channel for converged personal network application service interworking, e.g. OMA converged personal network services [CPNS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/12—Synchronisation between the display unit and other units, e.g. other display units, video-disc players
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W88/00—Devices specially adapted for wireless communication networks, e.g. terminals, base stations or access point devices
- H04W88/02—Terminal devices
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2354/00—Aspects of interface with display user
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2370/00—Aspects of data communication
- G09G2370/04—Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2370/00—Aspects of data communication
- G09G2370/20—Details of the management of multiple sources of image data
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/14—Display of multiple viewports
Definitions
- This disclosure relates generally to display of information on electronic devices. More specifically, this disclosure relates to managing display of information on multiple devices based on context for a user task.
- Performing a practical task on an electronic device may require using multiple applications.
- Each individual application may have a specific narrow scope of information obtained from a previously used application.
- different applications are often not designed to work together or have limited sharing of information between the applications. Frequently users need to switch back and forth between applications or memorize information from previously accessed applications, which reduces user experience.
- Embodiments of the present disclosure provide management of information context for a user task on multiple devices.
- a method for managing information context among devices includes switching from displaying information of a first application to displaying information of a second application.
- the method also includes identifying information of the first application that is relevant to the second application.
- the relevant information includes at least a portion of the displayed information of the first application.
- the method includes sending an indication of the relevant information to a second electronic device for display of the relevant information at the second electronic device.
- an electronic device for managing information context among devices includes a display, at least one processor, and a transceiver.
- the at least one processor is configured to cause the display to switch from displaying information of a first application to displaying information of a second application and identify information of the first application that is relevant to the second application.
- the relevant information including at least a portion of the displayed information of the first application.
- the transceiver is configured to send an indication of the relevant information to a second electronic device for display of the relevant information at the second electronic device.
- an electronic device for managing information context among devices includes a display, at least one processor, and a transceiver.
- the transceiver is configured to receive, from a second electronic device, an indication of relevant information of a first application executed on the second electronic device that was switched from.
- the display is configured to display the relevant information.
- Embodiments of the present disclosure provide management of information context for a user task on multiple devices.
- FIG. 1 illustrates an example wireless communication system in which various embodiments of the present disclosure may be implemented
- FIG. 2 illustrates an example of relevant information display in accordance with various embodiments of the present disclosure
- FIG. 3 illustrates a process for managing information context among devices in accordance with various embodiments of the present disclosure
- FIG. 4 illustrates another process for managing information context among devices in accordance with various embodiments of the present disclosure.
- Couple and its derivatives refer to any direct or indirect communication between two or more elements, whether or not those elements are in physical contact with one another.
- transmit and “communicate,” as well as derivatives thereof, encompass both direct and indirect communication.
- the term “or” is inclusive, meaning and/or.
- controller means any device, system or part thereof that controls at least one operation. Such a controller may be implemented in hardware or a combination of hardware and software and/or firmware. The functionality associated with any particular controller may be centralized or distributed, whether locally or remotely.
- phrases “at least one of,” when used with a list of items, means that different combinations of one or more of the listed items may be used, and only one item in the list may be needed.
- “at least one of: A, B, and C” includes any of the following combinations: A, B, C, A and B, A and C, B and C, and A and B and C.
- various functions described below can be implemented or supported by one or more computer programs, each of which is formed from computer readable program code and embodied in a computer readable medium.
- application and “program” refer to one or more computer programs, software components, sets of instructions, procedures, functions, objects, classes, instances, related data, or a portion thereof adapted for implementation in a suitable computer readable program code.
- computer readable program code includes any type of computer code, including source code, object code, and executable code.
- computer readable medium includes any type of medium capable of being accessed by a computer, such as read only memory (ROM), random access memory (RAM), a hard disk drive, a compact disc (CD), a digital video disc (DVD), or any other type of memory.
- ROM read only memory
- RAM random access memory
- CD compact disc
- DVD digital video disc
- a “non-transitory” computer readable medium excludes wired, wireless, optical, or other communication links that transport transitory electrical or other signals.
- a non-transitory computer readable medium includes media where data can be permanently stored and media where data can be stored and later overwritten, such as a rewritable optical disc or an erasable memory device.
- FIGS. 1 through 4 discussed below, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged system or device.
- Embodiments of the present disclosure recognize and take into account that relevant information can be lost to a user upon application switch.
- Embodiments of the present disclosure recognize and take into account that some solutions to loss of information between application switch may involve cloning/migrating applications between different devices, splitting application visualization components between multiple devices or portions of a screen, splitting application visualization and application control between different devices or portions of a screen, and selecting and migrating a portion of the UI to another device or portion of a screen.
- Embodiments of the present disclosure recognize that none of these address display of contextually relevant information on multiple devices.
- a user task to join a teleconference call may include launching a calendar application, retrieving a reminder or appointment from a calendar application, finding the number to call in the calendar appointment, pressing the number in the reminder, and switching to a telephone call application to make the call.
- the calendar application includes a display of details of the appointment and the telephone call application places a call.
- embodiments of the present disclosure recognize that relationships and as a result some calendar appointment information, for instance the meeting number or conference pin, is lost in the process of switching to the telephone call application and the user has to switch back to the calendar appointment to retrieve (and possibly memorize) the lost information.
- embodiments of the present disclosure when a user performs a task that requires an application switch, embodiments of the present disclosure detect the information displayed in the previous application that is still relevant to the overall user task and can forward that information to be displayed on a second electronic device while the user interface (UI) for the new application being displayed on the primary device.
- UI user interface
- FIG. 1 illustrates an example electronic device 105 in a wireless communication system 100 in which various embodiments of the present disclosure may be implemented.
- the embodiment of the system 100 and the device 105 illustrated in FIG. 1 is for illustration only, the system 100 and the device 105 may come in a wide variety of configurations and include additional or alternative components.
- FIG. 1 does not limit the scope of this disclosure to any particular implementation of an electronic device or wireless communication system.
- the electronic device 105 includes an antenna 108, a transceiver 110, transmit (TX) processing circuitry 115, a microphone 120, and receive (RX) processing circuitry 125.
- the electronic device 105 also includes a speaker 130, a processor 140, an input/output (I/O) interface (IF) 145, a touchscreen 150, a display 155, a memory 160, and one or more sensors 165.
- the memory 160 includes an operating system (OS) 161 and one or more applications 162.
- the transceiver 110 receives, from the antenna 108, an incoming RF signal transmitted by an access point (e.g., base station, WiFi router, Bluetooth device) for a network (e.g., a WiFi, Bluetooth, cellular, 5G, LTE, LTE-A, WiMAX, or any other type of wireless network).
- the transceiver 110 down-converts the incoming RF signal to generate an intermediate frequency (IF) or baseband signal.
- the IF or baseband signal is sent to the RX processing circuitry 125, which generates a processed baseband signal by filtering, decoding, and/or digitizing the baseband or IF signal.
- the RX processing circuitry 125 transmits the processed baseband signal to the speaker 130 (such as for voice data) or to the processor 140 for further processing (such as for web browsing data).
- the access point e.g., base station, WiFi router, Bluetooth device
- a network e.g., a WiFi, Bluetooth, cellular, 5G, LTE,
- the TX processing circuitry 115 receives analog or digital voice data from the microphone 120 or other outgoing baseband data (such as web data, e-mail, or interactive video game data) from the processor 140.
- the TX processing circuitry 115 encodes, multiplexes, and/or digitizes the outgoing baseband data to generate a processed baseband or IF signal.
- the transceiver 110 receives the outgoing processed baseband or IF signal from the TX processing circuitry 115 and up-converts the baseband or IF signal to an RF signal that is transmitted via the antenna 108.
- the transceiver 110, the TX processing circuitry 115, and the RX processing circuitry 125 may be implemented within the same block or chip, such as a WiFi and/or Bluetooth module and/or chip.
- the processor 140 can include one or more processors or other processing devices and execute the OS 161 stored in the memory 160 in order to control the overall operation of the electronic device 105.
- the processor 140 could control the reception of forward channel signals and the transmission of reverse channel signals by the transceiver 110, the RX processing circuitry 125, and the TX processing circuitry 115 in accordance with well-known principles.
- the processor 140 includes at least one microprocessor or microcontroller.
- the processor 140 is also capable of executing other processes and programs resident in the memory 160.
- the processor 140 can move data into or out of the memory 160 as required by an executing process.
- the processor 140 is configured to execute the applications 162 based on the OS 161 or in response to signals received from eNBs or an operator.
- the processor 140 is also coupled to the I/O interface 145, which provides the electronic device 105 with the ability to connect to other devices, such as laptop computers and handheld computers.
- the I/O interface 145 is the communication path between these accessories and the processor 140.
- the processor 140 is also coupled to the touchscreen 150 and the display 155.
- the operator of the electronic device 105 can use the touchscreen 150 to enter data and/inputs into the electronic device 105.
- the display 155 may be a liquid crystal display, light-emitting diode (LED) display, optical LED (OLED), active matrix OLED (AMOLED), or other display capable of rendering text and/or at graphics, such as from web sites, videos, games, etc.
- the touchscreen 150 can include a touch panel, a (digital) pen sensor, a key, or an ultrasonic input device.
- the touchscreen 150 can recognize, for example, a touch input in at least one scheme among a capacitive scheme, a pressure sensitive scheme, an infrared scheme, or an ultrasonic scheme.
- the touchscreen 150 can also include a control circuit. In the capacitive scheme, the touchscreen 150 can recognize touch or proximity.
- the memory 160 is coupled to the processor 140.
- Part of the memory 160 could include a random access memory (RAM), and another part of the memory 160 could include a Flash memory or other read-only memory (ROM).
- RAM random access memory
- ROM read-only memory
- Electronic device 105 further includes one or more sensors 165 that can meter a physical quantity or detect an activation state of the electronic device 105, and convert metered or detected information into an electrical signal.
- sensor 165 may include one or more buttons for touch input, a camera, a gesture sensor, a gyroscope or gyro sensor, an air pressure sensor, a magnetic sensor or magnetometer, an acceleration sensor or accelerometer, a grip sensor, a proximity sensor, a color sensor 165H (e.g., a Red Green Blue (RGB) sensor), a temperature/humidity sensor, an illumination sensor, etc.
- the sensor(s) 165 can further include a control circuit for controlling at least one of the sensors included therein. As will be discussed in greater detail below, one or more of these sensor(s) 165 may be used to determine a task being performed by a user to identify information relevant to the task being performed.
- the system 100 includes a network 175, which facilitates communication between various components in the system 100.
- the network 175 may communicate Internet Protocol (IP) packets, frame relay frames, or other information between network addresses.
- IP Internet Protocol
- the network 175 may also be a heterogeneous network including broadcasting networks, such as cable and satellite communication links.
- the network 175 may include one or more local area networks (LANs); metropolitan area networks (MANs); wide area networks (WANs); all or a portion of a global network, such as the Internet; or any other communication system or systems at one or more locations.
- the network 175 facilitates communications between at least one server 180 and various client electronic devices 105, 170, and 185.
- Each server 180 includes any suitable computing or processing device that can provide computing services for one or more client devices.
- Each server 180 could, for example, include one or more processing devices, one or more memories storing instructions and data, and one or more network interfaces facilitating communication over the network 175.
- Each client electronic devices 105, 170, and 185 represents any suitable computing or processing device that interacts with at least one server or other computing device(s) over the network 175.
- the client electronic devices 105, 170, and 185 communicate via one or more base stations, such as cellular base stations or eNodeBs.
- the client electronic devices 105, 170, and 185 may communicate via one or more wireless access points, such as IEEE 802.11 wireless access points.
- wireless access points such as IEEE 802.11 wireless access points.
- each client device could communicate directly with the network 175 or indirectly with the network 175 via any suitable intermediate device(s) or network(s).
- the electronic devices 105, 170, and 185 may communicate with each other either directly (e.g., via Bluetooth or another peer to peer connection) or over the network 175.
- the electronic devices 105, 170, and 185 may include circuitry and/or programming for managing display of information on multiple devices based on context for a user task.
- the electronic devices 105, 170, and 185 may take different forms and the present disclosure is not limited to any particular form.
- one or more of electronic devices 105, 170, and 185 may be a mobile communication device, such as, for example, a user equipment, a mobile station, subscriber station, wireless terminal, smart phone, a tablet, etc.
- one or more of electronic devices 105, 170, and 185 may take the form of a wearable electronic device, such as for example, a smart watch, smart glasses, an activity tracker, a head mounted display, or other type of wearable device having a display.
- a wearable electronic device such as for example, a smart watch, smart glasses, an activity tracker, a head mounted display, or other type of wearable device having a display.
- one or more of electronic devices 105, 170, and 185 may be stationary devices such as, for example, a television or kiosk.
- FIG. 1 illustrates one example of an electronic device 105
- various changes may be made to FIG. 1.
- various components in FIG. 1 could be combined, further subdivided, or omitted and additional components could be added according to particular needs.
- the processor 140 could be divided into multiple processors, such as one or more central processing units (CPUs) and one or more graphics processing units (GPUs).
- the speaker 130, the microphone 120, and or the touchscreen 150 may not be included in the device 105.
- the display 155 and the touchscreen 150 may be combined or may be implemented as different layers of a screen for the device 105.
- FIG. 1 illustrates one example of a communication system 100
- the system 100 could include any number of each component in any suitable arrangement.
- computing and communication systems come in a wide variety of configurations, and FIG. 1 does not limit the scope of this disclosure to any particular configuration.
- FIG. 1 illustrates one operational environment in which various features disclosed in this patent document can be used, these features could be used in any other suitable system.
- Embodiments of the present disclosure recognize that when user performs an action that causes an active application change, some information that is still relevant in the context of the user task is not accessible in the new application. For example, interaction with a phone number in a calendar application switches the view of the phone to a phone call application while a teleconference PIN is left in the calendar application and is inaccessible.
- interaction with an address in a messaging application e.g., text messaging, chant messaging, or email application
- interactions with a link e.g., URL
- a message switches a messaging application to a web browser application, while comments about the link included in the information of the previous messaging application are inaccessible in the currently displayed UI for the web browser application.
- Embodiments of the present disclosure provide for dynamic display of contextually relevant information on multiple devices.
- embodiments of the present disclosure identify and extract relevant information from prior application(s) and provide this relevant information for use in the present application.
- embodiments of the present disclosure identify a user’s overall task (i.e., the task being performed using the present application or series of applications used by the user) and dynamically detect and select information that is relevant to the user’s overall task.
- this information is seamlessly forwarded and or displayed in a second electronic device.
- Embodiments of the present disclosure allow a user to interact with this information on the second electronic device to automatically input, forward, update and/or populate data into the new application on the first electronic device.
- FIG. 2 illustrates an example of relevant information display in accordance with various embodiments of the present disclosure.
- the example illustrated in FIG. 2 is for illustration only. Additional relevant information display between applications may be implemented in addition to the example illustrated in FIG. 2, for example, as discussed hereinafter.
- the electronic devices 200 and 205 illustrated in FIG. 2 may be an example of implementations of any of the electronic devices 105, 170, and 185 in FIG. 1.
- the adjectives “first” and “second” are used to distinguish between different devices, use of the term “first” or “second” is not meant to imply any physical or architectural limitation on the functions that can be performed on either device in accordance with the principles of the present disclosure.
- either of the first or second electronic devices may undergo the application switch, identify the relevant information, forward an indication of the relevant information, display the relevant information, or forward an input of the relevant information to the other of the devices.
- a user had selected an address for a meeting location in a text messaging application on the first electronic device 200 which caused the first electronic device 200 to switch the display to that of map application 210.
- the electronic device 200 identifies that additional information is relevant to the user’s overall task of navigating to the meeting location, which in this example, is parking information for the meeting location, is not available in and otherwise lost by the map application 210.
- the electronic device 200 forwards an indication of the relevant information to the second electronic device 205, in this example a smart watch, for display of the relevant information, in this example the entire text message, by the second electronic device 205.
- the user is provided with all the relevant information for navigating to the meeting location without being required to switch back and forth between applications or memorize the information.
- the user can forward the relevant information back to the first electronic device for use of the information in the application on the first electronic device. For example, performing some action, such as, a button press on the relevant information on the second electronic device triggers action on the first electronic device.
- This action may also be a hold-and-swipe, to forward the relevant information to a relevant application field on the first electronic device.
- the forwarded information may be copied to a clipboard for use with the application on the first electronic device, inserted into fillable fields (e.g., shipping or payment information fields) of the application on the first electronic device, may be inserted and used as in input (e.g., a PIN or search input).
- the methods and electronic devices of the present disclosure can detect which information is relevant to the user’s overall task and is not present in the new application.
- the electronic device may track the task being performed by the user between application switches and shift a portion of the UI switched from to another electronic device based on an estimation of the user task. This may include the electronic device identifying a type of relationship between the old and new applications and/or the mechanism used for switching between applications.
- the electronic device can identify the type of link selected (e.g., a URL, a recognized address, a recognized phone number, etc.) and the type of application of the switched from and switched to applications.
- the electronic device may track a sequence, type, and relationship between or among of applications switched between or among either manually or automatically, within a given period of time to estimate the user task.
- the electronic device may estimate the overall task being performed by the user search the switched from application to determine whether any information relevant to the overall user task is located in the switched from application but not otherwise available in the switched to application. For example, the electronic device may perform a text search for information that may be relevant to a user application, which may include, for example, optical character recognition of text in images. In these examples, the electronic device may forward the relevant information by forwarding the UI for the prior application, certain text from the prior application, or an image from the prior application for display by the second electronic device. The electronic device sends an indication of the relevant information that may include, for example, the relevant information itself an indication of the relevant information for the second electronic device to obtain the relevant information. For example, the first electronic device may send a message identifier or application identifier for the second electronic device to launch a corresponding application and/or retrieve a corresponding message, for example, from a server.
- a text search for information that may be relevant to a user application, which may include, for example, optical character recognition of text in images.
- the electronic device may identify information about the context of the task being performed using inputs from sensors (e.g., such as sensor(s) 165) located on one or more of the first electronic device, the second electronic device, or sensors in the environment.
- the second electronic device may include and use sensor(s) to identify audio played by the first electronic device and display information relevant to the audio played (e.g., purchase information or related products to a movie, television show, or song played on the first electronic device).
- the electronic device may receive information from sensors that are part of home automation equipment or system, such as for example, to estimate the task being performed by the user.
- the electronic device may identify a location of the electronic device and estimate the user task based on tasks performable at the identified location of the electronic device, and possibly also based on the application executing on the electronic device.
- the electronic device may identify the relevant information based on default or manual user preferences. For example, the user may have pre-established preferences, that when selecting a link switching to a new application, certain information from or the UI for the switched application is forwarded or “pushed” for display on the second electronic device. In other examples, the user preference may be specific to particular applications that are the switched from and/or switched to applications.
- FIG. 3 illustrates a process for managing information context among devices in accordance with various embodiments of the present disclosure.
- the process depicted in FIG. 3 may be performed by the electronic device 105 in FIG. 1.
- the process may also be implemented by either of the electronic devices 200 or 205 in FIG 2.
- the process begins with the electronic device displaying information of a first application (step 305). For example, in step 305, the electronic device may display a UI for the first application that includes a selectable link triggering a switch to a second application.
- the electronic device determines whether a switch from first application to a second application is requested (step 310). For example, in step 310, the electronic device may detect a selection of a link or a manual application switch by a user.
- the electronic device determines whether relevant information from the first application can be used in the second application (step 315). For example, in step 315, the electronic device may search the first application to determine whether missing or helpful information displayed in the UI for the first application is not available or displayed in the UI for the second application, for example, by a comparison of the information displayed for the respective applications.
- the electronic device determines that relevant information from the first application can be used in the second application, the electronic device identifies relevant information of the first application (step 320). For example, in step 320, the electronic device may identify the relevant information based on a setting indicating a preference for the relevant information to be displayed at the second electronic device when switching from the first application to the second application. In another example, the electronic device may estimate a task being performed using the second application and identify the relevant information of the first application based on the estimated task.
- the electronic device may identify a relationship between the first and second applications based on properties of the first and second applications, respectively (e.g., tasks performed by the respective applications, a link selected to cause the switch between the applications) and identify the relevant information of the first application based on the identified relationship.
- the electronic device may identify context information based on an input from one or more sensors located in the electronic device, a second electronic device, or in the environment and identify the relevant information of the first application based on the identified context information.
- the electronic device sends an indication of the relevant information to a second electronic device (step 325).
- the electronic device may send the indication for the second electronic device to display of the relevant information at the second electronic device.
- the indication may include the relevant information as text, an application UI, an image, etc.
- the indication may include an indicator of an application to launch and/or a message or image to retrieve, for example, from a server.
- the electronic device displays information of the second application (step 330).
- the electronic device determines whether input information from the second electronic device has been received (step 335). For example, in step 335, the electronic device may receive an input that includes at least a portion of the relevant information displayed at the second electronic device.
- the electronic device uses the input information in the second application (step 340). For example, in step 340, the electronic device may copy the received information to a clipboard for a user to paste into a relevant portion, display the relevant information, for example, in a notification or pop-up, and/or populate relevant fields for the second application (e.g., fill-in shipping or address fields, enter the information into a search field and process a search, enter the information as a PIN, etc.)
- relevant fields for the second application e.g., fill-in shipping or address fields, enter the information into a search field and process a search, enter the information as a PIN, etc.
- FIG. 4 illustrates another process for managing information context among devices in accordance with various embodiments of the present disclosure.
- the process depicted in FIG. 4 may be performed by the electronic device 105 in FIG. 1.
- the process may also be implemented by either of the electronic devices 200 or 205 in FIG 2.
- the electronic device performing the process illustrated in FIG. 4 may be the second electronic device referred to in the process of FIG. 3 and vice versa.
- the process begins with the electronic device identifying relevant information of a first application executed on a second electronic device (step 405).
- the electronic device may receive, from the second electronic device, an indication of the relevant information of the first application executed on the second electronic device that was switched from.
- the electronic device may send a request to the second electronic device for the indication of the relevant information of the first application executed on the second electronic device that was switched from, for example, based on a user input or an indication that the user is looking at the display of the electronic device (e.g., detecting an inclination angle or range of the electronic device using an accelerometer or gyroscope, processing image data captured by a camera of the electronic device showing eyes of the user looking in a direction of the display of the electronic device, etc.
- the electronic device may estimate the task being performed at the second electronic device using inputs from sensors (e.g., sensor(s) 165) and identify the relevant information of the first application based on the estimated task being performed at the second electronic device. For example, the electronic device may identify context information for the task being performed based on audio inputs sensed by a microphone and identify a song, movie, or television show using the context information, where the first electronic device is a television or media player.
- the electronic device displays the relevant information (step 410). For example, in step 410, the electronic device may display information from the first application switched from and/or information determined to be relevant to the task the user is performing at the second electronic device.
- the electronic device determines whether a user input has been received with respect to the displayed relevant information (step 415). For example, in step 415, the electronic device may detect a touch of a portion of the displayed information or a swiping motion on the display of the electronic device toward a direction the second electronic device.
- the electronic device sends at least a portion of the displayed relevant information as an input to the second electronic device (step 420). For example, in step 420, the electronic device may send the portion of the relevant information to the first application or a second application executed on the second electronic device in response to a user input.
- FIGS. 3 and 4 illustrate examples of processes for managing information context among devices
- various changes could be made to FIGS. 3 and 4.
- steps in each figure could overlap, occur in parallel, occur in a different order, occur multiple times, or be omitted in one or more embodiments of the present disclosure.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Multimedia (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Methods and electronic devices for managing information context among devices. The method includes switching from displaying information of a first application to displaying information of a second application. The method also includes identifying information of the first application that is relevant to the second application. The relevant information includes at least a portion of the displayed information of the first application. Additionally, the method includes sending an indication of the relevant information to a second electronic device for display of the relevant information at the second electronic device. The method may also include, while displaying the information of the second application, receiving input information from the second electronic device. The input information may include at least a portion of the relevant information displayed at the second electronic device. Additionally, the method may include using the input information in the second application.
Description
This disclosure relates generally to display of information on electronic devices. More specifically, this disclosure relates to managing display of information on multiple devices based on context for a user task.
Performing a practical task on an electronic device, such as a smart phone, may require using multiple applications. Each individual application may have a specific narrow scope of information obtained from a previously used application. Moreover, different applications are often not designed to work together or have limited sharing of information between the applications. Frequently users need to switch back and forth between applications or memorize information from previously accessed applications, which reduces user experience.
Embodiments of the present disclosure provide management of information context for a user task on multiple devices.
In one exemplary embodiment, a method for managing information context among devices is provided. The method includes switching from displaying information of a first application to displaying information of a second application. The method also includes identifying information of the first application that is relevant to the second application. The relevant information includes at least a portion of the displayed information of the first application. Additionally, the method includes sending an indication of the relevant information to a second electronic device for display of the relevant information at the second electronic device.
In another exemplary embodiment, an electronic device for managing information context among devices is provided. The electronic device includes a display, at least one processor, and a transceiver. The at least one processor is configured to cause the display to switch from displaying information of a first application to displaying information of a second application and identify information of the first application that is relevant to the second application. The relevant information including at least a portion of the displayed information of the first application. The transceiver is configured to send an indication of the relevant information to a second electronic device for display of the relevant information at the second electronic device.
In yet another exemplary embodiment, an electronic device for managing information context among devices is provided. The electronic device includes a display, at least one processor, and a transceiver. The transceiver is configured to receive, from a second electronic device, an indication of relevant information of a first application executed on the second electronic device that was switched from. The display is configured to display the relevant information.
Other technical features may be readily apparent to one skilled in the art from the following figures, descriptions, and claims.
Embodiments of the present disclosure provide management of information context for a user task on multiple devices.
For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which like reference numerals represent like parts:
FIG. 1 illustrates an example wireless communication system in which various embodiments of the present disclosure may be implemented;
FIG. 2 illustrates an example of relevant information display in accordance with various embodiments of the present disclosure;
FIG. 3 illustrates a process for managing information context among devices in accordance with various embodiments of the present disclosure;
FIG. 4 illustrates another process for managing information context among devices in accordance with various embodiments of the present disclosure.
Before undertaking the DETAILED DESCRIPTION below, it may be advantageous to set forth definitions of certain words and phrases used throughout this patent document. The term “couple” and its derivatives refer to any direct or indirect communication between two or more elements, whether or not those elements are in physical contact with one another. The terms “transmit,” “receive,” and “communicate,” as well as derivatives thereof, encompass both direct and indirect communication. The terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation. The term “or” is inclusive, meaning and/or. The phrase “associated with,” as well as derivatives thereof, means to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, have a relationship to or with, or the like. The term “controller” means any device, system or part thereof that controls at least one operation. Such a controller may be implemented in hardware or a combination of hardware and software and/or firmware. The functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. The phrase “at least one of,” when used with a list of items, means that different combinations of one or more of the listed items may be used, and only one item in the list may be needed. For example, “at least one of: A, B, and C” includes any of the following combinations: A, B, C, A and B, A and C, B and C, and A and B and C.
Moreover, various functions described below can be implemented or supported by one or more computer programs, each of which is formed from computer readable program code and embodied in a computer readable medium. The terms “application” and “program” refer to one or more computer programs, software components, sets of instructions, procedures, functions, objects, classes, instances, related data, or a portion thereof adapted for implementation in a suitable computer readable program code. The phrase “computer readable program code” includes any type of computer code, including source code, object code, and executable code. The phrase “computer readable medium” includes any type of medium capable of being accessed by a computer, such as read only memory (ROM), random access memory (RAM), a hard disk drive, a compact disc (CD), a digital video disc (DVD), or any other type of memory. A “non-transitory” computer readable medium excludes wired, wireless, optical, or other communication links that transport transitory electrical or other signals. A non-transitory computer readable medium includes media where data can be permanently stored and media where data can be stored and later overwritten, such as a rewritable optical disc or an erasable memory device.
Definitions for other certain words and phrases are provided throughout this patent document. Those of ordinary skill in the art should understand that in many if not most instances, such definitions apply to prior as well as future uses of such defined words and phrases.
FIGS. 1 through 4, discussed below, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged system or device.
Embodiments of the present disclosure recognize and take into account that relevant information can be lost to a user upon application switch. Embodiments of the present disclosure recognize and take into account that some solutions to loss of information between application switch may involve cloning/migrating applications between different devices, splitting application visualization components between multiple devices or portions of a screen, splitting application visualization and application control between different devices or portions of a screen, and selecting and migrating a portion of the UI to another device or portion of a screen. Embodiments of the present disclosure recognize that none of these address display of contextually relevant information on multiple devices.
For example, a user task to join a teleconference call may include launching a calendar application, retrieving a reminder or appointment from a calendar application, finding the number to call in the calendar appointment, pressing the number in the reminder, and switching to a telephone call application to make the call. In this example, the calendar application includes a display of details of the appointment and the telephone call application places a call. However, embodiments of the present disclosure recognize that relationships and as a result some calendar appointment information, for instance the meeting number or conference pin, is lost in the process of switching to the telephone call application and the user has to switch back to the calendar appointment to retrieve (and possibly memorize) the lost information.
In accordance with various embodiments of the present disclosure, when a user performs a task that requires an application switch, embodiments of the present disclosure detect the information displayed in the previous application that is still relevant to the overall user task and can forward that information to be displayed on a second electronic device while the user interface (UI) for the new application being displayed on the primary device.
FIG. 1 illustrates an example electronic device 105 in a wireless communication system 100 in which various embodiments of the present disclosure may be implemented. The embodiment of the system 100 and the device 105 illustrated in FIG. 1 is for illustration only, the system 100 and the device 105 may come in a wide variety of configurations and include additional or alternative components. FIG. 1 does not limit the scope of this disclosure to any particular implementation of an electronic device or wireless communication system.
As shown in FIG. 1, the electronic device 105 includes an antenna 108, a transceiver 110, transmit (TX) processing circuitry 115, a microphone 120, and receive (RX) processing circuitry 125. The electronic device 105 also includes a speaker 130, a processor 140, an input/output (I/O) interface (IF) 145, a touchscreen 150, a display 155, a memory 160, and one or more sensors 165. The memory 160 includes an operating system (OS) 161 and one or more applications 162.
The transceiver 110 receives, from the antenna 108, an incoming RF signal transmitted by an access point (e.g., base station, WiFi router, Bluetooth device) for a network (e.g., a WiFi, Bluetooth, cellular, 5G, LTE, LTE-A, WiMAX, or any other type of wireless network). The transceiver 110 down-converts the incoming RF signal to generate an intermediate frequency (IF) or baseband signal. The IF or baseband signal is sent to the RX processing circuitry 125, which generates a processed baseband signal by filtering, decoding, and/or digitizing the baseband or IF signal. The RX processing circuitry 125 transmits the processed baseband signal to the speaker 130 (such as for voice data) or to the processor 140 for further processing (such as for web browsing data). In some embodiments, the
The TX processing circuitry 115 receives analog or digital voice data from the microphone 120 or other outgoing baseband data (such as web data, e-mail, or interactive video game data) from the processor 140. The TX processing circuitry 115 encodes, multiplexes, and/or digitizes the outgoing baseband data to generate a processed baseband or IF signal. The transceiver 110 receives the outgoing processed baseband or IF signal from the TX processing circuitry 115 and up-converts the baseband or IF signal to an RF signal that is transmitted via the antenna 108. In some embodiments, the transceiver 110, the TX processing circuitry 115, and the RX processing circuitry 125 may be implemented within the same block or chip, such as a WiFi and/or Bluetooth module and/or chip.
The processor 140 can include one or more processors or other processing devices and execute the OS 161 stored in the memory 160 in order to control the overall operation of the electronic device 105. For example, the processor 140 could control the reception of forward channel signals and the transmission of reverse channel signals by the transceiver 110, the RX processing circuitry 125, and the TX processing circuitry 115 in accordance with well-known principles. In some embodiments, the processor 140 includes at least one microprocessor or microcontroller.
The processor 140 is also capable of executing other processes and programs resident in the memory 160. The processor 140 can move data into or out of the memory 160 as required by an executing process. In some embodiments, the processor 140 is configured to execute the applications 162 based on the OS 161 or in response to signals received from eNBs or an operator. The processor 140 is also coupled to the I/O interface 145, which provides the electronic device 105 with the ability to connect to other devices, such as laptop computers and handheld computers. The I/O interface 145 is the communication path between these accessories and the processor 140.
The processor 140 is also coupled to the touchscreen 150 and the display 155. The operator of the electronic device 105 can use the touchscreen 150 to enter data and/inputs into the electronic device 105. The display 155 may be a liquid crystal display, light-emitting diode (LED) display, optical LED (OLED), active matrix OLED (AMOLED), or other display capable of rendering text and/or at graphics, such as from web sites, videos, games, etc. The touchscreen 150 can include a touch panel, a (digital) pen sensor, a key, or an ultrasonic input device. The touchscreen 150 can recognize, for example, a touch input in at least one scheme among a capacitive scheme, a pressure sensitive scheme, an infrared scheme, or an ultrasonic scheme. The touchscreen 150 can also include a control circuit. In the capacitive scheme, the touchscreen 150 can recognize touch or proximity.
The memory 160 is coupled to the processor 140. Part of the memory 160 could include a random access memory (RAM), and another part of the memory 160 could include a Flash memory or other read-only memory (ROM).
Also as also illustrated in FIG. 1, the system 100 includes a network 175, which facilitates communication between various components in the system 100. For example, the network 175 may communicate Internet Protocol (IP) packets, frame relay frames, or other information between network addresses. The network 175 may also be a heterogeneous network including broadcasting networks, such as cable and satellite communication links. The network 175 may include one or more local area networks (LANs); metropolitan area networks (MANs); wide area networks (WANs); all or a portion of a global network, such as the Internet; or any other communication system or systems at one or more locations.
The network 175 facilitates communications between at least one server 180 and various client electronic devices 105, 170, and 185. Each server 180 includes any suitable computing or processing device that can provide computing services for one or more client devices. Each server 180 could, for example, include one or more processing devices, one or more memories storing instructions and data, and one or more network interfaces facilitating communication over the network 175.
Each client electronic devices 105, 170, and 185 represents any suitable computing or processing device that interacts with at least one server or other computing device(s) over the network 175. For example, the client electronic devices 105, 170, and 185 communicate via one or more base stations, such as cellular base stations or eNodeBs. Also, the client electronic devices 105, 170, and 185 may communicate via one or more wireless access points, such as IEEE 802.11 wireless access points. These are example communication connections and each client device could communicate directly with the network 175 or indirectly with the network 175 via any suitable intermediate device(s) or network(s). Still further, as will be discussed in greater detail below, the electronic devices 105, 170, and 185 may communicate with each other either directly (e.g., via Bluetooth or another peer to peer connection) or over the network 175.
As described in more detail below, the electronic devices 105, 170, and 185 may include circuitry and/or programming for managing display of information on multiple devices based on context for a user task. In various embodiments, the electronic devices 105, 170, and 185 may take different forms and the present disclosure is not limited to any particular form. For example, one or more of electronic devices 105, 170, and 185 may be a mobile communication device, such as, for example, a user equipment, a mobile station, subscriber station, wireless terminal, smart phone, a tablet, etc. In other examples, one or more of electronic devices 105, 170, and 185 may take the form of a wearable electronic device, such as for example, a smart watch, smart glasses, an activity tracker, a head mounted display, or other type of wearable device having a display. In further examples, one or more of electronic devices 105, 170, and 185 may be stationary devices such as, for example, a television or kiosk.
Although FIG. 1 illustrates one example of an electronic device 105, various changes may be made to FIG. 1. For example, various components in FIG. 1 could be combined, further subdivided, or omitted and additional components could be added according to particular needs. As a particular example, the processor 140 could be divided into multiple processors, such as one or more central processing units (CPUs) and one or more graphics processing units (GPUs). In one or more embodiments, the speaker 130, the microphone 120, and or the touchscreen 150 may not be included in the device 105. In some embodiments, the display 155 and the touchscreen 150 may be combined or may be implemented as different layers of a screen for the device 105.
Although FIG. 1 illustrates one example of a communication system 100, various changes may be made to FIG. 1. For example, the system 100 could include any number of each component in any suitable arrangement. In general, computing and communication systems come in a wide variety of configurations, and FIG. 1 does not limit the scope of this disclosure to any particular configuration. While FIG. 1 illustrates one operational environment in which various features disclosed in this patent document can be used, these features could be used in any other suitable system.
Embodiments of the present disclosure recognize that when user performs an action that causes an active application change, some information that is still relevant in the context of the user task is not accessible in the new application. For example, interaction with a phone number in a calendar application switches the view of the phone to a phone call application while a teleconference PIN is left in the calendar application and is inaccessible. In an example illustrated in FIG. 2, interaction with an address in a messaging application (e.g., text messaging, chant messaging, or email application) switches a view of the electronic device 200 to that of a UI for a map application 210. Additional notes (such as parking details) are inaccessible in the map application. In another example, interactions with a link (e.g., URL) in a message switches a messaging application to a web browser application, while comments about the link included in the information of the previous messaging application are inaccessible in the currently displayed UI for the web browser application.
In all of these non-exhaustive examples, information from a previously used application is often helpful or need as an additional input to the presently switched to application. Embodiments of the present disclosure provide for dynamic display of contextually relevant information on multiple devices. In lieu of requiring the user to manually switch back and forth between applications or memorize the information not otherwise available in the presently switched to application, embodiments of the present disclosure, identify and extract relevant information from prior application(s) and provide this relevant information for use in the present application. During an application view switch, embodiments of the present disclosure identify a user’s overall task (i.e., the task being performed using the present application or series of applications used by the user) and dynamically detect and select information that is relevant to the user’s overall task. In various embodiments, this information is seamlessly forwarded and or displayed in a second electronic device. Embodiments of the present disclosure allow a user to interact with this information on the second electronic device to automatically input, forward, update and/or populate data into the new application on the first electronic device.
FIG. 2 illustrates an example of relevant information display in accordance with various embodiments of the present disclosure. The example illustrated in FIG. 2 is for illustration only. Additional relevant information display between applications may be implemented in addition to the example illustrated in FIG. 2, for example, as discussed hereinafter. The electronic devices 200 and 205 illustrated in FIG. 2 may be an example of implementations of any of the electronic devices 105, 170, and 185 in FIG. 1. Additionally, while the adjectives “first” and “second” are used to distinguish between different devices, use of the term “first” or “second” is not meant to imply any physical or architectural limitation on the functions that can be performed on either device in accordance with the principles of the present disclosure. For example, either of the first or second electronic devices may undergo the application switch, identify the relevant information, forward an indication of the relevant information, display the relevant information, or forward an input of the relevant information to the other of the devices.
In the example illustrated in FIG. 2, a user had selected an address for a meeting location in a text messaging application on the first electronic device 200 which caused the first electronic device 200 to switch the display to that of map application 210. In this illustrative example, the electronic device 200 identifies that additional information is relevant to the user’s overall task of navigating to the meeting location, which in this example, is parking information for the meeting location, is not available in and otherwise lost by the map application 210. The electronic device 200 forwards an indication of the relevant information to the second electronic device 205, in this example a smart watch, for display of the relevant information, in this example the entire text message, by the second electronic device 205. As a result, the user is provided with all the relevant information for navigating to the meeting location without being required to switch back and forth between applications or memorize the information.
In various embodiments, the user can forward the relevant information back to the first electronic device for use of the information in the application on the first electronic device. For example, performing some action, such as, a button press on the relevant information on the second electronic device triggers action on the first electronic device. This action may also be a hold-and-swipe, to forward the relevant information to a relevant application field on the first electronic device. For example, without limitation, the forwarded information may be copied to a clipboard for use with the application on the first electronic device, inserted into fillable fields (e.g., shipping or payment information fields) of the application on the first electronic device, may be inserted and used as in input (e.g., a PIN or search input).
Various embodiments of the present disclosure provide identification of information a user finds relevant for various primary device applications through the use of context. In one embodiment, the methods and electronic devices of the present disclosure can detect which information is relevant to the user’s overall task and is not present in the new application. For example, the electronic device may track the task being performed by the user between application switches and shift a portion of the UI switched from to another electronic device based on an estimation of the user task. This may include the electronic device identifying a type of relationship between the old and new applications and/or the mechanism used for switching between applications. For example, whenever a link is selected to switch between applications, the electronic device can identify the type of link selected (e.g., a URL, a recognized address, a recognized phone number, etc.) and the type of application of the switched from and switched to applications. In another example, the electronic device may track a sequence, type, and relationship between or among of applications switched between or among either manually or automatically, within a given period of time to estimate the user task.
Using this identified contextual information, the electronic device may estimate the overall task being performed by the user search the switched from application to determine whether any information relevant to the overall user task is located in the switched from application but not otherwise available in the switched to application. For example, the electronic device may perform a text search for information that may be relevant to a user application, which may include, for example, optical character recognition of text in images. In these examples, the electronic device may forward the relevant information by forwarding the UI for the prior application, certain text from the prior application, or an image from the prior application for display by the second electronic device. The electronic device sends an indication of the relevant information that may include, for example, the relevant information itself an indication of the relevant information for the second electronic device to obtain the relevant information. For example, the first electronic device may send a message identifier or application identifier for the second electronic device to launch a corresponding application and/or retrieve a corresponding message, for example, from a server.
In some embodiments, in addition to or instead of the inter-application switch context information, the electronic device may identify information about the context of the task being performed using inputs from sensors (e.g., such as sensor(s) 165) located on one or more of the first electronic device, the second electronic device, or sensors in the environment. For example, the second electronic device may include and use sensor(s) to identify audio played by the first electronic device and display information relevant to the audio played (e.g., purchase information or related products to a movie, television show, or song played on the first electronic device). In another example, the electronic device may receive information from sensors that are part of home automation equipment or system, such as for example, to estimate the task being performed by the user. In another example, the electronic device may identify a location of the electronic device and estimate the user task based on tasks performable at the identified location of the electronic device, and possibly also based on the application executing on the electronic device.
In some embodiments, the electronic device may identify the relevant information based on default or manual user preferences. For example, the user may have pre-established preferences, that when selecting a link switching to a new application, certain information from or the UI for the switched application is forwarded or “pushed” for display on the second electronic device. In other examples, the user preference may be specific to particular applications that are the switched from and/or switched to applications.
FIG. 3 illustrates a process for managing information context among devices in accordance with various embodiments of the present disclosure. For example, the process depicted in FIG. 3 may be performed by the electronic device 105 in FIG. 1. The process may also be implemented by either of the electronic devices 200 or 205 in FIG 2.
The process begins with the electronic device displaying information of a first application (step 305). For example, in step 305, the electronic device may display a UI for the first application that includes a selectable link triggering a switch to a second application. The electronic device determines whether a switch from first application to a second application is requested (step 310). For example, in step 310, the electronic device may detect a selection of a link or a manual application switch by a user.
The electronic device determines whether relevant information from the first application can be used in the second application (step 315). For example, in step 315, the electronic device may search the first application to determine whether missing or helpful information displayed in the UI for the first application is not available or displayed in the UI for the second application, for example, by a comparison of the information displayed for the respective applications.
If the electronic device determines that relevant information from the first application can be used in the second application, the electronic device identifies relevant information of the first application (step 320). For example, in step 320, the electronic device may identify the relevant information based on a setting indicating a preference for the relevant information to be displayed at the second electronic device when switching from the first application to the second application. In another example, the electronic device may estimate a task being performed using the second application and identify the relevant information of the first application based on the estimated task. In another example, the electronic device may identify a relationship between the first and second applications based on properties of the first and second applications, respectively (e.g., tasks performed by the respective applications, a link selected to cause the switch between the applications) and identify the relevant information of the first application based on the identified relationship. In another example, the electronic device may identify context information based on an input from one or more sensors located in the electronic device, a second electronic device, or in the environment and identify the relevant information of the first application based on the identified context information.
The electronic device sends an indication of the relevant information to a second electronic device (step 325). For example, in step 325, the electronic device may send the indication for the second electronic device to display of the relevant information at the second electronic device. The indication may include the relevant information as text, an application UI, an image, etc. In other examples, the indication may include an indicator of an application to launch and/or a message or image to retrieve, for example, from a server.
Thereafter, the electronic device displays information of the second application (step 330). The electronic device then determines whether input information from the second electronic device has been received (step 335). For example, in step 335, the electronic device may receive an input that includes at least a portion of the relevant information displayed at the second electronic device.
The electronic device then uses the input information in the second application (step 340). For example, in step 340, the electronic device may copy the received information to a clipboard for a user to paste into a relevant portion, display the relevant information, for example, in a notification or pop-up, and/or populate relevant fields for the second application (e.g., fill-in shipping or address fields, enter the information into a search field and process a search, enter the information as a PIN, etc.)
FIG. 4 illustrates another process for managing information context among devices in accordance with various embodiments of the present disclosure. For example, the process depicted in FIG. 4 may be performed by the electronic device 105 in FIG. 1. The process may also be implemented by either of the electronic devices 200 or 205 in FIG 2. In this illustrative embodiment, the electronic device performing the process illustrated in FIG. 4 may be the second electronic device referred to in the process of FIG. 3 and vice versa.
The process begins with the electronic device identifying relevant information of a first application executed on a second electronic device (step 405). For example, in step 405, the electronic device may receive, from the second electronic device, an indication of the relevant information of the first application executed on the second electronic device that was switched from. In another example, the electronic device may send a request to the second electronic device for the indication of the relevant information of the first application executed on the second electronic device that was switched from, for example, based on a user input or an indication that the user is looking at the display of the electronic device (e.g., detecting an inclination angle or range of the electronic device using an accelerometer or gyroscope, processing image data captured by a camera of the electronic device showing eyes of the user looking in a direction of the display of the electronic device, etc. In another example, the electronic device may estimate the task being performed at the second electronic device using inputs from sensors (e.g., sensor(s) 165) and identify the relevant information of the first application based on the estimated task being performed at the second electronic device. For example, the electronic device may identify context information for the task being performed based on audio inputs sensed by a microphone and identify a song, movie, or television show using the context information, where the first electronic device is a television or media player.
The electronic device displays the relevant information (step 410). For example, in step 410, the electronic device may display information from the first application switched from and/or information determined to be relevant to the task the user is performing at the second electronic device. The electronic device determines whether a user input has been received with respect to the displayed relevant information (step 415). For example, in step 415, the electronic device may detect a touch of a portion of the displayed information or a swiping motion on the display of the electronic device toward a direction the second electronic device.
The electronic device sends at least a portion of the displayed relevant information as an input to the second electronic device (step 420). For example, in step 420, the electronic device may send the portion of the relevant information to the first application or a second application executed on the second electronic device in response to a user input.
Although FIGS. 3 and 4 illustrate examples of processes for managing information context among devices various changes could be made to FIGS. 3 and 4. For example, while shown as a series of steps, various steps in each figure could overlap, occur in parallel, occur in a different order, occur multiple times, or be omitted in one or more embodiments of the present disclosure.
Although the present disclosure has been described with an exemplary embodiment, various changes and modifications may be suggested to one skilled in the art. It is intended that the present disclosure encompass such changes and modifications as fall within the scope of the appended claims.
None of the description in this application should be read as implying that any particular element, step, or function is an essential element that must be included in the claim scope. The scope of patented subject matter is defined only by the claims. Moreover, none of the claims is intended to invoke 35 U.S.C. § 112(f) unless the exact words “means for” are followed by a participle.
Claims (15)
- A method for operating an electronic device, the method comprising:switching from displaying information of a first application to displaying information of a second application;identifying information of the first application that is relevant to the second application, the relevant information including at least a portion of the displayed information of the first application; andsending an indication of the relevant information to a second electronic device for display of the relevant information at the second electronic device.
- The method of claim 1, further comprising:while displaying the information of the second application, receiving input information from the second electronic device, the input information comprising at least a portion of the relevant information displayed at the second electronic device; andusing the input information in the second application.
- The method of claim 1, wherein identifying the relevant information comprises identifying the relevant information based on a setting indicating a preference for the relevant information to be displayed at the second electronic device when switching from the first application to the second application.
- The method of claim 1, wherein identifying the relevant information comprises:estimating a task being performed using the second application; andidentifying the relevant information of the first application based on the estimated task.
- The method of claim 1, wherein identifying the relevant information comprises:identifying a relationship between the first and second applications based on properties of the first and second applications, respectively; andidentifying the relevant information of the first application based on the identified relationship.
- The method of claim 1, wherein identifying the relevant information comprises:identifying context information based on an input from one or more sensors; andidentifying the relevant information of the first application based on the identified context information.
- The method of claim 6, wherein the one or more sensors are located in the second electronic device.
- The method of claim 1, wherein:identifying the relevant information comprises identifying the relevant information based on a user input while the information of the first application is displayed; andsending the indication of the relevant information to the second electronic device comprises sending the relevant information to the second electronic device in response to the user input.
- An electronic device comprising:a display;at least one processor, the at least one processor configured to:cause the display to switch from displaying information of a first application to displaying information of a second application; andidentify information of the first application that is relevant to the second application, the relevant information including at least a portion of the displayed information of the first application; anda transceiver configured to send an indication of the relevant information to a second electronic device for display of the relevant information at the second electronic device.
- The electronic device of claim 9, wherein:the transceiver is configured to receive, while the information of the second application is displayed, input information from the second electronic device, the input information comprising at least a portion of the relevant information displayed at the second electronic device; andthe at least one processor is configured to use the input information in the second application.
- The electronic device of claim 9, wherein the at least one processor is configured to identify the relevant information based on a setting indicating a preference for the relevant information to be displayed at the second electronic device when switching from the first application to the second application.
- The electronic device of claim 9, wherein the at least one processor is configured to:estimate a task being performed using the second application; andidentify the relevant information of the first application based on the estimated task.
- The electronic device of claim 9, wherein the at least one processor is configured to:identify a relationship between the first and second applications based on properties of the first and second applications, respectively; andidentify the relevant information of the first application based on the identified relationship.
- The electronic device of claim 9, wherein the at least one processor is configured to:identify context information based on an input from one or more sensors; andidentify the relevant information of the first application based on the identified context information.
- The electronic device of claim 9, wherein:the at least one processor is configured to identify the relevant information based on a user input while the information of the first application is displayed; andthe transceiver is configured to send the relevant information to the second electronic device in response to the user input.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020187014892A KR20180075606A (en) | 2015-11-25 | 2016-11-08 | Manage display of information for multiple devices based on context for user tasks |
EP16868813.3A EP3345415A4 (en) | 2015-11-25 | 2016-11-08 | Managing display of information on multiple devices based on context for a user task |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/952,798 US10078482B2 (en) | 2015-11-25 | 2015-11-25 | Managing display of information on multiple devices based on context for a user task |
US14/952,798 | 2015-11-25 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017090916A1 true WO2017090916A1 (en) | 2017-06-01 |
Family
ID=58721849
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/KR2016/012782 WO2017090916A1 (en) | 2015-11-25 | 2016-11-08 | Managing display of information on multiple devices based on context for a user task |
Country Status (4)
Country | Link |
---|---|
US (1) | US10078482B2 (en) |
EP (1) | EP3345415A4 (en) |
KR (1) | KR20180075606A (en) |
WO (1) | WO2017090916A1 (en) |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009143294A2 (en) | 2008-05-20 | 2009-11-26 | Citrix Systems, Inc. | Methods and systems for using external display devices with a mobile computing device |
US20100262673A1 (en) | 2009-04-14 | 2010-10-14 | Jae Young Chang | Terminal and controlling method thereof |
US20110167340A1 (en) * | 2010-01-06 | 2011-07-07 | Bradford Allen Moore | System and Method for Issuing Commands to Applications Based on Contextual Information |
EP2562638A1 (en) | 2011-08-23 | 2013-02-27 | Samsung Electronics Co., Ltd. | Display method and apparatus in portable terminal |
US20140122726A1 (en) | 2012-10-26 | 2014-05-01 | Microsoft Corporation | Updating services during real-time communication and sharing-experience sessions |
US20140173625A1 (en) * | 2012-12-13 | 2014-06-19 | Microsoft Corporation | Task completion through inter-application communication |
WO2014137200A1 (en) | 2013-03-08 | 2014-09-12 | Lg Electronics Inc. | Mobile terminal and control method thereof |
US20140365513A1 (en) * | 2010-10-15 | 2014-12-11 | Microsoft Corporation | Providing information to users based on context |
US20150019665A1 (en) * | 2011-03-15 | 2015-01-15 | Idt Messaging, Llc | Linking context-based information to text messages |
US20150058869A1 (en) * | 2013-08-23 | 2015-02-26 | Microsoft Corporation | Reverse launch protocol |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7693861B2 (en) * | 2005-06-28 | 2010-04-06 | Microsoft Corporation | Schematization of establishing relationships between applications |
KR101685364B1 (en) * | 2010-01-05 | 2016-12-12 | 엘지전자 주식회사 | Mobile terminal, Mobile Terminal System and operation control method thereof |
KR102014778B1 (en) * | 2012-12-14 | 2019-08-27 | 엘지전자 주식회사 | Digital device for providing text messaging service and the method for controlling the same |
US9185062B1 (en) * | 2014-05-31 | 2015-11-10 | Apple Inc. | Message user interfaces for capture and transmittal of media and location content |
US10075536B2 (en) * | 2015-04-09 | 2018-09-11 | Apple Inc. | Transferring a pairing from one pair of devices to another |
-
2015
- 2015-11-25 US US14/952,798 patent/US10078482B2/en active Active
-
2016
- 2016-11-08 EP EP16868813.3A patent/EP3345415A4/en not_active Withdrawn
- 2016-11-08 KR KR1020187014892A patent/KR20180075606A/en unknown
- 2016-11-08 WO PCT/KR2016/012782 patent/WO2017090916A1/en unknown
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2009143294A2 (en) | 2008-05-20 | 2009-11-26 | Citrix Systems, Inc. | Methods and systems for using external display devices with a mobile computing device |
US20100262673A1 (en) | 2009-04-14 | 2010-10-14 | Jae Young Chang | Terminal and controlling method thereof |
US20110167340A1 (en) * | 2010-01-06 | 2011-07-07 | Bradford Allen Moore | System and Method for Issuing Commands to Applications Based on Contextual Information |
US20140365513A1 (en) * | 2010-10-15 | 2014-12-11 | Microsoft Corporation | Providing information to users based on context |
US20150019665A1 (en) * | 2011-03-15 | 2015-01-15 | Idt Messaging, Llc | Linking context-based information to text messages |
EP2562638A1 (en) | 2011-08-23 | 2013-02-27 | Samsung Electronics Co., Ltd. | Display method and apparatus in portable terminal |
US20140122726A1 (en) | 2012-10-26 | 2014-05-01 | Microsoft Corporation | Updating services during real-time communication and sharing-experience sessions |
US20140173625A1 (en) * | 2012-12-13 | 2014-06-19 | Microsoft Corporation | Task completion through inter-application communication |
WO2014137200A1 (en) | 2013-03-08 | 2014-09-12 | Lg Electronics Inc. | Mobile terminal and control method thereof |
US20150058869A1 (en) * | 2013-08-23 | 2015-02-26 | Microsoft Corporation | Reverse launch protocol |
Non-Patent Citations (1)
Title |
---|
See also references of EP3345415A4 |
Also Published As
Publication number | Publication date |
---|---|
US10078482B2 (en) | 2018-09-18 |
KR20180075606A (en) | 2018-07-04 |
EP3345415A1 (en) | 2018-07-11 |
US20170147269A1 (en) | 2017-05-25 |
EP3345415A4 (en) | 2018-07-11 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2018222018A1 (en) | Systems and methods for window control in virtual reality environment | |
WO2022156368A1 (en) | Recommended information display method and apparatus | |
US10769797B2 (en) | Virtual reality experience sharing | |
CN113489937B (en) | Video sharing method, device, equipment and medium | |
WO2017135797A2 (en) | Method and electronic device for managing operation of applications | |
US20230176806A1 (en) | Screen Projection Display Method and System, Terminal Device, and Storage Medium | |
CN112350981B (en) | Method, device and system for switching communication protocol | |
WO2018070803A1 (en) | Method and apparatus for session control support for field of view virtual reality streaming | |
CN109697008B (en) | Content sharing method, terminal and computer readable storage medium | |
EP3345401A1 (en) | Content viewing device and method for displaying content viewing options thereon | |
CN114650264B (en) | Unread message display method and device, electronic equipment and storage medium | |
CN114422461B (en) | Message quoting method and device | |
CN103051865A (en) | Method and terminal for controlling picture and video meeting device | |
US10067656B2 (en) | Method and apparatus for context based application grouping in virtual reality | |
KR20140137736A (en) | Method and apparatus for displaying group message | |
EP3583789A1 (en) | Systems and methods for notification delivery | |
CA2946372A1 (en) | Access management system, communication system, recording medium, and access management method | |
CN112333337B (en) | Message checking method, device, equipment and storage medium | |
CN111367444A (en) | Application function execution method and device, electronic equipment and storage medium | |
CN109067975B (en) | Contact person information management method and terminal equipment | |
CN114064173A (en) | Method, device, medium and equipment for creating instant session page | |
CN111131540B (en) | Name setting method and electronic equipment | |
WO2023046105A1 (en) | Message sending method and apparatus and electronic device | |
WO2022228270A1 (en) | Information processing method, information processing apparatus, electronic device and readable storage medium | |
WO2017090916A1 (en) | Managing display of information on multiple devices based on context for a user task |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16868813 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 20187014892 Country of ref document: KR Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |