KR20200131250A - Method and apparatus for saving and restoring navigation context - Google Patents

Abstract

The present invention relates to a method of restoring a navigation context for a first media content. The navigation context data includes media access data and identification of a set of one or more media objects of interest in a first user interface, a UI, a first portion of the first media content currently displayed in the window. Restoration of the navigation context includes loading (269) a first media content based on media access data, a second media content comprising a set of one or more media objects of interest based on the loaded first media content Generating (270), and in a second user interface window, displaying (290) a second portion of the second media content including at least one media object of interest from among the set of one or more media objects of interest. Includes.

Description

Method and apparatus for saving and restoring navigation context

The present invention relates to the field of communication and digital data processing, and more particularly, to a method of storing a navigation context, a method of restoring a navigation context, and an electronic device and a computer program associated therewith.

The US patent application published as US 2017/0214640A1 discloses a method of sharing media content among multiple users. The method is combined with topic media content to specify which portion of the topic media content the contributing media content is associated with and/or how the topic media content should be combined with each contributing media content to play the annotated media content. And generating one or more contributing media content that generates media content annotated according to metadata (eg, associated data).

The topic media content must be transmitted to a recipient device to be combined with one or more annotated media content where the topic media content will be played, for example from the sending device.

In the disclosed method, the topic media content may be any media content displayed in a software application. For example, the media content is a "static" web page (including text, photos, videos, interactive features/tools, etc.) where the content remains the same for a given URL (Uniform Resource Location). Can be In this case, the topic media content can be played on the receiver device simply by loading the web page from the media server using the URL of the web page. Conversely, the topic media content may also be dynamically updatable media content for the same URL, in which the content may change over time or according to the user location/user profile of the user in which the content is displayed. In this latter case, if only the URL of the web page is used, it may not be sufficient to restore a web page whose state and content are the same as the initial web page. For example, the content of a web page corresponding to a given URL may depend on the user profile or may be updated by the source media server. For example, searching for web pages using a search engine based on keywords may generate a list of web pages depending on the user profile and/or the user's cookie and/or the date the search was performed.

Further, in the disclosed method, a portion of the topic media content referenced by the contributing media content may be identified using coordinates in, for example, a two-dimensional coordinate system associated with a web page. However, the size and/or resolution of the display screen of the recipient device may be different from that of the transmitting device: therefore, the coordinate system is completely different and the stored coordinates may not be sufficient to play a web page with the same content (at least visually). As such, the layout on the screen may be different for each display screen). In addition, the web page loaded using the URL of the source page may be modified according to the type (mobile version) of the target device on which the web page is to be displayed.

Also, privacy issues can arise in at least some of the web pages to be shared. For example, a web page displayed to a user on a social network, blog, or microblogging website may be one or more media objects that depend on entered keywords, subscriptions, user profile and/or relationships with other users of the social network. May include (eg, publications, feeds, messages, search results, etc.). The web page may contain, for example, media objects ready to be shared by the relevant user and other media items that the subscribed user does not want to share.

One solution is to take a screenshot of the web page to be shared, or to create at least part of the web page to be shared. However, a screenshot is a "dead copy" of a web page starting from these screenshots and, as long as no interaction and/or navigation is possible. In this case, the receiving device cannot restore the interaction context.

Accordingly, there is a need to restore the context of the navigation, which stores the context of the navigation and/or the interaction of the media content and/or the context of the navigation.

In general, in one aspect, the present invention relates to a method for restoring a navigation context. The method includes storing a navigation context, storing media access data that enables access to a first media content displayed in the software, detecting, in the first media content, a set of one or more media objects. Identifying, in the set of one or more media objects, a set of one or more media objects of interest in a first user interface, a UI, a first portion of the first media content currently visible in a window, and the one or more And storing the navigation context, including storing the identification of media objects of interest. The method includes restoring the navigation context, loading the first media content based on the media access data, generating second media content based on the loaded first media content, the method comprising: The second media content comprises a set of one or more restored media objects of interest corresponding to the identified one or more media objects of interest, and is currently visible in a second UI window. Restoring the navigation context, including displaying the second media content such that the second portion includes at least one restored media object of interest from the set of one or more restored media objects of interest. Include.

According to a second aspect, the present invention relates to a method for restoring a navigation context. The method comprises obtaining navigation context data, wherein the navigation context data is media access data enabling access to a first media content, the first media content comprising a set of one or more media objects, Obtaining the media access data and the navigation context data comprising identification of a set of one or more media objects of interest displayed in a first portion of the first media content in a first user interface UI window during navigation. Include. The method includes restoring the navigation context, loading the first media content based on the media access data, generating second media content based on the loaded first media content, the method comprising: Generating second media content based on the loaded first media content, wherein the second media content comprises a set of one or more reconstructed media objects of interest corresponding to the identified one or more media objects of interest And displaying the second media content such that a second portion currently visible in a second UI window includes at least one restored media object of interest from among the set of the one or more restored media objects of interest. And restoring the navigation context.

According to a third aspect, the present invention relates to a method for restoring a navigation context of software, the method comprising: transmitting media access data to a recipient device to enable access to a first media content displayed on the software. Step, in the first media content, detecting a set of one or more media objects, in the set of one or more media objects, a first portion of the first media content currently visible in a first user interface, UI, window Identifying a set of one or more media objects of interest at, transmitting the identification of the one or more media objects of interest to the recipient device, and restoring the navigation context on the recipient device.

Restoring the navigation text may include loading the first media content based on the media access data, generating second media content based on the loaded first media content, the second media The content includes a set of one or more reconstructed media objects corresponding to the identified one or more media objects of interest, generating the second media content, and a second part currently visible in the second UI window And displaying the second media content to include at least one restored object of interest from the set of one or more restored media objects of interest.

According to a fourth aspect, the present invention includes a processor, and a memory operatively coupled to the processor, the memory causing the processor to cause the processor according to the first, second, or third aspect when executed by the processor. It relates to an electronic device comprising computer readable instructions for causing to perform steps of a method.

According to a fourth aspect, the present invention relates to a computer program product comprising computer readable instructions that, when executed by a computer, cause the computer to perform steps of a method according to the first, second, or third aspect. will be.

Further aspects and embodiments of the method according to the first, second or third aspect are defined by the dependent claims.

The invention will be better understood and its many aspects and advantages will become more apparent to those skilled in the art by referring to the following figures in connection with the appended specification:
1A is a schematic diagram of a system for sharing media content in accordance with one or more embodiments;
1B is a schematic diagram of an electronic device and a computing server in accordance with one or more embodiments;
2A-2B each show a flowchart of an exemplary method according to one or more embodiments;
3A-3B each show a flowchart of an exemplary method according to one or more embodiments;
4A-4C illustrate user interfaces in accordance with one or more embodiments;
5A-5C illustrate user interfaces in accordance with one or more embodiments;
6A-6C illustrate user interfaces in accordance with one or more embodiments;
7A-7D illustrate user interfaces in accordance with one or more embodiments;
8A-8C illustrate user interfaces in accordance with one or more embodiments.

The present invention relates to an apparatus, system and method for storing and/or restoring a navigation context. Different embodiments are disclosed herein.

The storage and/or restoration method can be applied to a system for sharing media content between users as described in US2017/214640A1. The storage and/or restoration method may be applied to any system implying the transmission of one or more media content to be played by the recipient electronic device.

Other advantages and other features of the components disclosed herein will become more readily apparent to those skilled in the art. The following detailed description of specific preferred embodiments taken in connection with the drawings indicates representative embodiments of the present technology, and like reference numerals indicate similar structural elements.

In addition, it should be apparent that the teachings of this specification may be implemented in various forms and any specific structure and/or function disclosed herein is merely representative. In particular, those skilled in the art will recognize that the embodiments disclosed herein can be implemented independently of any other embodiment, various embodiments can be combined in various ways, and one or more aspects of different embodiments can be combined in various ways. I will understand.

The invention is described below with reference to methods, systems, and functions of a computer program, engines, block diagrams, and flow charts in accordance with one or more exemplary embodiments. The blocks and flow charts of each of the described function, engine, block diagrams may be implemented in hardware, software, firmware, middleware, microcode, or any suitable combination thereof. When implemented in software, functions, engines, blocks of block diagrams and/or flowcharts may be implemented as computer program instructions or software code, which may be stored or transmitted through computer-readable media, or may be stored on a general purpose computer. , Can be loaded onto a special purpose computer or other programmable data processing device to create a machine, whereby computer program instructions or software code executed on the computer or other programmable data processing device implement the functions described herein. Create means for

Embodiments of computer-readable media include, but are not limited to, computer storage media and communication media including any media that facilitate transfer of a computer program from one place to another. Specifically, software instructions or computer-readable program code for performing the embodiments described herein are temporarily or permanently on a non-transitory computer-readable media of a local or remote storage device including one or more storage media, It can be stored in whole or in part.

As used herein, computer storage media can be any physical media that can be read, written, or more generally accessed by a computer. Examples of computer storage media include flash drives or other flash memory devices (e.g., memory keys, memory sticks, key drives), CD-ROM or other optical storage, DVD, magnetic disk storage or other magnetic storage devices, Carrying program code in the form of solid state memory, memory chips, RAM, ROM, EEPROM, smart cards, RDBMS (Relational Database Management System), traditional databases, or instructions or data structures that can be read by a computer processor or Includes, but is not limited to, any other suitable media that can be used to store. In addition, various types of computer-readable media including routers, gateways, servers, or other transmission devices, wired (coaxial cables, fiber, twisted pair, DSL cables) or wireless (infrared, radio, cellular, microwave) It can be used to send or convey commands to. The above instructions may include code from any computer-programming language, including but not limited to assembly, C, C++, Basic, SQL, MySQL, HTML, PHP, Python, Java, Javascript, and the like.

With reference to the drawings, like numbers indicate like parts across the various drawings, and FIG. 1 shows an exemplary system 100 in which various techniques and techniques described herein may be implemented.

System 100 includes one or more computing servers 103A-103E and a plurality of electronic devices 104A-104C (e.g., user devices) operatively and communicatively coupled to each other via a network 105. Includes.

Network 105 is any data transmission network, for example, a wired (coaxial cable, fiber, twisted pair, DSL cable, etc.) or wireless (radio, infrared, cellular, microwave, etc.) network, a local area network (LAN), Internet area network (IAN), metropolitan area network (MAN) or wide area network (WAN) such as the Internet, public or private network, virtual private network (VPN), telecommunication network with data transfer functions, Wi-Fi or Bluetooth cell It may be a single radio cell with a single point of access, such as.

Each electronic device 104A-104C may be implemented as a single hardware device in the form of, for example, a desktop personal computer (PC), laptop, personal digital assistant (PDA), smart phone, or having wired and/or wireless segments. , May be implemented on separate interconnected hardware devices coupled to each other by a communication link.

Each electronic device 104A-104C generally operates under the control of an operating system and executes or relies on various computer software applications, components, programs, objects, modules, data structures, and the like.

As schematically shown in FIG. 1B, in one or more embodiments, the electronic devices 104A-104C include a processor 110, a memory 111, one or more computer storage media 112, input/output interfaces ( For example, device interfaces such as USB interfaces, network interfaces such as Ethernet interfaces, etc.), and a media drive 113 for reading and writing one or more computer storage media 112. .

The memory 111 of the electronic devices 104A-104C is a random access memory (RAM), a cache memory, a nonvolatile memory, a backup memory (e.g., programmable or flash memories), read-only memories, or these May be any combination of. Processor 110 of electronic devices 104A-104C may be any suitable microprocessor, integrated circuit, or central processor (CPU) including at least one hardware-based processor or processing core.

In one or more embodiments, each of the computer storage media or media 112 of the electronic devices 104A-104C, when executed by the processor 110, allows the electronic devices 104A-104C to connect to the electronic devices 104A-104C. Computer program instructions that cause one or more of the methods described herein to be performed. The processor 110 of the electronic devices 104A-104C, when executed by the processor, is configured to generate computer program instructions or software code that, when executed by the processor, cause the processor to perform the steps of the methods described herein on the electronic devices 104A-104C. It may be configured to access the one or more computer storage media 112 for storage, reading and/or loading. The processor 110 of the electronic devices 104A-104C, for example, to load computer program instructions and store data generated during execution of the computer program instructions, the method described herein for the electronic devices 104A-104C. It may be configured to use the memory 111 of the electronic devices 104A to 104C when executing the steps of.

In addition, each of the electronic devices 104A to 104C generally receives a number of inputs and outputs for communicating information externally. For interfacing with users 101A-101C or an operator, electronic devices 104A-104C generally include one or more user input/output devices, e.g. a keyboard, pointing device, display, printer, etc. It includes (114). Otherwise, user input is from one or more external computers, for example one or more electronic devices 104A-104C or other computing servers 103A-103E, for example via a network interface coupled to the network 105. Can be received.

Returning to Fig. 1A, each computing server 103A-103E may be implemented as a single hardware device, or on separate interconnected hardware devices connected to each other by a communication link having wired and/or wireless segments. Can be. Each of the computing servers 103A to 103E may be implemented in a cloud computing environment.

In one or more embodiments, the set of computing servers 103A-103E includes at least three media servers 103A-103C, a database server 103D and a front-end server 103E.

Each computing server 103A-103E generally operates under the control of an operating system and executes or relies on various computer software applications, components, programs, objects, modules, data structures, and the like.

1B, in one or more embodiments, each computing server 103A-103E includes a processor 120, a memory 121, one or more computer storage media 122, and input/output. Interfaces (e.g., device interfaces such as USB interfaces, network interfaces such as Ethernet interfaces, etc.), other associated hardware such as media drive 123 for reading and writing one or more computer storage media 122 Includes.

The memory 121 of the computing servers 103A-103E may include random access memory (RAM), cache memory, nonvolatile memory, backup memory (e.g., programmable or flash memories), read-only memories, or these May be any combination of. Processor 120 of electronic devices 103A-103E may be any suitable microprocessor, integrated circuit, or central processor (CPU) including at least one hardware-based processor or processing core.

In one or more embodiments, each of the computer storage media or media 122 of the computing servers 103A-103E, when executed by the processor 120, the computing servers 103A-103E to the computing servers 103A-103E. Computer program instructions that cause one or more of the methods described herein to be performed. The processor 120 of the computing servers 103A-103E, when executed by the processor, may contain computer program instructions or software code that, when executed by the processor, cause the processor to perform the steps of the methods described herein on the computing servers 103A-103E. It may be configured to access the one or more computer storage media 112 for storage, reading and/or loading. The processor 120 of the computing servers 103A-103E, the method described herein for the computing servers 103A-103E, for example to load computer program instructions and store data generated during execution of the computer program instructions. It can be configured to use the memory 121 of the computing servers 103A to 103E when executing the steps of.

Each electronic device 104A-104C is operatively connected to one or more computing servers 103A-103E via a network 105. Each electronic device 104A to 104C is configured to communicate with at least one of the computing servers 103A to 103E through at least one communication link.

In one or more embodiments, each electronic device 104A-104C, when executed by the processor of the electronic device, causes the processor to perform the method steps described herein on any of the electronic devices 104A-104C. Execute computer program instructions of the software application 106 (also referred to as "client application 106").

In one or more embodiments, computing server 103E is a server that, when executed by one of computing servers 103E, causes such processor to perform the method steps described herein on one of computing servers 103E. It is configured to execute computer program instructions of the software application 107 (also referred to as "server application 107").

In one or more embodiments, the server application 107 is executed by the front-end server 103E, which itself is used to implement the steps of the method described herein for the server application 107. 103C and 103D).

Each instance of the client software application 106 and the server software application 107 is configured to be operatively coupled to each other to communicate in a client/server mode over at least one communication link. The communication link between the client software application 106 and the server software application 107 may use any suitable communication protocol. For example, a HTTP (Hyper Text Transfer) based protocol such as a web service communication protocol such as SOAP (simple object access protocol) may be used. Any other protocol can be used, for example a proprietary protocol.

In one or more embodiments, the software application 106 of the electronic device 104A-104C is operatively connected to the server application 107 to implement a method for storing and restoring the navigation context(s). The software application 106 includes computer program instructions for communicating via messages with the server application 107. In one or more embodiments, software application 106 includes computer program instructions for generating messages, sending to server application 107, and receiving and processing messages received from server application 107.

In one or more embodiments, the software application 106 is software for displaying and navigating the world wide web, intranet, Internet, and the like, for example. However, this description is applicable to any kind of software application for displaying media content.

In one or more embodiments, the user interface of the software application 106 of the electronic device 104A may include at least one user interface item for creating and/or selecting one or more media content. For example, the user interface may include user interface items (buttons, menus, etc.) for triggering display of the user interface for searching and/or selecting one or more media content. For example, the user interface includes a toolbar for creating or importing media content.

Each media server 103A-103C is operatively coupled to one or more media content databases 102A-102C to store media content and associated metadata. Each media server 103A to 103C requests data for media content stored in one media content database 102A to 102C, for example, a request for storing media content in one media content database 102A to 102C. It receives and processes, and provides an interface for retrieving, searching, and modifying media content stored in one media content database 102A to 102C.

Each media server 103A-103C is from any web server application, a remote server application, a storage server in the cloud, or more generally from a remote device for storing and retrieving media content and uploading or downloading one or more media content. May be any software application configured to process the respective download requests or upload requests. For example, the media server 103A is a web server accessible via the Internet from all electronic devices 104A to 104C, and the media server 103B is a media server in a cloud computing environment, which is the electronic devices 104A. To 104C).

In one or more embodiments, the media server 103C is a content delivery network (CDN) for storing content that should be available from any of the electronic devices 104A-104C as well as the front-end server 103E. Can be implemented as The media server 103C is used to store media content, which is one of the electronic devices 104A to 104C or media server 103A when such media content should be accessible from each of the electronic devices 104A to 104C. To 103B).

In the context of the present invention, media content may include video content, audio content, text content, image content, graphic content or any combination thereof, for example web pages or multimedia content. The video content may be 3D content, stereoscopic 3D video content, 2D video content, a list of independently coded images, and the like. The audio content may include mono audio content, stereo audio content, 3D audio content, and the like. Media content may be coded in any format including, for example, JPEG, TIFF, MPEG, WAV, DOC, HTM, OBJ, DAE, FBX, DXF, X3D, MOV (QUICKTIME®), and the like.

Media content data may be encoded within one or more data files. For example, video content can be encoded as a group of images, and each image is encoded into a separate data file.

In one or more embodiments, the first media content (e.g., web page) is displayed in a user interface (UI) window of the software application 106 on the source electronic device (e.g., electronic device 104A). . The user 101A of the source electronic device is searching for the first media content by scrolling the first media content or triggering operations, for example. At some point during the navigation, for example, when performing a given interaction, at the request of the user 101A, or when some predetermined condition is satisfied, a navigation that defines a navigation context for the currently displayed first media content The context data is saved.

In one or more embodiments, the software application 106 analyzes the content of the first media content (e.g., HTML code of a web page), detects one or more media objects of interest in the first media content, and detects And determine navigation context data based on the media objects of interest. Restoration of the navigation context for the first media content is performed based on the detected media objects of interest and to restore such one or more media objects of interest.

The media object may be any object included in the first media content such as text, photo, video, menu, hyperlink, graphic, document preview, icon, button, etc. The detected one or more media objects may be objects that are currently visible (ie, currently displayed) in the UI window or objects that are not visible in the current UI window.

The media object may be an interactive media object or a non-interactive media object. For example, menus, hyperlinks, or buttons are common interactive media objects. For example, when a URL is associated with a photo, the photo may be made interactive. The interactive media object may be configured to trigger the execution of one or more interaction functions.

The navigation context data defining a navigation context for the currently displayed first media content may be used to restore the navigation context for this first media content on the destination electronic device (eg, electronic device 104B). The destination electronic device may be distinguishable from the source electronic device or may be the same device.

In one or more embodiments, second media content is generated that at least partially plays the first media content to restore the navigation context. One or more media object(s) of interest may be restored from the second media content so that the visible content of the current UI window (from the same viewpoint of the media content) is the same (or at least similar) to the visible content when the navigation context is stored. I can.

In one or more embodiments, one or more volatile media object(s) may be identified and restored in the second media content, such that the dynamic state of the first media content (e.g., context sensitive content) is It is restored from the content. Therefore, the navigation in the second media content may be performed with the same context parameters as the navigation in the first media content when the navigation context is stored. Volatile media objects refer to the content of the first media content depending on the navigation context. Volatile media objects may be media objects that can be presented in a variable, modified, present/absent, different order depending on one or more context parameters for a given media access data. In one or more embodiments, context parameters may include user location, user cookie, user profile, user rights and/or current time. Non-volatile media objects may refer to permanent content of the first media content that does not depend on context parameters. Non-volatile media objects may be retrieved independently of navigation using media access data, but it may be impossible to retrieve volatile media objects using only media access data.

For example, searching for web pages using a search engine based on keywords may generate a list of web links, which list depends on the user profile and/or the user's cookie and/or the date the search was performed. . Thus, the search results, the list of results and the order in which the results are presented are dependent on the navigation context, e.g. personalization parameters (e.g., user profile, user location and/or user cookie) and/or the date the search was performed Can be variable depending on In any case, in order to restore the navigation context, the resource access data and the sorted result list of all search results on the web page are used to restore the web page to the same result list constituting the search results in the same order. 103E).

For example, on a website for selling products, media objects for presenting a list of products created by a search can be identified as volatile media objects, while presenting a product description indicated by an item in the list. Media objects for use are treated as non-volatile media objects. Thus, the identification of media objects may be suitable for the type of web site or type of web application or may be web site specific. In one or more embodiments, all media objects within a page may be identified as volatile media objects.

The layout of the second media content may be the same as or different from the layout of the first media content. In one or more embodiments, the layout of the second media content is different from the layout of the first media content, for example to fit the resolution and/or size of the display screen of the destination electronic device.

In one or more embodiments, the stored navigation context metadata may be used as a bookmark to enable restoration of the navigation and/or interaction context within the media content when generating the bookmark. In one or more embodiments, a bookmark refers to one or more media objects of interest within a particular perspective and/or given media content. For the same media content, one or more of these bookmarks may be stored to enable restoration of the corresponding navigation context. One or more bookmarks may be stored for different media content.

In one or more embodiments, the navigation context data (or bookmarks) is stored on the computing server 103E so that such navigation context data (or bookmarks) can be shared with different users/different destination electronic devices. In one or more embodiments, the navigation context data (or bookmarks) is stored locally by the source electronic device and may be further transmitted to the destination electronic device for restoration. Navigation context data (or bookmarks) may be stored in memory, a database, configuration files associated with software, and the like.

The user 101B of the destination electronic device may search for the second media content. In one or more embodiments, the interaction function associated with the interactive media object(s) in the second media content is restored.

2A shows a flow diagram of a method for storing a navigation context in accordance with one or more embodiments. While the various steps in the flowchart are presented and described sequentially, those skilled in the art will understand that some or all of the steps may be executed in different orders, may be combined or omitted, and some or all of the steps may be executed in parallel. .

The method steps may be implemented by a source electronic device (eg, electronic device 104A of user 101A) and a storage device for storing navigation context data according to any embodiment disclosed herein. The storage device may be included in the source electronic device or may be a separate device. In one or more embodiments, the storage device may be a computing server (e.g., front-end server 103E), a database 102A-102B, another electronic device 104B and 104C, or be included within such a device. have.

In one or more embodiments, in order to share the media content, the media content displayed on the display screen of the source electronic device 104A is transmitted via the computing server 103E to a plurality of electronic devices 104B to 104C (i.e., Destination electronic devices) to other electronic devices. Computing server 103E manages a database for storing media access data and associated metadata. The associated metadata includes navigation context metadata from which the navigation context can be restored.

In the description below, the method steps are implemented by the client application 106 of the source electronic device 104A and the server application 107 of the front-end server 103E used as a storage device, but any other storage device and Applicable to any other software application for displaying media content.

In step 200, the first media content M1 (eg, a web page) is displayed in the first UI window W1 of the software application 106 on the display screen of the source electronic device 104A. In one or more embodiments, only the first portion of the first media content M1 is visible in the UI window. In one or more embodiments, the software application 106 is configured to initiate a process of storing a navigation context. The process may be started, for example, when performing a given interaction, at the request of the user 101A, or when some predetermined condition is met.

In step 201, the software application 106 is configured to transmit media access data to a storage device (eg, computing server 103E) that allows access to the first media content M1. The media access data may include a web page for accessing the first media content through the media servers 103A, 103B, and 103C, an address and/or a URL for accessing with an identifier. The media access data may include data encoding the first media content M1. The media access data is stored by a storage device (eg, computing server 103E). In one or more embodiments, the navigation context data includes received media access data.

In step 205, the software application 106 is configured to take a screenshot of the first portion of the first media content M1 currently displayed in the UI window W1.

In step 206, the software application 106 is configured to send the screenshot to the storage device (e.g., computing server 103E) for storage by the storage device (e.g., computing server 103E). do. In one or more embodiments, the navigation context data includes a received screenshot.

In step 210, the software application 106 includes text, images, videos, animations, sounds, applets and/or any documents, icons, and/or any documents contained in the media content in the first media content M1. Configured to identify a set of one or more media objects, such as buttons, menus, interactive objects, and the like. In the case of a web page, media objects generally correspond to resources of a web page. For example, in the case of a web page, the web page's code (e.g., HTML code) defines the web address of the web page, the layout of the web page, and resource data that defines the media objects of the web page: They can be identified by analyzing the code of the web page data and the resource access data can be extracted from the code of the web page.

The granularity by which media objects are identified may depend on the application/requirements or coding scheme used for the first media content. In one or more embodiments, the media object is one or more media sub-objects, an object container or a group of objects comprising one or more media objects, one or more media objects arranged in a hierarchical structure, and a media comprising sub-objects. It may be a composite media object including a hierarchical structure of objects, a list of ordered media objects, and the like. For example, the text and associated pictures of a document can be grouped and processed as a single media object or separately processed and identified.

In one or more embodiments, the software application 106 is configured to cause the user 101A of the source electronic device 104A to identify and exclude one or more media objects that the user does not want to share. For example, media objects that are not part of the set of one or more media objects of interest are excluded. In another example, user 101A may select one media object and perform some predetermined operation to request that other media objects be excluded. In one or more embodiments, the one or more excluded media objects are dimmed from the UI window (see, e.g., FIGS. 8A-8B described below), darkened, or removed to the user 101A for exclusion. You can provide feedback. In one or more embodiments, the set of one or more media objects is updated to remove the excluded one or more media objects.

In one or more embodiments, the identification of the media object may be transmitted along with the resource access data of the associated media object. In the case of a web page, the identification of the media object of interest may be extracted from the code of the web page, or as a media object detected by the software application 106 and/or storage device (e.g., computing server 103E). Can be assigned.

In step 215, the software application 106 is configured to identify a set of one or more volatile media objects in the set of one or more media objects identified in step 210. Thus, the identification of media objects may be suitable for the type of web site or type of web application or may be web site specific. In one or more embodiments, all media objects on the page may be identified as volatile media objects.

At step 216, the software application 106 accesses a resource for accessing, for a set of one or more volatile media objects, to resources that enable restoration (e.g., playback) of the set of one or more volatile media objects. It is configured to transmit data to a storage device (eg, computing server 103E). For example, when the first media content is a web page, the resource access data may be pointers (eg, URL) to resources of a related web page. In one or more embodiments, the resource access data may include the resources themselves (eg, data encoding media objects). Data encoding media objects may include, for example, object properties and/or state, data encoding image(s), text, and the like. The resource access data is stored by a storage device (eg, computing server 103E) in association with the media access data. In one or more embodiments, the navigation context data includes received resource access data.

At step 220, the software application 106 is configured to identify a set of one or more interactive media objects associated with one or more interactive functions in the set of one or more volatile media objects. Interaction functions include, for example, loading a web page indicated by a URL associated with an interactive media object, displaying a menu, displaying a sub-window, downloading content, sending an email, launching an application, and , Opening a communication session, and the like. The triggering data may include a hyperlink (eg, URL) on a target web page, a definition of menus and related resources, an interaction function code, and the like. In the case of a web page, triggering data may be extracted from the code of the web page.

In step 221, the software application 106 performs, for each identified interactive media object, the execution of one or more interactive functions associated with the identified interactive media object to a storage device (e.g., computing server 103E). It is configured to transmit triggering data for triggering. The triggering data may include a URL and/or a part of code (eg, HTML code) extracted from the code of the first media content. The triggering data is associated with the media access data and stored by a storage device (eg, computing server 103E). In one or more embodiments, the navigation context data includes received triggering data.

In step 230, the software application 106 stores the coordinates of the first portion of the first media content M1 currently displayed in the first UI (user interface) window (e.g., computing server 103E). ). The coordinates of the first portion are associated with the media access data and stored by a storage device (eg, computing server 103E). In one or more embodiments, the navigation context data includes the received coordinates of the first portion.

The first portion of the first media content M1 currently displayed in the first UI window is an area of interest for the user 101A, for example, an area in which the user 101A can interact or the user 101A Indicates an area in which an annotation can be created or an area that the user 101A wants to share with other users. In one or more embodiments, the coordinates of the first portion are defined relative to the coordinate system associated with the media content. The coordinates of the first part may depend on the resolution and/or size of the display screen of the source electronic device 104A.

In step 240, the software application 106, in the set of one or more media objects identified in step 210, the first portion of the first media content M1 currently displayed in the first UI window of the software application. Is configured to identify a set of one or more media objects of interest. The media object of interest may or may not be interactive. In one or more embodiments, the one or more media objects of interest in the set of one or more media objects of interest are interactive media object(s). Media objects of interest may be volatile or non-volatile. In one or more embodiments, the one or more media objects of interest in the set of one or more media objects of interest are volatile media object(s).

Identification of media objects of interest can be performed in different ways. In one or more embodiments, the identification of the media object of interest includes detecting an interaction of the user 101A on the object of the first portion and identifying the media object from which the interaction was detected. In one or more embodiments, the identification of the media object of interest is determining the viewing direction of the user 101A viewing the first media content and the display area intersects the viewing direction or the display area is the viewing direction and the display screen. Identifying one or more media objects of interest closest to the intersection. In one or more embodiments, the identification of the media object of interest includes determining one or more media objects closest to a central location of the first portion of the first media content. In one or more embodiments, identification of a media object of interest allows the user to select one or more media objects of interest (e.g., by contact on a touch-sensitive display) and identify the selected one or more media objects of interest. Includes that.

In step 241, the software application 106 is configured to transmit the identification of the media object(s) of interest identified in step 240 to a storage device (eg, computing server 103E). The identification of the media object(s) of interest is stored in association with the media access data. In one or more embodiments, the navigation context data includes a received identification of the media object(s) of interest.

In step 250, the software application 106, for each object of interest, for the first portion (area of interest) of the first media content M1 currently displayed in the first UI window W1. It is configured to determine the object coordinates of the present object.

In step 251, the software application 106 is configured to send, for each object of interest, object coordinates of the object of interest to a storage device (eg, computing server 103E). Object coordinates are associated with media access data and stored by a storage device (eg, computing server 103E). In one or more embodiments, the navigation context data includes received object coordinates.

Steps 200 to 251 may be repeated such that each navigation context is stored for another portion of the first media content or a portion of another media content, for example. For each execution of steps 200 to 251, the navigation context data is stored by the storage device (e.g., computing server 103E) and the corresponding navigation context can be restored based on this navigation context data. .

2B shows a flow diagram of a method for restoring a navigation context in accordance with one or more embodiments. While the various steps in the flowchart are presented and described sequentially, those skilled in the art will understand that some or all of the steps may be executed in different orders, may be combined or omitted, and some or all of the steps may be executed in parallel. .

The method steps may be implemented by a destination electronic device (eg, electronic device 104B) and a storage device for storing navigation context data according to any embodiment disclosed herein. The storage device may be included in the source electronic device or may be a separate device. In one or more embodiments, the storage device may be or be included in a computing server (e.g., front-end server 103E), database 102A-102B, another electronic device 104B and 104C. have.

In one or more embodiments, in order to share the media content, the first media content displayed on the source electronic device 104A is transmitted via the computing server 103E to a plurality of electronic devices 104B to 104C (i.e., destination Electronic devices) to other electronic devices. The first media content may be locally generated on the source electronic device 104A or may be loaded from the media servers 1031 to 103C. Computing server 103E manages a database for storing media access data and associated metadata. Associated metadata includes navigation context metadata.

In the description below, the method steps are implemented by the client application 106 of the source electronic device 104A and the server application 107 of the front-end server 103E used as a storage device, but any other storage device and Applicable to any other software application for displaying media content.

In step 260, the software application 106 obtains the navigation context data from the storage device (e.g., received from the database, computing server 103E or source electronic device 104A) and navigates based on the navigation context data. Start context restoration. In one or more embodiments, the navigation context data is generated as described with reference to FIG. 2A.

Navigation context data may include:

Media access data for accessing the first media content M1 through the media server; And/or

A screenshot of the first portion of the first media content M1; And/or

Coordinates of the first part for the first media content M1; And/or

Identification of media object(s) in the first media content; And/or

Identification of media object(s) of interest within the first portion; And/or

Identification of volatile media object(s) in the first media content M1; And/or

Resource access data for restoring a set of one or more volatile media object(s) of the first media content M1; And/or

Triggering data for restoring execution of one or more interaction function(s) associated with one or more interactive media object(s) of the first media content M1

Object coordinates of each media object of interest for the first portion of the first media content.

In step 265, the software application 106 is configured to display the screenshot in the second UI window. In one or more embodiments, the display of the screenshot is performed prior to displaying the second portion of the second media content M2 in the second UI window (see step 290). In one or more embodiments, the screenshot may be used as a visual representation of a bookmark corresponding to the navigation context data.

In one or more embodiments, one or more screenshots are displayed, and a navigation context corresponding to the screenshot is restored upon detection of the user's interaction with the associated screenshot and based on the associated navigation context data.

In step 269, the software application 106 is configured to load the first media content using the media access data. In one or more embodiments, the first media content M1B is loaded from media servers 103A, 103B, and 103C. The loaded first media content M1B may be different from the originally displayed first media content M1 (step 200). For example, one or more volatile media objects of the first media content M1 as originally displayed in step 200 may no longer be accessible when the first media content M1B is loaded.

In step 270, the software application 106 is configured to generate a second media content M2 that at least partially plays the first media content M1. In one or more embodiments, the generation of the second media content M2 is performed based on navigation context data. In one or more embodiments, the second media content includes a set of one or more media objects of interest identified in step 240. In one or more embodiments, the second media content includes a set of one or more volatile media objects identified in step 215.

In one or more embodiments, the second media content M2 is the one identified in step 215 based on the resource access data of the set of one or more volatile media object(s) of the first media content M1. It is created by modifying the loaded first media content M1B to restore the above set of volatile media objects. In some cases, for example, when the set of volatile media objects affects, is not modified or altered, the loaded first media content M1B compared to the originally displayed first media content M1, the volatile media Restoration of some of the objects may not be necessary (step 200). In one or more embodiments, the restoration is performed for one or more volatile media objects not present in the loaded first media content M1B or one or more volatile media objects for which one or more modifications occur. The modification may be, for example, a modification to a volatile media object or a modification of the order in which volatile media objects are presented.

In one or more embodiments, the interaction context is restored for the set of one or more media objects in the second media content based on triggering data to trigger execution of the interaction function(s) associated with the interactive media objects.

In one or more embodiments, the creation of the second media content M2 may cause the interaction function(s) associated with the interactive media objects to be triggered in the same manner as in the first media content M1 and the software application 106 ) Is configured to trigger an interaction function associated with the interactive media object based on the triggering data used to generate the second media content M2. For example, after the user is redirected to the target web page by clicking a media object in the first media content M1, and then clicking the corresponding media object in the second media content M2, the user can also go to the same target web page. Is redirected. For example, a menu is opened by clicking a media object in the first media content M1, and then the same menu will be opened in the second media content M2 by clicking a corresponding media object in the second media content M2. . Accordingly, the interaction context can be restored even for volatile media objects.

In step 275, the software application 106 creates a second media content (M2) that is inaccessible from the recipient electronic device 104B and/or to the user 101B whose navigation context is restored. ). In one or more embodiments, the software application 106 is configured to determine whether a media object in the set of one or more media objects is accessible to the user 101B. This may depend on the user rights/profile of the user 101B compared to the user rights/profile of the user 101A for the software application (eg, web application) to which the first media content was accessed. This may relate to media content that is not publicly available or to media content that is personal to the user 101A. Creation of the second media content M2 means that the media object that is not accessible to the user 101B of the software application 106 for which the navigation context is restored is the second media content (e.g., for non-volatile media content). It is performed so as not to be removed from (M2) or inserted into the second media content (M2) (for example, for a volatile media object not present in the loaded first media content).

In step 280, the software application 106 is configured to identify a portion of interest (hereinafter, a second portion) in the second media content M2 to be displayed. In one or more embodiments, the second portion is identified based on the set of one or more media objects of interest (e.g., for one or more media objects of interest identified in step 240). Contains at least one of the set of media objects of interest

In one or more embodiments, the coordinates of the second portion for the second media content M2 are determined based on the received object coordinates for the one or more objects of interest. In one or more embodiments, the coordinates of the second portion for the second media content M2 are, for a given object of interest, the object coordinates of the media object of interest, for the second portion of the second media content M2 Are determined to be the same (or as close as possible) to the object coordinates of the same media object of interest, for the first portion of the first media content M1. In one or more embodiments, the coordinates of the second portion are further determined according to the resolution and/or size of the display screen of the destination electronic device 104B. In one or more embodiments, the coordinates of the second portion are such that the media object(s) of interest is located within the second portion even when the layout of the second media content M2 is different from the layout of the first media content M1 And it is determined to ensure that it is positioned at a position relative to the display window as similar as possible to the layout of the first media content M1.

In one or more embodiments, the generation of the second media content M2 is compared to the resolution and/or size of the display screen of the source electronic device 104A and/or the resolution and/or of the display screen of the destination electronic device 104B. Depending on the size, the layout of the second media content M2 is performed so that the layout of the first media content M1 is the same as or as much as possible. If the display screens have the same resolution and/or size, the same layout may be used.

In step 290, the software application 106 is configured to display a second portion of the second media content M2 in a second UI window. In one or more embodiments, the display of the second portion of the second media content M2 is performed upon detection of an interaction with the user 101B in the second UI window while the screenshot is being displayed (see step 265). ). When the second part is displayed, the user 101B can trigger the interaction function(s) for the interactive object(s) currently displayed in the second part, or the second media content M2 to change the currently displayed part. You can explore.

Steps 260 to 290 may be repeated so that each navigation context is restored based on the navigation context data.

In one or more embodiments, the method for storing described with reference to FIG. 2A and the method for restoring the navigation context described with reference to FIG. 2B may be implemented by the same software application (e.g., software application 106). Can, but still means transmitting the navigation context metadata to a storage device (e.g., computing server 103E) (optionally of the media content itself). Thus, the methods can be used to remotely store media content (eg, a web page or any hypertext document) on a storage device and, optionally, navigation context metadata for the media content itself.

3A-3B illustrate a flowchart of a method for storing and restoring a navigation context on the same electronic device (eg, electronic device 104A) according to one or more embodiments. Although the various steps in the flowchart are presented and described sequentially, those skilled in the art will appreciate that some or all of the steps may be executed in different orders, combined or omitted, and some or all of the steps may be executed in parallel.

The method steps may be implemented by an electronic device (eg, electronic device 104A) according to any embodiment disclosed herein. The method steps may be implemented by the client application 106 of the source electronic device 104A, for example.

The method may be implemented without implying the transfer of media content or metadata to a storage device (eg, computing server 103E). Thus, the method may be used to locally store media content (e.g., a web page, or any hypertext document) and, optionally, navigation context metadata for the media content itself on electronic device 104A. have.

The steps 300 to 390 described below are simplified due to the fact that the steps are the same and are performed on a single electronic device and correspond to steps 200 to 290 described with reference to FIGS. 2A to 2B, respectively. For example, the data need not be transferred to the storage device. For example, there is no need to consider the privacy of media objects. For the sake of simplicity, the embodiments and aspects already described with reference to Figs. 2A to 2B are applicable without being repeated herein.

3A, in step 300, a first media content M1 (eg, a web page) is displayed in a first UI window W1 of the software application 106 on the display screen of the source electronic device 104A. ) Is displayed, and the software application 106 is configured to begin the process of storing the navigation context. The process may be initiated, for example, when performing a given interaction, at the request of the user 101A, or when some predetermined condition is met. In step 301, the software application 106 is configured to store media access data that enables access to the first media content M1. The media access data may include a hyperlink (e.g., a URL) for accessing a web page, a shortcut, an address, and/or an identifier for accessing the media content through the media servers 103A, 103B, and 103C. have.

In step 305, the software application 106 is configured to take a screenshot of the first portion of the first media content M1 currently displayed in the UI window W1.

In step 306, the software application 106 is configured to store the screenshot.

In step 310, the software application 106 is configured to identify a set of one or more media objects in the first media content M1.

In step 315, the software application 106 is configured to identify a set of one or more volatile media objects in the set of one or more media objects obtained in step 310. Thus, the identification of media objects may be suitable for the type of web site or type of web application or may be web site specific. In one or more embodiments, all media objects on the page may be identified as volatile media objects.

In step 316, the software application 106 provides, for a set of one or more volatile media objects, a resource for accessing resources that enable restoration (e.g., playback) of the set of one or more volatile media objects. It is configured to store access data.

At step 320, the software application 106 is configured to identify a set of one or more interactive media objects associated with one or more interactive functions in the set of one or more volatile media objects.

At step 321, the software application 106 is configured to store, for each identified interactive media object, triggering data for triggering execution of one or more interaction functions associated with the identified interactive media object.

In step 330, the software application 106 is configured to store the coordinates of the first portion in the first media content M1 currently displayed in the first UI (user interface) window.

In step 340, the software application 106 is a set of one or more media objects of interest in a first portion of the first media content M1 currently displayed in a first UI window of the software application in the set of one or more media objects. Is configured to identify.

In step 341, the software application 106 is configured to store an identification of the media object(s) of interest identified in step 340.

In step 350, the software application 106 calculates the object coordinates of the object of interest for the first portion (region of interest) of the first media content M1 currently displayed in the first UI window for each object of interest. Is configured to decide.

In step 351, the software application 106 is configured to store object coordinates of the object of interest for each object of interest.

Steps 300 to 351 may be repeated such that each navigation context is stored, for example, for another portion of the first media content or for another portion of another media content.

For each execution of steps 300 to 351, the navigation context data is stored by the source electronic device 104A, and the corresponding navigation context may be restored based on the stored navigation context data.

Thus, the stored navigation context data may include:

Media access data for accessing the first media content M1 through the media server; And/or

A screenshot of the first portion of the first media content M1; And/or

Coordinates of the first part for the first media content M1; And/or

Identification of media object(s) in the first media content; And/or

Identification of media object(s) of interest within the first portion; And/or

Identification of volatile media object(s) within the first media content M1; And/or

Resource access data for restoring a set of one or more volatile media object(s) of the first media content M1; And/or

Triggering data for triggering execution of the interaction function(s) associated with the interactive media objects; And/or

Object coordinates of each media object of interest for the first portion of the first media content.

3B, in step 360, the software application 106 retrieves the navigation context data (e.g., retrieves from a database, data file, etc.) and initiates a navigation context restoration based on the retrieved navigation context data. .

In step 365, the software application 106 is configured to display the screenshot in the second UI window. In one or more embodiments, the display of the screenshot is performed prior to displaying the second portion of the second media content M2 in the second UI window (see step 390). In one or more embodiments, the screenshot may be used as a visual representation of a bookmark corresponding to the navigation context data. In one or more embodiments, one or more screenshots are displayed, and a navigation context corresponding to the screenshot is restored based on the associated navigation context data and upon detection of the user's interaction with the associated screenshot.

In step 369, the software application 106 is configured to load the first media content using the media access data. In one or more embodiments, the first media content M1B is loaded from media servers 103A, 103B, and 103C. The loaded first media content M1B may be different from the originally displayed first media content M1 (step 200).

In step 370, the software application 106 is configured to generate a second media content M2 that at least partially plays the first media content M1. In one or more embodiments, the second media content includes a set of one or more media objects of interest identified in step 340. In one or more embodiments, the second media content includes a set of one or more volatile media objects identified in step 315.

In one or more embodiments, a first loaded to restore the set of one or more volatile media objects identified in step 315 based on resource access data each associated with the volatile media object(s) of the first media content M1. The second media content M2 is generated by modifying the first media content M1B. In some cases, for example, when the set of volatile media objects does not affect, modify or change the loaded first media content M1B compared to the originally displayed first media content M1 , It may not be necessary to restore some of the volatile media objects (step 300). In one or more embodiments, the restoration is performed for one or more volatile media objects not present in the loaded first media content M1B or one or more volatile media objects for which one or more modifications occur. The modification may be, for example, a modification to a volatile media object or a modification of the order in which the volatile media objects are presented.

In one or more embodiments, the creation of the second media content M2 may cause the interaction function(s) associated with the interactive media objects to be triggered in the same manner as in the first media content M1 and the software application 106 ) Is configured to trigger an interaction function associated with the interactive media object based on the triggering data used to generate the second media content M2.

In one or more embodiments, the interaction context is restored for the set of one or more media objects in the second media content based on triggering data to trigger execution of the interaction function(s) associated with the interactive media objects. Even in the case of interactive volatile objects, interactivity may be restored.

In step 380, the software application 106 is configured to identify a portion of interest (hereinafter, a second portion) in the second media content M2 to be displayed. In one or more embodiments, the second portion is identified based on the set of one or more media objects of interest and one or more media objects of interest (e.g., one or more media objects of interest identified in step 340). ), including at least one media object of interest.

In one or more embodiments, the coordinates of the second portion for the second media content M2 are determined based on the received object coordinates for the one or more objects of interest.

In one or more embodiments, the coordinates of the second portion for the second media content M2 are, for a given object of interest, the object coordinates for the second portion of the second media content M2 of the media object of interest. Are determined to be the same (or as close as possible) to the object coordinates for the first portion of the first media content M1 of the same media object of interest. In step 390, the software application 106 is configured to display a second portion of the second media content M2 in a second UI window. In one or more embodiments, the display of the second portion of the second media content M2 is performed upon detection of an interaction with the user 101B in the second UI window while the screenshot is being displayed (see step 365). ). Once the second portion is displayed, the user 101B can trigger the interaction function(s) on the interactive object(s) currently displayed in the second portion or to change the currently displayed portion of the second media content (M2). ), you can search.

Steps 360 to 390 may be repeated so that each navigation context is restored based on the navigation context data.

4A-4C and 5A-5C illustrate media content and user interfaces in accordance with one or more embodiments.

4A shows a first picture (P1) and associated text (T1), a second picture (P2) and associated texts (T21 and T22), a third picture (P3) and associated text (T3), and a fourth picture ( P4) and a first media content M1 (e.g., a web page) comprising several media objects such as associated text T4.

4B shows a first portion of the first media content M1 of FIG. 4A displayed in a window W1 of the software application on the display screen of the source electronic device 104A. In the first part, only the third picture (P3), text (T3), fourth picture (P4) and text (T4), text (T22) are visible and other media objects (P1, T1, P2, and T21) Is invisible or is only partially visible.

FIG. 4C shows that when a picture P4 is selected by the user 101A of the source electronic device 104A, a media object is identified as a media object of interest, and the media object of interest is the picture P4 and associated It contains at least a part of the text T4. In the illustrated example, the object coordinates of the media object of interest are determined to be at a height Y=75% from the top of the window W1. In the illustrated example, only the vertical position of the media object of interest is considered, but both the vertical and horizontal position relative to the window W1 may be considered.

FIG. 5A shows a screen shot SC0 of a first portion of the first media content M1 displayed in the window W1 shown in FIG. 4B. In the screenshot SC0, the third picture P3, the text T3, the fourth picture P4, and the text T4 are visible.

FIG. 5B shows a screenshot SC0 of FIG. 5A displayed in a window W2 of the software application on the display screen of the destination electronic device 104B (see, eg, steps 265 or 365). As shown in FIG. 5B, due to the difference between the size and resolution of the display screen of the source electronic device 104A and the display screen of the destination electronic device 104B, the displayed screenshot may not completely cover the window W2. And/or may not fit the display ratio of the display screen of the destination electronic device 104B.

5C illustrates contents of the second window W2 when the second media contents are generated based on the first media contents. Only the portion of the second media content containing the media object of interest (ie, the portion of the picture P4 and associated text T4) is visible. In the illustrated example, the object coordinates of the media object of interest for the portion displayed on the second window W2 are the same as in FIG. 4C. In the illustrated example, the vertical position of the media object of interest is concentrated at a height Y=75% from the top of the window W2. Because the size of the display screen of the recipient electronic device 104B is larger than the display screen of the source electronic device 104B, additional media objects (part of the picture P2 in the example of FIG. 5C) are It may be visible in comparison with media objects visible in the UI window W1.

6A-6C and 7A-7C illustrate media content and user interfaces in accordance with one or more embodiments.

6A shows a first media content M1 (e.g., a web page) including a list of media objects O1 to O7, each media object including a picture and accompanying short text Group (or container). The list of media objects O1 to O7 are results of a search performed by a search engine based on a search criterion input by the user 101A of the source electronic device 104A. The result of such a search may vary depending on the time and/or personalization parameters (e.g., user profile and/or user cookie and/or user location) at which the search is performed, and the media objects O1 to O7 The list is considered to be a set of volatile media objects and the associated resource access data is stored to enable restoration of this list of media objects O1 to O7 in the same order.

6B shows a first portion of the first media content M1 of FIG. 6A displayed in a window W1 of the software application on the display screen of the source electronic device 104A. In the first part, only media objects O2 to O6 are visible and other media objects O1 and O7 are invisible.

6C shows that when the media object O4 is selected by the user 101A of the source electronic device 104A, the volatile media object O4 is identified as a media object of interest. In the illustrated example, the object coordinates of the media object of interest are determined to be at a height Y=50% from the top of the window W1. In the illustrated example, only the vertical position of the media object of interest is considered, but both vertical and horizontal positions for the window W1 may be considered. In the illustrated example, the volatile media object O4 includes a picture VP and a short text ST.

7A shows a screen shot SC1 of a first portion of the first media content M1 displayed in the window W1 shown in FIG. 6B. Media objects O2 to O6 are visible in the screenshot SC1.

7B shows a screenshot SC1 of FIG. 7A displayed in a window W2 of the software application on the display screen of the destination electronic device 104B (see, for example, steps 265 or 365). As shown in FIG. 7B, due to the difference between the size and resolution of the display screen of the source electronic device 104A and the display screen of the destination electronic device 104B, the displayed screenshot may not completely cover the window W2. And/or may not fit the display ratio of the display screen of the destination electronic device 104B.

7C illustrates contents of the second window W2 when the second media content M2 is generated based on the first media content M1. Even if the search is triggered based on the same search criteria, the list of media objects O1 to O7 is played by restoring the correct order of the media objects O1 to O7 corresponding to the initial contents of the first media content (see FIG. This will result in a different list of media objects when a search is requested by user 101B on destination electronic device 104B.

As shown in FIG. 7C, only a portion of the second media content is visible in the second window W2, and this portion includes a media object O4 of interest. In the illustrated example, the object coordinates of the media object of interest for the portion displayed in the second window are the same as in FIG. 6C. In the illustrated example, the vertical position of the media object of interest is concentrated at a height Y=50% from the top of the window W2. Since the size of the display screen of the recipient electronic device 104B is larger than the display screen of the source electronic device 104B, the additional media objects are visible compared to the media objects visible in the UI window W1 of the electronic device 104A. Can be

FIG. 7D shows the contents of the second window W2 when an interaction of the second media content M2 on the media object O4 of interest is detected in the window W2 of FIG. 7C. When an interaction on the media object O4 is detected, one or more interaction functions may be triggered. In the illustrated example, the triggered function is a redirection to the target web page (eg, based on a hyperlink included in the triggering data) and the target web page is displayed in window W2. The target web page may include additional information associated with the media object O4 of interest, for example, a picture (VP) and full text (FT). Thus, the interaction context is restored and the user can continue browsing from these target web pages.

8A-8C illustrate additional user interfaces in accordance with one or more embodiments. FIG. 8A shows a screenshot SC2 of a first portion of the first media content M1 displayed in the window W1 shown in FIG. 6B, wherein some of the media objects are media objects of interest (Example For example, it is distinguished by blurring (automatically) at the request of the user 101A or upon identification of the media object O4 of interest. Thus, the media objects O2 to O3 and O5 to O6 are not readable in the screenshot SC2. In the screenshot SC2, the media object O4 includes a picture VP and a summary text ST.

FIG. 8B shows a screenshot SC2 of FIG. 7A displayed in a window W2 of the software application on the display screen of the destination electronic device 104B (see, eg, steps 265 or 365). As shown in FIG. 7B, due to the difference between the size and resolution of the display screen of the source electronic device 104A and the display screen of the destination electronic device 104B, the displayed screenshot may not completely cover the window W2. And/or may not fit the display ratio of the display screen of the destination electronic device 104B.

FIG. 8B shows contents of the second window W2 when the second media content M2 is generated based on the first media content M1. The list of media objects O1 to O7 is not played. Instead, only the media object O4 of interest is played and other media objects are removed.

As shown in FIG. 8C, since only a single media object O4 in the list of media objects should be restored from the second media content, a larger view of the media object O4 can be provided. For example, if the media object O4 contains a hyperlink to a target web page containing more information associated with the media object O4, providing a larger view to the media object O4 is the target web It may include displaying the page. In the illustrated example, a larger view of the reconstructed media object O4 includes a picture VP and full text FT in place of the short text ST previously visible in FIGS. 6A-6C.

Embodiments described herein are a web application or an application for navigation to media content (e.g., a map application that displays an interactive map of a city where the media object of interest may be a location such as a city and/or a subway station, etc.) ), it can be applied to restore navigation context in any kind of software.

Claims (17)

As a method of restoring the navigation context of the software,
Saving the navigation context,
Storing (301) media access data to enable access to the first media content displayed in the software;
Detecting (310) a set of one or more media objects in the first media content;
Identifying (340), in the set of one or more media objects, one or more sets of media objects of interest in a first user interface, a UI, a first portion of the first media content currently visible in a window; And
Storing the identification of the one or more media objects of interest (341); storing the navigation context, and
Restoring the navigation context,
Loading (369) the first media content based on the media access data;
Generating (370) second media content based on the loaded first media content, wherein the second media content is one or more reconstructed media objects of interest corresponding to the identified one or more media objects of interest Generating (370) the second media content comprising a set of; And
Displaying (390) the second media content so that the second portion currently visible in the second UI window includes at least one restored media object of interest from among the set of the one or more restored media objects of interest; Comprising restoring the navigation context. The method of claim 1,
The step of storing the navigation context,
Identifying a set of one or more volatile media objects in the set of one or more media objects; And
Storing resource access data for accessing resources for restoring the set of one or more volatile media objects; and
Generating the second media content,
And at least partially modifying the loaded first media content by restoring at least one volatile media object of the set of one or more volatile media objects from the second media content based on the resource access data. . The method according to claim 1 or 2,
The step of storing the navigation context,
Identifying a set of one or more interactive media objects associated with one or more interaction functions in the set of one or more volatile media objects; And
For each identified interactive media object, storing triggering data for triggering execution of the one or more interaction functions associated with the identified interactive media object; and further comprising,
The second media content includes at least one interactive media object from a set of one or more interactive media objects,
Restoring the navigation context may include, for each interactive media object in the second media content,
Enabling a user to trigger the one or more interaction functions associated with the considered interactive media object; And
Triggering the one or more interaction functions associated with the media object of interest considered based on the triggering data. The method of claim 3,
Wherein at least one media object of interest in the set of one or more media objects of interest is an interactive media object in the set of one or more interactive media objects. The method according to any one of claims 1 to 4,
For at least one object of interest,
The step of storing the navigation context,
The step of storing object coordinates of an object of interest in a first portion of the first media content currently visible in the first UI window,
Restoring the navigation context includes:
Further comprising determining second coordinates of a second portion in second media content based on the object coordinates. The method according to any one of claims 1 to 5,
The step of storing the navigation context,
And taking a screenshot of a first portion of the first media content,
Restoring the navigation context includes:
And displaying the screenshot on the second UI window,
Displaying the screenshot is performed before displaying the second portion of the second media content in the second UI window,
The displaying of the second portion of the second media content in the second UI window of the software is performed upon detection of a user's interaction in the second UI window. The method according to any one of claims 1 to 6,
The step of identifying one or more media objects of interest in the first part of the first media content may include detecting an interaction of a user on at least one media object in the first part, and the at least one media in which the interaction is detected Identifying the object. The method according to any one of claims 1 to 7,
The step of identifying one or more media objects of interest in the first portion of the first media content may include determining a viewing direction of a user viewing the first media content, and a display area crossing the viewing direction or displaying Identifying one or more media objects of interest whose region is closest to the intersection of the display screen and the viewing direction. The method according to any one of claims 1 to 8,
Wherein identifying one or more media objects of interest in the first portion of the first media content comprises determining one or more media objects closest to a central location of the first portion of the first media content. The method according to any one of claims 1 to 9,
The step of identifying one or more media objects of interest in the first portion of the first media content includes allowing a user to select one or more media objects of interest, and identifying the selected one or more media objects of interest. Containing, method. The method according to any one of claims 1 to 10,
The step of generating the second media content comprises: determining whether a media object of the set of one or more media objects is accessible to a user for which the navigation context is restored, and only if the considered media object is accessible Inserting a media object into the second media content. The method according to any one of claims 1 to 11,
The layout of the second media content is different from the layout of the first media content and is adapted to a screen resolution and/or a screen size of a display screen. As a method of restoring the navigation context of the software,
Acquiring the navigation context data (260; 360), the navigation context data,
Media access data enabling access to a first media content comprising a set of one or more media objects;
Obtaining (260) the navigation context data, including, during navigation, identification of a set of one or more media objects of interest displayed in a first portion of the first media content in a first user interface, UI, window; 360); And
Restoring the navigation context,
Loading (269; 369) first media content based on the media access data;
Generating (270; 370) second media content based on the loaded first media content, wherein the second media content is one or more reconstructed media of interest corresponding to the identified one or more media objects of interest. Generating (270; 370) second media content, comprising a set of objects; And
Displaying (290; 390) the second media content so that the second portion currently visible in the second UI window includes at least one restored media object of interest from among a set of one or more restored media objects of interest; Including, restoring the navigation context; Including, method. The method of claim 13,
The first media content comprises a set of one or more volatile media objects,
The navigation context data further includes resource access data for accessing resources for restoring a set of one or more volatile media objects identified in the set of one or more media objects,
Generating the second media content,
And at least partially modifying the loaded first media content by restoring at least one volatile media object of the set of one or more volatile media objects from the second media content based on the resource access data. . As a method to restore the navigation context of the software,
Transmitting (201) media access data to a recipient device enabling access to the first media content displayed on the software;
Detecting (210) a set of one or more media objects in the first media content;
In the set of one or more media objects, identifying (240) a set of one or more media objects of interest in a first user interface, a UI, a first portion of the first media content currently visible in a window;
Transmitting (241) the identification of the one or more media objects of interest to the recipient device; And
Including; restoring the navigation context on the receiver device; and
The step of restoring the navigation text,
Loading (269) first media content based on the media access data;
Generating (270) second media content based on the loaded first media content, wherein the second media content comprises a set of one or more reconstructed media objects corresponding to the identified one or more media objects of interest. Generating (270) the second media content comprising; And
Displaying (290) the second media content so that the second portion currently visible in the second UI window includes at least one restored object of interest from among the set of one or more restored media objects of interest (290); How to. Processor; And
A memory operatively coupled to the processor; and
The electronic device comprising computer-readable instructions that, when executed by the processor, cause the processor to perform the steps of the method according to any one of claims 1 to 15. A computer program product comprising computer readable instructions that, when executed by a computer, cause the computer to perform the steps of the method according to claim 1.

Images